Programming one DTMC transition

We now combine the ideas from the previous lessons. Starting from one SIR state, the program calculates valid probabilities, draws one uniform random number, selects one event and returns exactly one next state.

The unique purpose of this lesson

A DTMC transition is one movement from the current state \(X_n\) to the next state \(X_{n+1}\) during one fixed time step.

This lesson performs the complete mechanism once. It does not use a loop, store a trajectory or simulate an entire epidemic. Those are the new ideas in Lesson 6.

Scenario: one step of an SIR epidemic

Consider a closed population of 1,000 people at time step \(n\):

\[ X_n=(S_n,I_n,R_n)=(990,10,0). \]

Use \(\beta=0.3\) per day, \(\gamma=0.1\) per day and \(\Delta t=0.1\) day. As derived in Lesson 3:

\[ p_{\mathrm{inf}}=0.297,\qquad p_{\mathrm{rec}}=0.100,\qquad p_{\mathrm{stay}}=0.603. \]

Only one of the three events may be applied during this step.

The one-transition algorithm

1. Current state\((S,I,R)\)

→

2. Random selectionprobabilities + one \(U\)

→

3. Next state\((S',I',R')\)

Read the current state and parameter values.
Calculate infection, recovery and no-change probabilities.
Check that the probabilities are valid.
Generate one uniform random number \(U\).
Use the cumulative intervals to select one event.
Apply only the compartment changes belonging to that event.
Check that the next state is biologically valid.

How each event changes the state

Infection

\[(S,I,R)\longrightarrow(S-1,I+1,R).\]

One susceptible person becomes infectious. The total population is unchanged.

Recovery

\[(S,I,R)\longrightarrow(S,I-1,R+1).\]

One infectious person moves to the removed compartment.

No change

\[(S,I,R)\longrightarrow(S,I,R).\]

No infection or recovery occurs during this time step.

The random number chooses the event; the event determines the state change. We must not apply the probabilities themselves as fractional changes to the compartment counts.

From interval selection to state updating

if u < p_infection:
    event = "infection"
    next_S = S - 1
    next_I = I + 1
    next_R = R
elif u < p_infection + p_recovery:
    event = "recovery"
    next_S = S
    next_I = I - 1
    next_R = R + 1
else:
    event = "no change"
    next_S = S
    next_I = I
    next_R = R

Each branch assigns all three next-state variables. Therefore, after the conditional block, next_S, next_I and next_R always describe one complete state.

Why use new variable names?

The names S, I and R retain the current state. The names next_S, next_I and next_R hold the next state.

This separation makes the transition easy to inspect:

\[ \underbrace{(S,I,R)}_{\text{before the step}} \quad\longrightarrow\quad \underbrace{(S',I',R')}_{\text{after the step}}. \]

It also prevents accidental partial updating. For example, changing S before calculating every quantity could cause later calculations to mix the old and new states.

Conditions for a valid transition

Valid probabilitiesEach probability is between 0 and 1, and their sum is 1.

Non-negative countsThe next values of \(S\), \(I\) and \(R\) cannot be negative.

Population conservation\(S'+I'+R'=S+I+R=N\).

The biological rates automatically give zero probability to impossible events: if \(S=0\), infection probability is zero; if \(I=0\), both infection and recovery probabilities are zero.

Interactive Python laboratory

Click Run code. The program will perform one transition and display the random number, probability intervals, selected event, before-and-after table and graph.

Interactive PythonOne SIR DTMC transition

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

def one_sir_transition(S, I, R, beta, gamma, dt, rng):
    # Check the supplied state and parameters
    if min(S, I, R) < 0:
        raise ValueError("Compartment counts cannot be negative.")

if beta < 0 or gamma < 0 or dt <= 0:
        raise ValueError("beta and gamma must be non-negative; dt must be positive.")

# Population size before the transition
    N = S + I + R

if N == 0:
        raise ValueError("The population must be greater than zero.")

# Transition probabilities from the current state
    p_infection = beta * S * I / N * dt
    p_recovery = gamma * I * dt
    p_no_change = 1 - p_infection - p_recovery

# A negative no-change probability means dt is too large
    if p_no_change < -1e-12:
        raise ValueError("Time step is too large: probabilities exceed 1.")

# Remove a tiny negative value caused only by rounding
    p_no_change = max(0.0, p_no_change)

# One random number selects one event
    u = rng.random()

if u < p_infection:
        event = "infection"
        next_S = S - 1
        next_I = I + 1
        next_R = R
    elif u < p_infection + p_recovery:
        event = "recovery"
        next_S = S
        next_I = I - 1
        next_R = R + 1
    else:
        event = "no change"
        next_S = S
        next_I = I
        next_R = R

# Validate the next state
    if min(next_S, next_I, next_R) < 0:
        raise ValueError("A compartment became negative.")

if next_S + next_I + next_R != N:
        raise ValueError("Population was not conserved.")

return {
        "event": event,
        "u": u,
        "probabilities": (p_infection, p_recovery, p_no_change),
        "next_state": (next_S, next_I, next_R)
    }

# Current state and parameters
S, I, R = 990, 10, 0
beta = 0.3
gamma = 0.1
dt = 0.1

# Change the seed to obtain another valid transition
seed = 7
rng = np.random.default_rng(seed)

result = one_sir_transition(S, I, R, beta, gamma, dt, rng)

p_infection, p_recovery, p_no_change = result["probabilities"]
next_S, next_I, next_R = result["next_state"]

print(f"Random number u: {result['u']:.6f}")
print(f"Selected event: {result['event']}")
print()
print("Probability intervals")
print(f"Infection: [0, {p_infection:.3f})")
print(f"Recovery:  [{p_infection:.3f}, "
      f"{p_infection + p_recovery:.3f})")
print(f"No change: [{p_infection + p_recovery:.3f}, 1)")
print()

state_table = pd.DataFrame({
    "compartment": ["Susceptible", "Infectious", "Removed", "Total"],
    "before": [S, I, R, S + I + R],
    "after": [
        next_S, next_I, next_R,
        next_S + next_I + next_R
    ],
    "change": [
        next_S - S,
        next_I - I,
        next_R - R,
        (next_S + next_I + next_R) - (S + I + R)
    ]
})

print("State transition")
print(state_table.to_string(index=False))

labels = ["Susceptible", "Infectious", "Removed"]
changes = [next_S - S, next_I - I, next_R - R]
colours = ["#2563eb" if value == 0 else
           "#dc2626" if value < 0 else "#16a34a"
           for value in changes]

plt.figure(figsize=(8, 4.6))
bars = plt.bar(labels, changes, color=colours)
plt.axhline(0, color="black", linewidth=1)
plt.bar_label(bars, fmt="%+g")
plt.ylabel("Change during one step")
plt.ylim(-1.4, 1.4)
plt.title(f"Compartment changes: {result['event']}")
plt.grid(axis="y", alpha=0.25)
plt.tight_layout()

Output

Run the code to see the result.

Read the result correctly

With seed 7, the generator produces one particular random number. The interval containing that number determines the event. The table then shows how that event changes the state.

If no change is selected, identical before-and-after states are a genuine stochastic outcome—not evidence that the program failed.

One transition is one possible realisation. It does not represent the average epidemic behaviour and cannot tell us what will happen in every simulation.

Understand the function

Code	Purpose
`def one_sir_transition(...):`	Packages the entire one-step rule into a reusable function.
`rng`	Receives the random-number generator instead of creating a new generator inside every call.
`raise ValueError(...)`	Stops the calculation and reports why the supplied model inputs cannot produce a valid transition.
`max(0.0, p_no_change)`	Replaces only a tiny negative round-off value with zero after genuinely invalid values have already been rejected.
`min(next_S, next_I, next_R)`	Finds the smallest next-state count. If it is negative, at least one compartment is invalid.
`return {...}`	Returns a dictionary containing the event, random number, probabilities and next state.
`result["next_state"]`	Retrieves the next-state tuple from the returned dictionary.

Why compare with `-1e-12` rather than exactly zero?

Decimal quantities are stored approximately by a computer. A calculation that should mathematically give zero may occasionally produce a tiny value such as \(-10^{-16}\).

if p_no_change < -1e-12:
    raise ValueError(...)

Here 1e-12 means \(10^{-12}\). A value below \(-10^{-12}\) is treated as a real probability error. A much smaller negative value is treated as harmless numerical round-off and replaced by zero.

This is different from a direct input check such as if dt <= 0. The value of dt is supplied explicitly, so zero and negative values can be rejected directly. The no-change probability is produced by floating-point arithmetic and may contain tiny rounding error.

Boundary state: no infectious people

If \(I=0\), then:

\[ p_{\mathrm{infection}}=0,\qquad p_{\mathrm{recovery}}=0,\qquad p_{\mathrm{stay}}=1. \]

Every possible random number therefore selects no change. The disease-free state is absorbing in this closed SIR model: without an infectious person or imported infection, transmission cannot restart.

Try these experiments

Change seed = 7 to several other integers. Observe different valid one-step outcomes.
Use S, I, R = 990, 0, 10. Confirm that no change is always selected.
Set dt = 1.0. Read the error message and explain why the probabilities are invalid for the current state.
Set seed = 2, then temporarily replace the infection update with next_S = S. The selected infection will make the population-conservation check detect the mistake.
Use a state with more infectious people and examine how the infection interval changes.

Biological interpretation

During this one modelled time step, the population can experience one infection, one recovery or no recorded event. The selected transition respects the direction of the biological flows: susceptible people can become infectious, and infectious people can recover.

The next state remains an integer-valued population state. Unlike a deterministic Euler step, the program does not move fractional people according to average rates.

What this lesson has—and has not—done

You can now:

combine a current state, transition probabilities and one random draw;
apply infection, recovery or no-change updates;
return a complete next state from a Python function;
check non-negativity and population conservation; and
interpret one transition as one stochastic realisation.

Not yet: the function has been called only once. Lesson 6 will introduce a loop that repeatedly feeds each next state into the following step, stores the results and constructs a complete epidemic trajectory.

Programming one DTMC transition

The unique purpose of this lesson

Scenario: one step of an SIR epidemic

The one-transition algorithm

How each event changes the state

Infection

Recovery

No change

From interval selection to state updating

Why use new variable names?

Conditions for a valid transition

Interactive Python laboratory

Read the result correctly

Understand the function

Why compare with -1e-12 rather than exactly zero?

Boundary state: no infectious people

Try these experiments

Biological interpretation

What this lesson has—and has not—done

Why compare with `-1e-12` rather than exactly zero?