States, fixed time steps and random transitions

A discrete-time Markov chain records a biological system at fixed times. Its next state is uncertain, so repeated epidemics starting from the same state need not follow the same path.

Biological scenario

A closed group contains five people. At the end of each day, we record how many are susceptible, infectious and recovered.

On day 0 the state is four susceptible, one infectious and no recovered people. We want to understand how this information is represented before calculating transition probabilities or generating random events.

What is a state?

A state contains the information used to describe the system at one time. For this small SIR model,

\[X_n=(S_n,I_n,R_n).\]

\(S_n\)susceptible count at step \(n\)

\(I_n\)infectious count at step \(n\)

\(R_n\)recovered count at step \(n\)

The initial state is

\[X_0=(4,1,0).\]

The subscript \(n\) labels the discrete step. It does not mean multiplication.

State space

The state space is the collection of all states allowed by the model. In a closed population of five, a valid state must satisfy

\[S_n\geq0,\qquad I_n\geq0,\qquad R_n\geq0,\qquad S_n+I_n+R_n=5.\]

Examples include \((4,1,0)\), \((3,2,0)\) and \((3,1,1)\). A state such as \((4,2,0)\) is invalid because its total is six.

Fixed time steps

The process is observed at

\[t_n=t_0+n\Delta t,\]

where \(\Delta t\) is fixed. Here \(\Delta t=1\) day.

\(t_0=0\)

\(t_1=1\)

\(t_2=2\)

\(t_3=3\)

\(t_4=4\)

\(t_5=5\)

The DTMC records a state at these discrete times. It does not specify the exact time within a day at which a biological event occurred.

What is a random transition?

A transition is movement from the current state \(X_n\) to the next state \(X_{n+1}\). It is random because more than one next state may be possible from the same current state.

Current state
\(X_n=(4,1,0)\)

→

one possible next state: \((3,2,0)\)

another possible next state: \((4,1,0)\)

another possible next state: \((4,0,1)\)

This lesson identifies possible next states only. Lesson 2 assigns transition probabilities, and later lessons use random numbers to select an outcome.

The Markov property

The Markov assumption states that, once the current state is known, the probability distribution of the next state does not require the complete earlier path:

\[\Pr(X_{n+1}=x\mid X_n,X_{n-1},\ldots,X_0)=\Pr(X_{n+1}=x\mid X_n).\]

This does not mean that the past had no biological effect. Its relevant effect is assumed to be summarised by the current state.

If infection risk also depends on how long each person has been infectious, then \((S,I,R)\) may not contain enough information for the Markov property. The state would need to be expanded or the assumption reconsidered.

One trajectory versus all possibilities

One realised trajectory

A single sequence that happened in one simulation or observation:

\[(4,1,0)\to(3,2,0)\to(3,2,0)\to(3,1,1).\]

Distribution of trajectories

The collection of possible paths and their probabilities across repeated realisations.

One path cannot show the complete uncertainty of the model.

Store a supplied trajectory in Python

We use a supplied path so that this lesson does not yet generate randomness.

States as tuples

states = [
    (4, 1, 0),
    (3, 2, 0),
    (3, 2, 0),
    (3, 1, 1),
    (2, 2, 1),
    (2, 1, 2)
]

A tuple is an ordered collection written with parentheses. Each tuple stores one complete \((S,I,R)\) state.

Unpack one state

S, I, R = states[0]

Tuple unpacking assigns the three entries to three names in the same order.

Run and inspect the trajectory

Interactive PythonSupplied DTMC trajectory

import numpy as np
import matplotlib.pyplot as plt

# Fixed observation settings
start_time = 0
dt = 1

# One supplied realised trajectory
states = [
    (4, 1, 0),
    (3, 2, 0),
    (3, 2, 0),
    (3, 1, 1),
    (2, 2, 1),
    (2, 1, 2)
]

# Create matching observation times
number_of_states = len(states)
time = start_time + dt * np.arange(number_of_states)

# Check every state
population = sum(states[0])

for state in states:
    S, I, R = state
    
    if min(S, I, R) < 0:
        raise ValueError("A state contains a negative count.")
    
    if S + I + R != population:
        raise ValueError("A state violates population conservation.")

# Display the trajectory
print("Step   Time   State (S, I, R)")
print("--------------------------------")

for step, state in enumerate(states):
    print(f"{step:4d}   {time[step]:4.1f}   {state}")

# Convert state tuples into arrays for plotting
state_array = np.array(states)
S_values = state_array[:, 0]
I_values = state_array[:, 1]
R_values = state_array[:, 2]

# Plot discrete-time states
plt.figure(figsize=(8, 4.8))
plt.step(
    time,
    S_values,
    where="post",
    marker="o",
    label="Susceptible"
)
plt.step(
    time,
    I_values,
    where="post",
    marker="o",
    label="Infectious"
)
plt.step(
    time,
    R_values,
    where="post",
    marker="o",
    label="Recovered"
)
plt.xlabel("Time (days)")
plt.ylabel("Number of people")
plt.title("One supplied discrete-time SIR trajectory")
plt.xticks(time)
plt.grid(alpha=0.25)
plt.legend()
plt.tight_layout()

Output

Run the code to see the result.

Understand the new code

Code	Meaning
`(4, 1, 0)`	A tuple representing one ordered SIR state.
`sum(states[0])`	Adds the entries of the initial state to obtain the population.
`for state in states`	Visits each stored state in sequence.
`state_array[:, 1]`	Selects every row and column 1, the infectious counts.
`plt.step(..., where="post")`	Draws a step plot that holds each recorded value until the next discrete time.
`plt.xticks(time)`	Places horizontal-axis tick marks at the recorded times.

DTMC time is not Euler time

DTMC step	Euler step
Part of the stochastic model definition.	A numerical approximation choice for solving differential equations.
The next state is random.	The next approximation is determined by the current numerical formula.
Changing the step can change the transition model itself.	Reducing the step usually aims to improve numerical accuracy for the same ODE.

Boundary of this lesson

We have not yet calculated transition probabilities, divided a random-number interval, or simulated a transition. The trajectory was supplied only to demonstrate states, fixed times and one realised path.

Try another supplied path

Replace the states with:

states = [
    (4, 1, 0),
    (4, 0, 1),
    (4, 0, 1),
    (4, 0, 1)
]

Run the code and inspect the step plot.
Identify when the infectious population reaches zero.
Explain why this is one possible trajectory, not the only possible path from \((4,1,0)\).