Transition matrices and probability vectors

A simulated trajectory selects one state at each time. The matrix approach instead calculates the probability of being in every possible state after one time step.

A change of viewpoint

Trajectory viewpoint

Start in one state, draw a random number and select one next state.

\[ I_0=1\longrightarrow I_1=2 \quad\text{in one realisation}. \]

Distribution viewpoint

Start with probabilities over the states and calculate all next-state probabilities simultaneously.

\[ \boldsymbol\pi_0 \longrightarrow \boldsymbol\pi_1. \]

A probability distribution assigns a probability to every possible state. Its entries are non-negative and sum to 1.

The matrix calculation uses no random number. For a finite DTMC with a known transition matrix, the one-step distribution update is exact apart from ordinary computer rounding.

A small SIS example

Use a closed population of \(N=3\). The state is the infectious count:

\[ \mathcal S=\{0,1,2,3\}. \]

Choose \(\beta=0.6\) per day, \(\gamma=0.2\) per day and \(\Delta t=0.5\) day. From state \(i\):

\[ p_i=\beta\frac{(N-i)i}{N}\Delta t, \qquad q_i=\gamma i\Delta t, \qquad r_i=1-p_i-q_i. \]

These are infection, recovery and no-change probabilities, as established in the SIS lesson.

Fix the matrix convention before calculating

This page uses a row-stochastic matrix:

\[ P_{ij} = \Pr(I_{n+1}=j\mid I_n=i). \]

Row \(i\) is the current state. Column \(j\) is the next state. Therefore every row must sum to 1.

Choose current stateRead across row \(i\)

→

Find next-state probabilitiesColumns \(j=0,1,2,3\)

The states are ordered \(0,1,2,3\). Row and column positions must use this same ordering.

Construct each row biologically

From an interior state \(i\), only three destinations are possible:

\[ P_{i,i+1}=p_i,\qquad P_{i,i-1}=q_i,\qquad P_{i,i}=r_i. \]

All other entries in that row are zero because a one-event step cannot jump over a neighbouring state.

Current state	Next 0	Next 1	Next 2	Next 3
Current 0	1.0	0	0	0
Current 1	0.1	0.7	0.2	0
Current 2	0	0.2	0.6	0.2
Current 3	0	0	0.3	0.7

Hence:

\[ P= \begin{pmatrix} 1&0&0&0\\ 0.1&0.7&0.2&0\\ 0&0.2&0.6&0.2\\ 0&0&0.3&0.7 \end{pmatrix}. \]

Interpret important rows

Row 0: \((1,0,0,0)\). Extinction is absorbing.
Row 1: recovery gives state 0, no change gives state 1, and infection gives state 2.
Row 2: one step can reach states 1, 2 or 3, but not state 0.
Row 3: infection is impossible because no susceptible people remain; recovery can reach state 2.

The many zero entries encode biologically impossible one-step movements. Because non-zero entries lie only on the main diagonal and its two neighbours, this birth–death transition matrix is tridiagonal.

The probability row vector

Using the same state order, define:

\[ \boldsymbol\pi_n= \big( \Pr(I_n=0), \Pr(I_n=1), \Pr(I_n=2), \Pr(I_n=3) \big). \]

This page writes \(\boldsymbol\pi_n\) as a row vector. If the process certainly begins in state 1:

\[ \boldsymbol\pi_0=(0,1,0,0). \]

The 1 means 100% probability at state 1. It does not mean the infectious count itself is stored in the vector.

Calculate one distribution update

With a row probability vector and row-stochastic matrix:

\[ \boldsymbol\pi_{n+1} = \boldsymbol\pi_nP. \]

For the certain initial state 1:

\[ (0,1,0,0) \begin{pmatrix} 1&0&0&0\\ 0.1&0.7&0.2&0\\ 0&0.2&0.6&0.2\\ 0&0&0.3&0.7 \end{pmatrix} = (0.1,0.7,0.2,0). \]

Because the initial vector places all probability on state 1, the next vector is simply row 1 of \(P\).

What matrix multiplication is doing

For each next state \(j\):

\[ \pi_{n+1,j} = \sum_{i=0}^{N} \pi_{n,i}P_{ij}. \]

The calculation adds contributions arriving at state \(j\) from every possible current state \(i\). Each contribution is:

\[ \underbrace{\pi_{n,i}}_{\text{probability currently at }i} \times \underbrace{P_{ij}}_{\text{probability of }i\to j}. \]

For a certain initial state, only one current-state contribution is non-zero. For a mixed initial distribution, several rows contribute.

Row versus column conventions

Probability representation	Update using this page's row-stochastic (P)
Row vector	\(\boldsymbol\pi_{n+1}=\boldsymbol\pi_nP\)
Column vector	\(\mathbf p_{n+1}=P^{\mathsf T}\mathbf p_n\)

Some books instead define columns as current states. Their transition matrix is the transpose of ours, and they write \(\mathbf p_{n+1}=P\mathbf p_n\). Both conventions are valid.

Never mix conventions. First state whether rows or columns represent the current state, then check the corresponding sums and use the matching multiplication order.

Three essential checks

ShapeWith four states, \(P\) must have shape \(4\times4\).

EntriesEvery matrix entry must lie between 0 and 1.

Row sumsEvery row must sum to 1 under our convention.

The probability vector must also contain four non-negative entries summing to 1.

Interactive Python laboratory

The code constructs \(P\) from the SIS formulas, validates it, performs exactly one distribution update and displays the matrix and vectors as tables and graphs.

Interactive PythonOne transition-matrix update

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Small SIS model
N = 3
beta = 0.6
gamma = 0.2
dt = 0.5

states = np.arange(N + 1)
number_of_states = len(states)

# P[current state, next state]
P = np.zeros((number_of_states, number_of_states))

for i in states:
    susceptible = N - i

p_infection = beta * susceptible * i / N * dt
    p_recovery = gamma * i * dt
    p_no_change = 1 - p_infection - p_recovery

if p_no_change < -1e-12:
        raise ValueError(f"Invalid probabilities at state {i}.")

# Stay in the current state
    P[i, i] = max(0.0, p_no_change)

# Infection: i to i + 1
    if i < N:
        P[i, i + 1] = p_infection

# Recovery: i to i - 1
    if i > 0:
        P[i, i - 1] = p_recovery

# Validate the row-stochastic matrix
if P.shape != (number_of_states, number_of_states):
    raise ValueError("P has the wrong shape.")

if (P < -1e-12).any() or (P > 1 + 1e-12).any():
    raise ValueError("A matrix entry is outside [0, 1].")

row_sums = P.sum(axis=1)

if not np.allclose(row_sums, 1.0):
    raise ValueError("The rows of P do not sum to 1.")

# Certain initial state: I_0 = 1
pi_current = np.array([0.0, 1.0, 0.0, 0.0])

if not np.isclose(pi_current.sum(), 1.0):
    raise ValueError("Initial probabilities do not sum to 1.")

# One distribution update
pi_next = pi_current @ P

if not np.isclose(pi_next.sum(), 1.0):
    raise ValueError("Updated probabilities do not sum to 1.")

labels = [f"I={i}" for i in states]

matrix_table = pd.DataFrame(
    P,
    index=[f"current {i}" for i in states],
    columns=[f"next {j}" for j in states]
)

vector_table = pd.DataFrame({
    "state": labels,
    "current probability": pi_current,
    "next probability": pi_next
})

print("Transition matrix P")
print(matrix_table.to_string())
print()
print("Row sums")
print(pd.Series(row_sums, index=matrix_table.index).to_string())
print()
print("One probability-vector update")
print(vector_table.to_string(index=False))
print()
print(f"Sum of next probabilities: {pi_next.sum():.6f}")

# Figure 1: transition-matrix heatmap
plt.figure(figsize=(7, 5.4))
image = plt.imshow(P, cmap="YlGn", vmin=0, vmax=1)
plt.colorbar(image, label="Transition probability")
plt.xticks(states, [f"next {j}" for j in states])
plt.yticks(states, [f"current {i}" for i in states])

for i in states:
    for j in states:
        plt.text(j, i, f"{P[i, j]:.1f}",
                 ha="center", va="center", color="black")

plt.title("Row-stochastic SIS transition matrix")
plt.tight_layout()

# Figure 2: distribution before and after one step
x = np.arange(number_of_states)
width = 0.36

plt.figure(figsize=(8, 4.8))
plt.bar(x - width/2, pi_current, width,
        label="Current distribution", color="#2563eb")
plt.bar(x + width/2, pi_next, width,
        label="After one step", color="#f59e0b")
plt.xticks(x, labels)
plt.xlabel("Infectious-count state")
plt.ylabel("Probability")
plt.ylim(0, 1.05)
plt.title("One exact probability-distribution update")
plt.legend()
plt.grid(axis="y", alpha=0.25)
plt.tight_layout()

Output

Run the code to see the result.

Understand the new matrix code

Code	Meaning
`np.zeros((4, 4))`	Creates a 4-by-4 matrix initially filled with zeros.
`P[i, j]`	Accesses the entry in current-state row \(i\) and next-state column \(j\).
`P.sum(axis=1)`	Sums across columns within each row. `axis=1` therefore checks row sums.
`np.allclose(...)`	Checks whether all calculated floating-point values are sufficiently close to their targets.
`pi_current @ P`	Uses `@` for matrix multiplication to calculate the next distribution.
`pi_current * P`	Would perform element-by-element multiplication with broadcasting; it is not the required distribution update.
`P.shape`	Returns the matrix dimensions as a tuple.

Interpret the one-step result

Beginning certainly at \(I_0=1\), after one step:

\(\Pr(I_1=0)=0.1\): recovery causes extinction;
\(\Pr(I_1=1)=0.7\): no event occurs;
\(\Pr(I_1=2)=0.2\): infection increases the count; and
\(\Pr(I_1=3)=0\): reaching state 3 would require two infections in one step and is excluded.

The vector contains all four possible one-step outcomes and their probabilities. A single simulated trajectory would display only one of these outcomes.

Try these experiments

Change the certain initial state to \(I_0=2\) using pi_current = np.array([0, 0, 1, 0]). Which matrix row becomes the next vector?
Use the mixed distribution [0.2, 0.5, 0.3, 0.0]. Observe how several current-state rows contribute.
Change one entry of pi_current without preserving its sum. Confirm that the validation reports the problem.
Increase dt. Find when at least one state's no-change probability becomes invalid.
Replace axis=1 with axis=0. Compare column sums with the required row sums.

What this lesson has—and has not—done

You can now:

construct a transition matrix from biological event probabilities;
interpret rows as current states and columns as next states;
check shape, entries and row sums;
represent certain and mixed state distributions;
calculate one exact update using pi_current @ P; and
distinguish a probability distribution from one random trajectory.

Not yet: this page performs only one matrix update. Lesson 11 repeatedly multiplies the vector by \(P\), follows the distribution through time and studies probability conservation and extinction probability.