Part 3: Continuous-Time Markov Chains

A CTMC changes state at random event times. This pathway first separates rates from probabilities, then builds the waiting-time and event-selection mechanism, applies it to SIS, SIR and SEIR models, introduces generator matrices, and finally studies many trajectories and biological outcomes.

Stage 1 · Define continuous-time transitions

Events, rates and short intervals

Describe biological jumps and their instantaneous rates before connecting those rates to probabilities over a sufficiently small interval.

01Events and transition rates in continuous timeDefine CTMC states, biological jump events, instantaneous rates, units and boundary conditions—without simulating waiting times. 02From rates to small-interval probabilitiesInterpret rate × short interval, derive no-event and multiple-event orders, and state when the approximation is valid.

Stage 2 · Generate continuous-time events

Waiting times and the Gillespie mechanism

Separate the question “when does the next event occur?” from “which event occurs?”, then combine them into one complete CTMC simulation algorithm.

03Total event rate and exponential waiting timeAdd competing event rates and generate the random time to the next event from an exponential distribution. 04Selecting infection, progression or recovery eventsNormalise event rates by the total rate and select the event conditional on a jump occurring. 05Programming one CTMC eventCombine one exponential waiting time, one event choice and one biologically valid state update. 06Gillespie’s direct algorithm step by stepRepeat CTMC events, advance by unequal random times, store the jump path and stop under stated conditions.

Stage 3 · Apply the mechanism

SIS, SIR and SEIR CTMC models

Reuse the Gillespie mechanism while changing the biological compartments, events, boundaries and outcome definitions.

07Programming a CTMC SIS modelModel reinfection, birth–death rates, extinction and possible persistence before absorption. 08Programming a CTMC SIR modelModel irreversible recovery, final epidemic size, peak outcomes and the absorbing disease-free boundary. 09Programming a CTMC SEIR modelAdd exposure, latent progression, three competing rates and the correct extinction condition.

Stage 4 · Represent the transition structure

Generator matrices

Move from trajectory simulation to the matrix that records all instantaneous transition rates and their conservation structure.

10Generator matrices and transition structureConstruct off-diagonal rates, negative diagonal entries, row sums and the connection between a generator and short-time transitions.

Stage 5 · Estimate and communicate outcomes

Monte Carlo probabilities and epidemic summaries

Simulate independent continuous-time paths, define outcomes before counting them, quantify uncertainty and choose appropriate displays.

11Simulating many continuous-time trajectoriesRun independent Gillespie paths, align jump processes on observation grids and form empirical distributions. 12Extinction probability and outbreak probabilityDefine early fade-out, major outbreak and time-specific events before estimating probabilities and Monte Carlo uncertainty. 13Outbreak size, duration, peak size and peak timeCalculate trajectory-level outcomes, empirical distributions, dependence between summaries and conditional statistics. 14Event-path plots, step plots, distributions and tablesCommunicate irregular jump times, sample-path variability, outcome distributions and biological conclusions correctly.

Coverage rule

Lessons 1–6 teach the generic CTMC simulation mechanism once. Lessons 7–9 apply it to different epidemic structures. Lesson 10 introduces the generator viewpoint. Lessons 11–14 estimate and communicate stochastic outcomes. Each lesson may reuse earlier code, but its explanation should focus on genuinely new mathematics, biology or Python.