Part 2: Discrete-Time Markov Chains

This pathway separates the DTMC ideas carefully. First define states and valid probabilities, then learn random selection and trajectory construction, apply the method to SIS, SIR and SEIR, and finally study distributions, repeated simulations and outbreak outcomes.

Stage A: Define the stochastic model

Understand what the system records, what probabilities must satisfy and how biological events determine possible transitions.

01States, fixed time steps and random transitionsDefine states, state spaces, fixed times, the Markov property and one realised path—without calculating probabilities. 02Transition probabilities and their conditionsLearn conditional transition notation, non-negativity, normalisation, boundary states and time-step validity. 03Infection, recovery and no-change eventsTranslate biological event mechanisms into model-specific transition probabilities and a remaining no-change probability.

Stage B: Generate one stochastic trajectory

Move from probabilities to random event selection, then apply one transition repeatedly while storing a valid path.

04Uniform random numbers and probability intervalsMap one draw from \(U[0,1)\) to mutually exclusive outcomes and introduce reproducible random seeds. 05Programming one DTMC transitionCombine the current state, valid probabilities and one random draw to produce exactly one next state. 06Loops for a complete epidemic trajectoryRepeat the one-step mechanism, store times and states, enforce boundaries and stop under stated conditions.

Stage C: Build epidemic DTMC models

Apply the common simulation mechanism to three biologically distinct compartment structures.

07Programming a DTMC SIS modelRepresent reinfection, infectious-count states, birth–death movement and the absorbing extinction state. 08Programming a DTMC SIR modelTrack susceptible, infectious and removed counts, irreversible recovery and final outbreak size. 09Programming a DTMC SEIR modelAdd exposed states, progression events and the extra probability constraints created by four compartments.

Stage D: Calculate state distributions

Study the matrix approach, which evolves an entire probability distribution rather than selecting one random trajectory.

10Transition matrices and probability vectorsConstruct a stochastic matrix, fix its row/column convention, represent an initial distribution and calculate one distribution update. 11Matrix updating through timeRepeat matrix multiplication, check probability conservation and compare exact state probabilities with trajectory simulation.

Stage E: Estimate outcomes and communicate uncertainty

Use independent simulations to estimate distributions and probabilities, then select visual and tabular summaries appropriate to discrete stochastic data.

12Simulating many independent trajectoriesRun Monte Carlo experiments, manage reproducibility, align paths and form empirical distributions. 13Extinction and major-outbreak probabilitiesDefine outcome events and thresholds before estimating their probabilities and Monte Carlo uncertainty. 14Histograms, sample paths, probability plots and tablesCommunicate individual paths, empirical distributions, state probabilities and biological conclusions without confusing them.

Coverage and non-duplication rule

Lessons 1–6 teach the generic DTMC mechanism once. Lessons 7–9 apply that mechanism to different biological models. Lessons 10–11 calculate probability distributions without random sampling. Lessons 12–14 use Monte Carlo simulation to estimate outcomes and communicate uncertainty. Each later lesson may reuse earlier code, but should explain only its genuinely new modelling or programming idea.