Stochastic processes
A random variable describes an uncertain numerical quantity. Biological systems usually change through time, so we need a way to describe an uncertain quantity at many different times. This leads to a stochastic process.
From one random variable to a process
Suppose
\[I(10)=\text{number of infectious individuals on day 10}.\]This is one random variable. If instead we want to follow the infectious population through time, we consider
\[I(0),\ I(1),\ I(2),\ldots\]or, in continuous time,
\[I(t),\qquad t\ge0.\]The whole collection is a stochastic process.
Formal notation
A stochastic process is commonly written
\[\boxed{\{X(t):t\in T\}}.\]| Symbol | Meaning |
|---|---|
| \(X(t)\) | random state at time \(t\) |
| \(t\) | a particular time |
| \(T\) | the set of times being considered |
| \(\{X(t):t\in T\}\) | the complete stochastic process |
For each fixed time \(t\), \(X(t)\) is a random variable.
One process can produce many trajectories
Suppose the same epidemic is started many times with the same initial state and the same parameters. Infection and recovery events can occur in different orders, so the realised paths can differ.
A trajectory, sample path or realisation is one possible path produced by the stochastic process.
What does \(X(t)\) mean at one fixed time?
Take one particular time, say \(t=5\). Each possible trajectory has a definite value at that time. Across all possible trajectories, those values form the probability distribution of \(X(5)\).
This gives two related views of a stochastic process:
| View | Meaning |
|---|---|
| follow one trajectory horizontally through time | one realised history of the process |
| look vertically at one fixed time | possible values of the random variable \(X(t)\) |
State and state space
The state describes the system at one time. For a simple SIS epidemic we might use
\[X(t)=I(t).\]If the total population is fixed at \(N\), then \(S(t)=N-I(t)\), so knowing \(I(t)\) determines the full SIS state.
For an SEIR epidemic the state may be a vector:
\[\mathbf{X}(t)=\big(S(t),E(t),I(t),R(t)\big).\]The state space is the collection of all values the state is allowed to take. If \(I(t)\) counts infectious people in a population of size \(N\), then
\[\mathcal{S}=\{0,1,2,\ldots,N\}.\]Time can be discrete or continuous
Discrete time
The process is examined at separated time points, for example
\[t=0,1,2,3,\ldots\]with one update each day.
Continuous time
Events can happen at any instant, such as an infection at \(t=2.37\) days and a recovery at \(t=2.91\) days.
The state can also be discrete or continuous
| Time | State | Example |
|---|---|---|
| discrete | discrete | daily number infected |
| continuous | discrete | infection and recovery events at arbitrary times |
| discrete | continuous | biomass measured at regular intervals |
| continuous | continuous | continuous stochastic concentration or diffusion process |
Continuous time does not imply a continuous-valued state.
Continuous time can still produce jumps
In a continuous-time epidemic Markov chain, the event time is continuous but the population count changes by integer jumps:
\[I\to I+1\quad\text{for an infection},\] \[I\to I-1\quad\text{for a recovery}.\]Values at different times are connected
A stochastic process is not merely a collection of unrelated random variables. Usually, what happens now affects what can happen next.
If an epidemic has \(I(t)=100\) infectious people now, the distribution of the future will generally differ from the future distribution when \(I(t)=1\).
We therefore need rules describing movement between states.
Transition probabilities
A common quantity is
\[P\big(X(t+\Delta t)=j\mid X(t)=i\big).\]This means:
These transition probabilities connect the random variables at different times.
A simple stochastic SIS process
Suppose \(I(t)=i\). Over a sufficiently short interval \(\Delta t\), the main possibilities are:
| Event | New state | Approximate probability |
|---|---|---|
| infection | \(i+1\) | \(b(i)\Delta t\) |
| recovery | \(i-1\) | \(d(i)\Delta t\) |
| no event | \(i\) | \(1-[b(i)+d(i)]\Delta t\) |
For a simple SIS epidemic, one possible infection rate is
\[b(i)=\beta\frac{(N-i)i}{N}.\]As \(i\) changes, the event rates change, so the probability law of the future also changes.
Expectation and variance through time
Because \(X(t)\) is a random variable for every \(t\), we can define its mean function
\[m(t)=E[X(t)]\]and variance
\[\operatorname{Var}(X(t)).\]The mean tells us the centre of the distribution at time \(t\); the variance tells us how widely the possible states are spread around that mean.
Dependence between different times
The relationship between two times can be studied using quantities such as
\[\operatorname{Cov}(X(s),X(t)).\]This measures how the random state at time \(s\) is associated with the random state at time \(t\).
Important biological stochastic processes
| Process | Typical biological use |
|---|---|
| discrete-time Markov chain | random transitions at fixed time steps |
| continuous-time Markov chain | individual events occurring at random continuous times |
| birth–death process | population growth, decline and extinction |
| Poisson process | counting random events through time |
| branching process | early epidemic spread and extinction |
| diffusion process / SDE | continuous stochastic fluctuations |
What can a stochastic process answer?
Examples include
\[P(I(t)=0),\]the probability that infection is extinct by time \(t\);
\[P\left(\max_t I(t)>K\right),\]the probability that infection exceeds a threshold \(K\); and
\[E[I(t)],\qquad \operatorname{Var}(I(t)),\]the expected infectious population and its variability.
Process, trajectory and simulation
| Term | Meaning |
|---|---|
| stochastic process | the probability model governing random evolution through time |
| trajectory / sample path | one realised path from that process |
| simulation | a computational method used to generate a trajectory according to the process rules |
Where we go next
The next question is how much of the past is needed to determine the probability law of the future. This leads to the Markov property.