Expectation and the sample mean

Expectation is the centre of a model distribution; the sample mean is its finite-simulation estimate.

Expectation is a probability-weighted average

If a discrete random variable \(X\) can take values \(x\) with probabilities \(\Pr(X=x)\), then

\[\mathbb E[X]=\sum_x x\,\Pr(X=x).\]

Expectation describes the long-run centre of repeated outcomes. It need not be an attainable individual outcome: an expected infectious count may be 18.4 although every CTMC realisation is an integer.

Sample mean estimates expectation

When the theoretical probabilities are unavailable, simulate \(M\) independent values \(X_1,\ldots,X_M\) and calculate

\[\overline X_M=\frac{1}{M}\sum_{r=1}^{M}X_r.\]

The expectation is a property of the model distribution. The sample mean is a random estimate computed from a finite sample.

Why independence matters

Each epidemic must use new random draws while keeping the same model and initial conditions. Copying one outcome many times does not provide more information about the distribution.

Interactive Python laboratory

Estimate \(\mathbb E[I(20)]\) from 5,000 independent SIS CTMC simulations and watch the running sample mean stabilise.

Interactive PythonExpectation and sample mean

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

def infectious_at_time(rng, N, I0, beta, gamma, T):
    I = I0
    time = 0.0

while I > 0 and time < T:
        infection_rate = beta * (N - I) * I / N
        recovery_rate = gamma * I
        total_rate = infection_rate + recovery_rate
        wait = rng.exponential(1 / total_rate)

if time + wait > T:
            break

time += wait
        if rng.random() < infection_rate / total_rate:
            I += 1
        else:
            I -= 1

return I

M = 5000
rng = np.random.default_rng(220)
sample = np.array([
    infectious_at_time(rng, 100, 5, 0.25, 0.15, 20)
    for _ in range(M)
])

running_mean = np.cumsum(sample) / np.arange(1, M + 1)

sizes = [10, 50, 100, 500, 1000, 5000]
summary = pd.DataFrame({
    "sample size": sizes,
    "sample mean": [running_mean[m - 1] for m in sizes]
})

print("Estimates of E[I(20)]")
print(summary.round(4).to_string(index=False))
print()
print(f"Final estimate: {sample.mean():.4f}")

plt.figure(figsize=(9.5, 5.1))
plt.plot(np.arange(1, M + 1), running_mean,
         color="#174c34")
plt.axhline(sample.mean(), color="#b42318",
            linestyle="--", label="Final estimate")
plt.xlabel("Number of independent simulations")
plt.ylabel("Running sample mean of I(20)")
plt.title("The sample mean stabilises as evidence accumulates")
plt.legend()
plt.grid(alpha=0.25)
plt.tight_layout()

Output

Run the code to see the result.

Interpretation

The running mean fluctuates strongly when only a few epidemics have been observed. It becomes more stable with more independent runs, but a finite sample mean is still an estimate. Lesson 04 will quantify Monte Carlo error.

What this lesson adds

You can now distinguish expectation from one outcome and from a finite sample mean, calculate a running mean, and explain why larger independent samples provide more stable estimates.