Second moments, variance and covariance

The mean describes centre; second moments describe spread and joint variation between biological quantities.

Why the mean is insufficient

Two epidemic distributions can have the same mean infectious count but very different spreads. The second raw moment is \(\mathbb E[X^2]\), while variance is

\[\operatorname{Var}(X)=\mathbb E[X^2]-(\mathbb E[X])^2.\]

Variance is measured in squared units; standard deviation is its square root and uses the original units.

Covariance connects two variables

\[\operatorname{Cov}(X,Y)=\mathbb E[XY]-\mathbb E[X]\mathbb E[Y].\]

Positive covariance means above-average values tend to occur together. Negative covariance means one tends to be above average when the other is below average. Zero covariance does not generally prove independence.

Biological scenario

For an SIR epidemic, record susceptible and infectious populations on day 25. Infection transfers people from \(S\) to \(I\), so the two quantities may be negatively associated across epidemic histories.

Interactive Python laboratory

Interactive PythonSecond moments and covariance

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

def sir_state_at_time(rng, N, I0, beta, gamma, T):
    S, I, R = N - I0, I0, 0
    time = 0.0

while I > 0 and time < T:
        infection_rate = beta * S * I / N
        recovery_rate = gamma * I
        total_rate = infection_rate + recovery_rate
        wait = rng.exponential(1 / total_rate)

if time + wait > T:
            break

time += wait
        if rng.random() < infection_rate / total_rate:
            S -= 1
            I += 1
        else:
            I -= 1
            R += 1

return S, I

M = 4000
rng = np.random.default_rng(230)
sample = np.array([
    sir_state_at_time(rng, 200, 3, 0.25, 0.10, 25)
    for _ in range(M)
])
S_sample = sample[:, 0]
I_sample = sample[:, 1]

mean_S = S_sample.mean()
mean_I = I_sample.mean()
second_S = np.mean(S_sample ** 2)
second_I = np.mean(I_sample ** 2)
variance_S = S_sample.var(ddof=1)
variance_I = I_sample.var(ddof=1)
covariance_SI = np.cov(S_sample, I_sample, ddof=1)[0, 1]
correlation_SI = np.corrcoef(S_sample, I_sample)[0, 1]

summary = pd.DataFrame({
    "quantity": [
        "Mean S", "Mean I",
        "Second raw moment of S",
        "Second raw moment of I",
        "Sample variance of S",
        "Sample variance of I",
        "Sample covariance Cov(S,I)",
        "Sample correlation Corr(S,I)"
    ],
    "value": [
        mean_S, mean_I, second_S, second_I,
        variance_S, variance_I,
        covariance_SI, correlation_SI
    ]
})

print(summary.round(4).to_string(index=False))

plt.figure(figsize=(7.5, 5.5))
plt.scatter(S_sample, I_sample,
            s=10, alpha=0.22, color="#2563eb")
plt.axvline(mean_S, color="#174c34", linestyle="--")
plt.axhline(mean_I, color="#b42318", linestyle="--")
plt.xlabel("Susceptible population on day 25")
plt.ylabel("Infectious population on day 25")
plt.title("Joint sample reveals dependence between S and I")
plt.grid(alpha=0.2)
plt.tight_layout()

covariance_matrix = np.cov(sample.T, ddof=1)
plt.figure(figsize=(5.8, 4.8))
image = plt.imshow(covariance_matrix, cmap="RdBu_r")
plt.colorbar(image, label="Sample covariance")
plt.xticks([0, 1], ["S", "I"])
plt.yticks([0, 1], ["S", "I"])
for row in range(2):
    for column in range(2):
        plt.text(column, row,
                 f"{covariance_matrix[row, column]:.1f}",
                 ha="center", va="center")
plt.title("Covariance matrix on day 25")
plt.tight_layout()

Output

Run the code to see the result.

Raw moment versus central moment

Quantity	Centred?	Purpose
\(\mathbb E[X^2]\)	No	Second raw moment used in moment equations.
\(\operatorname{Var}(X)\)	Yes	Spread around the mean.
\(\mathbb E[XY]\)	No	Mixed raw moment.
\(\operatorname{Cov}(X,Y)\)	Yes	Joint variation around both means.

What this lesson adds

You can now distinguish second raw moments from variance, calculate covariance and correlation from paired epidemic samples, and interpret their biological signs without claiming that covariance proves causation.