Confidence intervals
A fitted parameter value is only a point estimate. If a new dataset were collected from the same biological process, the estimate would usually change. Confidence intervals describe this sampling uncertainty using a specified statistical procedure.
Point estimates are uncertain
Suppose a parameter \(\theta\) is estimated by
\[\hat\theta.\]The estimate depends on the observed sample. Repeating the experiment would generally produce a different \(\hat\theta\).
The sampling distribution of the estimator describes this variation.
Confidence level
A \(100(1-\alpha)\%\) confidence procedure produces intervals
\[\boxed{[L(\mathbf Y),U(\mathbf Y)]}\]such that, under the model and repeated sampling,
\[\boxed{P_\theta\!\left(L(\mathbf Y)\le\theta\le U(\mathbf Y)\right)=1-\alpha}\]or approximately this value for an approximate procedure.
What 95% confidence means
Imagine repeatedly collecting comparable datasets and calculating a 95% confidence interval from each one using the same procedure. Approximately 95% of those intervals should contain the true parameter when the assumptions and approximation are valid.
The parameter is treated as fixed; the endpoints vary from sample to sample.
Standard error
The standard error of an estimator is the standard deviation of its sampling distribution:
\[\boxed{\operatorname{SE}(\hat\theta)=\sqrt{\operatorname{Var}(\hat\theta)}}.\]Because the true sampling variance is usually unknown, it is itself estimated from the data and model.
Normal approximation
If
\[\hat\theta\approx N(\theta,\operatorname{SE}^2),\]then an approximate \(100(1-\alpha)\%\) interval is
\[\boxed{\hat\theta\pm z_{1-\alpha/2}\operatorname{SE}(\hat\theta)}.\]For a 95% interval, \(z_{0.975}\approx1.96\), giving
\[\hat\theta\pm1.96\operatorname{SE}(\hat\theta).\]Wald intervals
Intervals based on an estimate plus or minus a normal critical value times its standard error are commonly called Wald intervals.
They are simple but rely on the estimator being sufficiently close to normally distributed and the local uncertainty being reasonably symmetric.
Information matrix
For a maximum-likelihood estimate, local uncertainty can be approximated using the observed information matrix
\[J(\hat{\boldsymbol\theta})=-\nabla^2\ell(\hat{\boldsymbol\theta}).\]Under suitable regularity conditions and with enough information,
\[\boxed{\operatorname{Cov}(\hat{\boldsymbol\theta})\approx J(\hat{\boldsymbol\theta})^{-1}}.\]The square roots of the diagonal entries provide approximate standard errors.
Parameter covariance
For several parameters, the covariance matrix contains more information than separate standard errors.
Large off-diagonal terms indicate that parameter estimates vary together. Strong parameter correlation can make individual parameter values poorly determined even when the fitted trajectory looks precise.
Why symmetric intervals can fail
A parameter such as a transmission rate must satisfy
\[\beta>0.\]A symmetric Wald interval can nevertheless extend below zero:
\[\hat\beta-1.96\operatorname{SE}(\hat\beta)<0.\]This is one sign that a symmetric normal approximation may be inappropriate.
Intervals on a transformed scale
For a positive parameter, define
\[\eta=\log\theta.\]An interval can be constructed for \(\eta\) and transformed back:
\[[L_\eta,U_\eta]\quad\longrightarrow\quad[e^{L_\eta},e^{U_\eta}].\]This guarantees positive endpoints and naturally allows asymmetry on the original scale.
Profile-likelihood intervals
For parameter \(\theta_j\), fix \(\theta_j\) at a sequence of candidate values and re-optimise all remaining parameters. This gives the profile log-likelihood
\[\ell_p(\theta_j).\]Values can be retained while the likelihood-ratio statistic remains below an appropriate threshold:
\[\boxed{2\left[\ell(\hat{\boldsymbol\theta})-\ell_p(\theta_j)\right]\le\chi^2_{1,1-\alpha}}.\]Under suitable regularity conditions, the retained range gives an approximate confidence interval.
Why profile likelihood is useful
Profile intervals can be asymmetric and can reveal flat regions or one-sided uncertainty that a local Hessian approximation may hide.
They are especially useful for nonlinear mechanistic models where parameters can compensate for one another.
Bootstrap idea
The bootstrap approximates sampling variability by repeatedly generating or resampling datasets, refitting the model and examining the resulting parameter estimates.
If the bootstrap estimates are
\[\hat\theta^{*(1)},\ldots,\hat\theta^{*(B)},\]their empirical distribution approximates aspects of the estimator's sampling distribution under the bootstrap scheme.
Nonparametric bootstrap
A nonparametric bootstrap resamples observational units from the empirical data with replacement.
The resampling unit must respect the study design. For repeated measurements from individuals, resampling individual rows independently can destroy within-individual dependence.
Parametric bootstrap
A parametric bootstrap simulates new datasets from the fitted model:
\[\mathbf Y^*\sim p(\mathbf y\mid\hat{\boldsymbol\theta}),\]then re-estimates parameters for each simulated dataset.
This preserves the assumed model structure but depends directly on that model being a reasonable representation of the data-generating process.
Percentile bootstrap interval
A simple bootstrap interval uses empirical quantiles. For a 95% interval, approximately the 2.5th and 97.5th percentiles of the bootstrap estimates are used.
More refined bootstrap intervals exist because the basic percentile method may have imperfect coverage, especially for biased or asymmetric estimators.
Dependence matters in bootstrap sampling
Time-series, spatial, network and clustered data cannot always be bootstrapped by independently resampling individual observations.
The resampling procedure should preserve the dependence structure relevant to the model.
Confidence interval for a derived quantity
Often the biological quantity of interest is a function
\[g(\boldsymbol\theta).\]For an SIR model, for example,
\[R_0=\frac{\beta}{\gamma}.\]Uncertainty in \(\beta\) and \(\gamma\), including their covariance, must be propagated into uncertainty in \(R_0\).
Delta method
If \(g\) is sufficiently smooth, a local approximation gives
\[\boxed{\operatorname{Var}(g(\hat{\boldsymbol\theta}))\approx\nabla g^T\operatorname{Cov}(\hat{\boldsymbol\theta})\nabla g}.\]This is called the delta method.
It is useful when uncertainty is moderate and local linearisation is adequate.
Confidence intervals and prediction intervals
A confidence interval for a parameter is different from an interval for a future biological observation.
A prediction interval generally includes both parameter uncertainty and new observation or process variability, so it is often wider.
Confidence bands for trajectories
If a fitted model predicts
\[x(t;\hat{\boldsymbol\theta}),\]uncertainty in the parameters can be propagated to produce uncertainty bands for the predicted trajectory.
These should not automatically be called prediction intervals unless future observation/process variation has also been included appropriately.
Sample size and interval width
With regular models and informative independent data, increasing sample size generally reduces standard errors and narrows confidence intervals.
But collecting more observations of the same uninformative quantity may not solve an identifiability problem.
Confidence level and width
Higher confidence requires a wider interval when everything else is fixed. For example, a 99% interval is generally wider than a 95% interval because it is designed for greater long-run coverage.
Small samples
Asymptotic normal and likelihood-ratio approximations may be inaccurate when the sample is small.
Bootstrap or exact methods can sometimes help, but no method can create information that the data do not contain.
Parameters near boundaries
When a fitted parameter lies near a boundary such as zero, the sampling distribution can be strongly asymmetric and standard Wald intervals can perform poorly.
Profile likelihood or appropriately designed bootstrap procedures may better represent this uncertainty.
Non-identifiable parameters
If a parameter is structurally non-identifiable, the model and observations cannot uniquely determine it even with ideal data.
A conventional finite confidence interval may then be misleading.
If a parameter is only weakly practically identifiable, its interval may be extremely wide or effectively unbounded.
Narrow intervals can still be misleading
A narrow confidence interval is meaningful only relative to the assumed model, observation distribution and sampling design.
If the model is misspecified or systematic bias is present, a narrow interval can give precise uncertainty around the wrong quantity.
Reporting intervals
A useful report states the point estimate, confidence level, interval endpoints and method used to obtain the interval.
For example, saying “95% profile-likelihood confidence interval” is more informative than reporting endpoints without explaining their construction.
Comparing interval methods
| Method | Main feature |
|---|---|
| Wald | fast local normal approximation |
| transformed Wald | respects constraints such as positivity more naturally |
| profile likelihood | allows nonlinear and asymmetric likelihood shape |
| bootstrap | approximates repeated fitting through resampling or simulation |
No method is universally best. The choice depends on model structure, sample size, parameter constraints, computational cost and the type of uncertainty present.
Transition to identifiability
A wide interval may arise because the data are noisy, but it may also indicate a deeper problem: the observations may not distinguish the parameter from other parameter combinations. The next lesson develops structural and practical identifiability.