Maximum likelihood
Maximum likelihood estimates parameters using an explicit probability model for the observed data.
If the data are \(y=(y_1,\ldots,y_n)\), the likelihood is
\[L(\theta;y)=P(y\mid\theta)\]for discrete data, or the corresponding probability density for continuous data. The maximum-likelihood estimate is
\[\hat\theta=\arg\max_{\theta}L(\theta;y).\]Log-likelihood
It is usually easier to maximise
\[\ell(\theta)=\log L(\theta;y),\]which has the same maximiser.
Observation models
Counts may suggest Poisson or negative-binomial models, binary outcomes Bernoulli models, and continuous measurements may sometimes be approximated by normal errors.
Key idea. Likelihood connects parameter estimation directly to assumptions about how the observed data are generated.