Maximum-Likelihood (ML) Estimation

An important class of estimators is the maximum-likelihood (ML) estimators. The ML parameter estimate is the one that makes the set of mutually-independent observations ${\bf x} = \{ x_1, x_2, \ldots, x_n \}$ (which is an instance of the random sample $\{ X_1, X_2, \ldots, X_n \}$) most likely to occur. The random sample comprises mutually independent RVs, thereby making the joint PDF equivalent to the product of the marginal PDFs. This defines the likelihood function for the parameter $\theta$ as

$$L (\theta \vert {\bf x}) = P ({\bf x} \vert \theta)$$ (30)

$$= P (X_1 = x_1, X_2 = x_2, \ldots, X_n = x_n \vert \theta)$$ (31)

$$= \prod_{i=1}^{n} P_{X_i} (x_i \vert \theta),$$ (32)

The ML parameter estimate is

$$\theta^* = \mathop{\mbox{argmax }}_{\theta} L (\theta \vert {\bf x}).$$ (33)

An interesting, and useful, property about ML estimators is that all efficient estimators are necessarily ML estimators [123]. As an example, consider a Rician PDF, with $\sigma = 1$ and unknown $\mu$, that generates a sample comprising just a single observation $x$. Then, the likelihood function $L (\mu \vert x)$ would be:

$$L ( \mu \vert x) = \frac {1} {\eta} \frac {x} {\sigma^2} \exp \Bi... ...^2 + \mu^2} { 2 \sigma^2} \Bigg) % I_0 \Bigg( \frac {x \mu} {\sigma^2} \Bigg),$$ (34)

where $x$ and $\sigma$ are known constants, and $\eta$ is the normalization factor. Figure 2.4 shows the Rician-likelihood function for two different values of the observation $x$.

Figure 2.4: Rician likelihood functions with (a)  $x = 2, \sigma = 1$, and (b)  $x = 5, \sigma = 1$.