Maximum Likelihood - Properties

Properties

A maximum-likelihood estimator is an extremum estimator obtained by maximizing, as a function of θ, the objective function

\hat\ell(\theta|x)=\frac1n \sum_{i=1}^n \ln f(x_i|\theta),

this being the sample analogue of the expected log-likelihood, where this expectation is taken with respect to the true density f(·|θ0).

Maximum-likelihood estimators have no optimum properties for finite samples, in the sense that (when evaluated on finite samples) other estimators have greater concentration around the true parameter-value. However, like other estimation methods, maximum-likelihood estimation possesses a number of attractive limiting properties: As the sample-size increases to infinity, sequences of maximum-likelihood estimators have these properties:

  • Consistency: a subsequence of the sequence of MLEs converges in probability to the value being estimated.
  • Asymptotic normality: as the sample size increases, the distribution of the MLE tends to the Gaussian distribution with mean and covariance matrix equal to the inverse of the Fisher information matrix.
  • Efficiency, i.e., it achieves the Cramér–Rao lower bound when the sample size tends to infinity. This means that no asymptotically unbiased estimator has lower asymptotic mean squared error than the MLE (or other estimators attaining this bound).
  • Second-order efficiency after correction for bias.

Read more about this topic:  Maximum Likelihood

Famous quotes containing the word properties:

    “A drop of water has the properties of the sea, but cannot exhibit a storm. There is beauty of a concert, as well as of a flute; strength of a host, as well as of a hero.”
    —Ralph Waldo Emerson (1803–1882)

    “The reason why men enter into society, is the preservation of their property; and the end why they choose and authorize a legislative, is, that there may be laws made, and rules set, as guards and fences to the properties of all the members of the society: to limit the power, and moderate the dominion, of every part and member of the society.”
    —John Locke (1632–1704)