Fisher Information - Matrix Form

Matrix Form

When there are N parameters, so that θ is a N × 1 vector \theta = \begin{bmatrix} \theta_{1}, \theta_{2}, \dots, \theta_{N} \end{bmatrix}^{\mathrm T}, then the Fisher information takes the form of an N × N matrix, the Fisher Information Matrix (FIM), with typical element

{\left(\mathcal{I} \left(\theta \right) \right)}_{i, j} = \operatorname{E} \left[\left. \left(\frac{\partial}{\partial\theta_i} \log f(X;\theta)\right) \left(\frac{\partial}{\partial\theta_j} \log f(X;\theta)\right) \right|\theta\right].

The FIM is a N × N positive semidefinite symmetric matrix, defining a Riemannian metric on the N-dimensional parameter space, thus connecting Fisher information to differential geometry. In that context, this metric is known as the Fisher information metric, and the topic is called information geometry.

Under certain regularity conditions, the Fisher Information Matrix may also be written as

{\left(\mathcal{I} \left(\theta \right) \right)}_{i, j} = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta_i \, \partial\theta_j} \log f(X;\theta) \right|\theta\right]\,.

The metric is interesting in several ways; it can be derived as the Hessian of the relative entropy; it can be understood as a metric induced from the Euclidean metric, after appropriate change of variable; in its complex-valued form, it is the Fubini-Study metric.

Read more about this topic:  Fisher Information

Famous quotes containing the words matrix and/or form:

    ““The matrix is God?”
    “In a manner of speaking, although it would be more accurate ... to say that the matrix has a God, since this being’s omniscience and omnipotence are assumed to be limited to the matrix.”
    “If it has limits, it isn’t omnipotent.”
    “Exactly.... Cyberspace exists, insofar as it can be said to exist, by virtue of human agency.””
    —William Gibson (b. 1948)

    “I do not mean to imply that the good old days were perfect. But the institutions and structure—the web—of society needed reform, not demolition. To have cut the institutional and community strands without replacing them with new ones proved to be a form of abuse to one generation and to the next. For so many Americans, the tragedy was not in dreaming that life could be better; the tragedy was that the dreaming ended.”
    —Richard Louv (20th century)