Fisher Information Metric - Definition

Definition

Given a statistical manifold, with coordinates given by, one writes for the probability distribution. Here, is a specific value drawn from a collection of (discrete or continuous) random variables X. The probability is normalized, in that

The Fisher information metric then takes the form:

g_{jk}(\theta) = \int_X \frac{\partial \log p(x,\theta)}{\partial \theta_j} \frac{\partial \log p(x,\theta)}{\partial \theta_k} p(x,\theta) \, dx.

The integral is performed over all values x of all random variables X. Again, the variable is understood as a coordinate on the Riemann manifold. The labels j and k index the local coordinate axes on the manifold.

When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant. If there are n constraints holding n different expectation values constant, then the manifold is n-dimensional. In this case, the metric can be explicitly derived from the partition function; a derivation and discussion is presented there.

Substituting from information theory, an equivalent form of the above definition is:

g_{jk}(\theta) = \int_X \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} p(x,\theta) \, dx = \mathrm{E} \left[ \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} \right].

Read more about this topic:  Fisher Information Metric

Famous quotes containing the word definition:

    “The physicians say, they are not materialists; but they are:MSpirit is matter reduced to an extreme thinness: O so thin!—But the definition of spiritual should be, that which is its own evidence. What notions do they attach to love! what to religion! One would not willingly pronounce these words in their hearing, and give them the occasion to profane them.”
    —Ralph Waldo Emerson (1803–1882)

    “The definition of good prose is proper words in their proper places; of good verse, the most proper words in their proper places. The propriety is in either case relative. The words in prose ought to express the intended meaning, and no more; if they attract attention to themselves, it is, in general, a fault.”
    —Samuel Taylor Coleridge (1772–1834)

    “Although there is no universal agreement as to a definition of life, its biological manifestations are generally considered to be organization, metabolism, growth, irritability, adaptation, and reproduction.”
    —The Columbia Encyclopedia, Fifth Edition, the first sentence of the article on “life” (based on wording in the First Edition, 1935)