Fisher Information Metric - Definition

Definition

Given a statistical manifold, with coordinates given by, one writes for the probability distribution. Here, is a specific value drawn from a collection of (discrete or continuous) random variables X. The probability is normalized, in that

The Fisher information metric then takes the form:

g_{jk}(\theta) = \int_X \frac{\partial \log p(x,\theta)}{\partial \theta_j} \frac{\partial \log p(x,\theta)}{\partial \theta_k} p(x,\theta) \, dx.

The integral is performed over all values x of all random variables X. Again, the variable is understood as a coordinate on the Riemann manifold. The labels j and k index the local coordinate axes on the manifold.

When the probability is derived from the Gibbs measure, as it would be for any Markovian process, then can also be understood to be a Lagrange multiplier; Lagrange multipliers are used to enforce constraints, such as holding the expectation value of some quantity constant. If there are n constraints holding n different expectation values constant, then the manifold is n-dimensional. In this case, the metric can be explicitly derived from the partition function; a derivation and discussion is presented there.

Substituting from information theory, an equivalent form of the above definition is:

g_{jk}(\theta) = \int_X \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} p(x,\theta) \, dx = \mathrm{E} \left[ \frac{\partial^2 i(x,\theta)}{\partial \theta_j \partial \theta_k} \right].

Read more about this topic:  Fisher Information Metric

Famous quotes containing the word definition:

    “... we all know the wag’s definition of a philanthropist: a man whose charity increases directly as the square of the distance.”
    —George Eliot [Mary Ann (or Marian)

    “Mothers often are too easily intimidated by their children’s negative reactions...When the child cries or is unhappy, the mother reads this as meaning that she is a failure. This is why it is so important for a mother to know...that the process of growing up involves by definition things that her child is not going to like. Her job is not to create a bed of roses, but to help him learn how to pick his way through the thorns.”
    —Elaine Heffner (20th century)

    “According to our social pyramid, all men who feel displaced racially, culturally, and/or because of economic hardships will turn on those whom they feel they can order and humiliate, usually women, children, and animals—just as they have been ordered and humiliated by those privileged few who are in power. However, this definition does not explain why there are privileged men who behave this way toward women.”
    —Ana Castillo (b. 1953)