Asymptotic Equipartition Property - Definition

Definition

Given a discrete-time stationary ergodic stochastic process on the probability space, AEP is an assertion that

-\frac{1}{n} \log p(X_1^n) \to H(X) \quad \mbox{ as } \quad n\to\infty

where denotes the process limited to duration, and or simply denotes the entropy rate of, which must exist for all discrete-time stationary processes including the ergodic ones. AEP is proved for finite-valued (i.e. ) stationary ergodic stochastic processes in the Shannon-McMillan-Breiman theorem using the ergodic theory and for any i.i.d. sources directly using the law of large numbers in both the discrete-valued case (where is simply the entropy of a symbol) and the continuous-valued case (where is the differential entropy instead). The definition of AEP can also be extended for certain classes of continuous-time stochastic processes for which a typical set exists for long enough observation time. The convergence is proven almost sure in all cases.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word definition:

    “The man who knows governments most completely is he who troubles himself least about a definition which shall give their essence. Enjoying an intimate acquaintance with all their particularities in turn, he would naturally regard an abstract conception in which these were unified as a thing more misleading than enlightening.”
    —William James (1842–1910)

    “No man, not even a doctor, ever gives any other definition of what a nurse should be than this—”devoted and obedient.” This definition would do just as well for a porter. It might even do for a horse. It would not do for a policeman.”
    —Florence Nightingale (1820–1910)

    “Mothers often are too easily intimidated by their children’s negative reactions...When the child cries or is unhappy, the mother reads this as meaning that she is a failure. This is why it is so important for a mother to know...that the process of growing up involves by definition things that her child is not going to like. Her job is not to create a bed of roses, but to help him learn how to pick his way through the thorns.”
    —Elaine Heffner (20th century)