Asymptotic Equipartition Property - AEP For Discrete-time I.i.d. Sources

AEP For Discrete-time I.i.d. Sources

Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,

\lim_{n\to\infty}\Pr\left=0 \qquad \forall \epsilon>0.

since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,

\Pr\left=1

which implies the result from the weak law of large numbers.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word sources:

    “On board ship there are many sources of joy of which the land knows nothing. You may flirt and dance at sixty; and if you are awkward in the turn of a valse, you may put it down to the motion of the ship. You need wear no gloves, and may drink your soda-and-brandy without being ashamed of it.”
    —Anthony Trollope (1815–1882)