Asymptotic Equipartition Property - AEP For Discrete-time I.i.d. Sources

AEP For Discrete-time I.i.d. Sources

Given is an i.i.d. source, its time series X1, ..., Xn is i.i.d. with entropy H(X) in the discrete-valued case and differential entropy in the continuous-valued case. The weak law of large numbers gives the AEP with convergence in probability,

\lim_{n\to\infty}\Pr\left=0 \qquad \forall \epsilon>0.

since the entropy is equal to the expectation of . The strong law of large number asserts the stronger almost sure convergence,

\Pr\left=1

which implies the result from the weak law of large numbers.

Read more about this topic:  Asymptotic Equipartition Property

Famous quotes containing the word sources:

    “No drug, not even alcohol, causes the fundamental ills of society. If we’re looking for the sources of our troubles, we shouldn’t test people for drugs, we should test them for stupidity, ignorance, greed and love of power.”
    —P.J. (Patrick Jake)