AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“Concupiscence and force are the source of all our actions; concupiscence causes voluntary actions, force involuntary ones.”
—Blaise Pascal (1623–1662)
“Against war one might say that it makes the victor stupid and the vanquished malicious. In its favor, that in producing these two effects it barbarizes, and so makes the combatants more natural. For culture it is a sleep or a wintertime, and man emerges from it stronger for good and for evil.”
—Friedrich Nietzsche (1844–1900)
“Whether changes in the sibling relationship during adolescence create long-term rifts that spill over into adulthood depends upon the ability of brothers and sisters to constantly redefine their connection. Siblings either learn to accept one another as independent individuals with their own sets of values and behaviors or cling to the shadow of the brother and sister they once knew.”
—Jane Mersky Leder (20th century)
“The use of symbols has a certain power of emancipation and exhilaration for all men. We seem to be touched by a wand, which makes us dance and run about happily, like children. We are like persons who come out of a cave or cellar into the open air. This is the effect on us of tropes, fables, oracles, and all poetic forms. Poets are thus liberating gods.”
—Ralph Waldo Emerson (1803–1882)