AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“If, for instance, they have heard something from the postman, they attribute it to “a semi-official statement”; if they have fallen into conversation with a stranger at a bar, they can conscientiously describe him as “a source that has hitherto proved unimpeachable.” It is only when the journalist is reporting a whim of his own, and one to which he attaches minor importance, that he defines it as the opinion of “well-informed circles.””
—Evelyn Waugh (1903–1966)
“Sir, I do not call a gamester a dishonest man; but I call him an unsocial man, an unprofitable man. Gaming is a mode of transferring property without producing any intermediate good.”
—Samuel Johnson (1709–1784)
“It is so rare to meet with a man outdoors who cherishes a worthy thought in his mind, which is independent of the labor of his hands. Behind every man’s busy-ness there should be a level of undisturbed serenity and industry, as within the reef encircling a coral isle there is always an expanse of still water, where the depositions are going on which will finally raise it above the surface.”
—Henry David Thoreau (1817–1862)
“As usual I finish the day before the sea, sumptuous this evening beneath the moon, which writes Arab symbols with phosphorescent streaks on the slow swells. There is no end to the sky and the waters. How well they accompany sadness!”
—Albert Camus (1913–1960)