AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“The source of all life and knowledge is in man and woman, and the source of all living is in the interchange and the meeting and mingling of these two: man-life and woman-life, man-knowledge and woman-knowledge, man-being and woman-being.”
—D.H. (David Herbert)
“Most works of art are effectively treated as commodities and most artists, even when they justly claim quite other intentions, are effectively treated as a category of independent craftsmen or skilled workers producing a certain kind of marginal commodity.”
—Raymond Williams (1921–1988)
“I have defeated them all.... I was left with some money to battle with the world when quite young, and at the present time have much to feel proud of.... The Lord gave me talent, and I know I have done good with it.... For my brains have made me quite independent and without the help of any man.”
—Harriet A. Brown, U.S. inventor and educator. As quoted in Feminine Ingenuity, ch. 8, by Anne L. MacDonald (1992)
“Eloquence must be grounded on the plainest narrative. Afterwards, it may warm itself until it exhales symbols of every kind and color, speaks only through the most poetic forms; but first and last, it must still be at bottom a biblical statement of fact.”
—Ralph Waldo Emerson (1803–1882)