AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“Man is a stream whose source is hidden. Our being is descending into us from we know not whence. The most exact calculator has no prescience that somewhat incalculable may not balk the very next moment. I am constrained every moment to acknowledge a higher origin for events than the will I call mine.”
—Ralph Waldo Emerson (1803–1882)
“An evil soul producing holy witness
Is like a villain with a smiling cheek,
A goodly apple rotten at the heart.”
—William Shakespeare (1564–1616)
“The real, then, is that which, sooner or later, information and reasoning would finally result in, and which is therefore independent of the vagaries of me and you. Thus, the very origin of the conception of reality shows that this conception essentially involves the notion of a COMMUNITY, without definite limits, and capable of a definite increase of knowledge.”
—Charles Sanders Peirce (1839–1914)
“For all symbols are fluxional; all language is vehicular and transitive, and is good, as ferries and horses are, for conveyance, not as farms and houses are, for homestead.”
—Ralph Waldo Emerson (1803–1882)