AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“The dream of reason produces monsters. Imagination deserted by reason creates impossible, useless thoughts. United with reason, imagination is the mother of all art and the source of all its beauty.”
—Francisco José De Goya Y Lucientes (1746–1828)
“Death is an incident producing clay. Use it, mold it, learn from it.”
—John Gilling, British screenwriter. Dr. Knox (Peter Cushing)
“Men will say that in supporting their wives, in furnishing them with houses and food and clothes, they are giving the women as much money as they could ever hope to earn by any other profession. I grant it; but between the independent wage-earner and the one who is given his keep for his services is the difference between the free-born and the chattel.”
—Elizabeth M. Gilmer (1861–1951)
“Luckless is the country in which the symbols of procreation are the objects of shame, while the agents of destruction are honored! And yet you call that member your pudendum, or shameful part, as if there were anything more glorious than creating life, or anything more atrocious than taking it away.”
—Savinien Cyrano De Bergerac (1619–1655)