AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“To not know would be a source of pain.”
—Sophocles (497–406/5 B.C.)
“Death is an incident producing clay. Use it, mold it, learn from it.”
—John Gilling, British screenwriter. Dr. Knox (Peter Cushing)
“I have an intense personal interest in making the use of American capital in the development of China an instrument for the promotion of the welfare of China, and an increase in her material prosperity without entanglements or creating embarrassment affecting the growth of her independent political power, and the preservation of her territorial integrity.”
—William Howard Taft (1857–1930)
“I do not deny that there may be other well-founded causes for the hatred which various classes feel toward politicians, but the main one seems to me that politicians are symbols of the fact that every class must take every other class into account.”
—José Ortega Y Gasset (1883–1955)