AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“Because relationships are a primary source of self-esteem for girls and women, daughters need to know they will not lose our love if they speak up for what they want to tell us how they feel about things. . . . Teaching girls to make specific requests, rather than being indirect and agreeable, will help them avoid the pitfalls of having to be manipulative and calculating to get what they want.”
—Jeanne Elium (20th century)
“Sir, I do not call a gamester a dishonest man; but I call him an unsocial man, an unprofitable man. Gaming is a mode of transferring property without producing any intermediate good.”
—Samuel Johnson (1709–1784)
“We are independent of the change we detect. The longer the lever, the less perceptible its motion. It is the slowest pulsation which is the most vital. The hero then will know how to wait, as well as to make haste. All good abides with him who waiteth wisely; we shall sooner overtake the dawn by remaining here than by hurrying over the hills of the west.”
—Henry David Thoreau (1817–1862)
“My image is a statement of the symbols of the harsh, impersonal products and brash materialistic objects on which America is built today. It is a projection of everything that can be bought and sold, the practical but impermanent symbols that sustain us.”
—Andy Warhol (1928–1987)