AEP For Non-stationary Discrete-time Source Producing Independent Symbols
The assumptions of stationarity/ergodicity/identical distribution of random variables is not essential for the AEP to hold. Indeed, as is quite clear intuitively, the AEP requires only some form of the law of large numbers to hold, which is fairly general. However, the expression needs to be suitably generalized, and the conditions need to be formulated precisely.
We assume that the source is producing independent symbols, with possibly different output statistics at each instant. We assume that the statistics of the process are known completely, that is, the marginal distribution of the process seen at each time instant is known. The joint distribution is just the product of marginals. Then, under the condition (which can be relaxed) that for all i, for some M>0, the following holds (AEP):
where
Read more about this topic: Asymptotic Equipartition Property
Famous quotes containing the words source, producing, independent and/or symbols:
“Diligence is the basis of wealth, and thrift the source of riches.”
—Chinese proverb.
“The writer who aims at producing the platitudes which are “not for an age, but for all time” has his reward in being unreadable in all ages.... The man who writes about himself and his own time is the only sort of man who writes about all people and about all time.”
—George Bernard Shaw (1856–1950)
“Nevertheless, in the Lord woman is not independent of man or man independent of woman. For just as woman came from man, so man comes through woman; but all things come from God.”
—Bible: New Testament, 1 Corinthians 11:11.
In v. 9, Paul wrote “Neither was man created for woman, but woman for man.”
“There are those who would keep us slipping back into the darkness of division, into the snake pit of racial hatred, of racial antagonism and of support for symbols of the struggle to keep African-Americans in bondage.”
—Carol Moseley-Braun (b. 1947)