Entropy Rates For Markov Chains
Since a stochastic process defined by a Markov chain that is irreducible and aperiodic has a stationary distribution, the entropy rate is independent of the initial distribution.
For example, for such a Markov chain Yk defined on a countable number of states, given the transition matrix Pij, H(Y) is given by:
where μi is the stationary distribution of the chain.
A simple consequence of this definition is that the entropy rate of an i.i.d. stochastic process has an entropy rate that is the same as the entropy of any individual member of the process.
Read more about this topic: Entropy Rate
Famous quotes containing the words entropy, rates and/or chains:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“Good government cannot be found on the bargain-counter. We have seen samples of bargain-counter government in the past when low tax rates were secured by increasing the bonded debt for current expenses or refusing to keep our institutions up to the standard in repairs, extensions, equipment, and accommodations. I refuse, and the Republican Party refuses, to endorse that method of sham and shoddy economy.”
—Calvin Coolidge (1872–1933)
“Of all men living [priests] are our greatest enemies. If it were possible, they would extinguish the very light of nature, turn the world into a dungeon, and keep mankind for ever in chains and darkness.”
—George Berkeley (1685–1753)