Entropy Rates For Markov Chains
Since a stochastic process defined by a Markov chain that is irreducible and aperiodic has a stationary distribution, the entropy rate is independent of the initial distribution.
For example, for such a Markov chain Yk defined on a countable number of states, given the transition matrix Pij, H(Y) is given by:
where μi is the stationary distribution of the chain.
A simple consequence of this definition is that the entropy rate of an i.i.d. stochastic process has an entropy rate that is the same as the entropy of any individual member of the process.
Read more about this topic: Entropy Rate
Famous quotes containing the words entropy, rates and/or chains:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“One of the most important findings to come out of our research is that being where you want to be is good for you. We found a very strong correlation between preferring the role you are in and well-being. The homemaker who is at home because she likes that “job,” because it meets her own desires and needs, tends to feel good about her life. The woman at work who wants to be there also rates high in well-being.”
—Grace Baruch (20th century)
“Lap me in soft Lydian airs,
Married to immortal verse,
Such as the meeting soul may pierce
In notes with many a winding bout
Of linked sweetness long drawn out,
With wanton heed and giddy cunning,
The melting voice through mazes running,
Untwisting all the chains that tie
The hidden soul of harmony;”
—John Milton (1608–1674)