In the mathematical theory of probability, the entropy rate or source information rate of a stochastic process is, informally, the time density of the average information in a stochastic process. For stochastic processes with a countable index, the entropy rate H(X) is the limit of the joint entropy of n members of the process Xk divided by n, as n tends to infinity:
when the limit exists. An alternative, related quantity is:
For strongly stationary stochastic processes, . The entropy rate can be thought of as a general property of stochastic sources; this is the asymptotic equipartition property.
Read more about Entropy Rate: Entropy Rates For Markov Chains, Example
Famous quotes containing the words entropy and/or rate:
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)
“As a novelist, I cannot occupy myself with “characters,” or at any rate central ones, who lack panache, in one or another sense, who would be incapable of a major action or a major passion, or who have not a touch of the ambiguity, the ultimate unaccountability, the enlarging mistiness of persons “in history.” History, as more austerely I now know it, is not romantic. But I am.”
—Elizabeth Bowen (1899–1973)