Relation To Entropy
It can be shown that for the output of Markov information sources, Kolmogorov complexity is related to the entropy of the information source. More precisely, the Kolmogorov complexity of the output of a Markov information source, normalized by the length of the output, converges almost surely (as the length of the output goes to infinity) to the entropy of the source.
Read more about this topic: Kolmogorov Complexity
Famous quotes containing the words relation to, relation and/or entropy:
“It would be disingenuous, however, not to point out that some things are considered as morally certain, that is, as having sufficient certainty for application to ordinary life, even though they may be uncertain in relation to the absolute power of God.”
—René Descartes (1596–1650)
“Skepticism is unbelief in cause and effect. A man does not see, that, as he eats, so he thinks: as he deals, so he is, and so he appears; he does not see that his son is the son of his thoughts and of his actions; that fortunes are not exceptions but fruits; that relation and connection are not somewhere and sometimes, but everywhere and always; no miscellany, no exemption, no anomaly,—but method, and an even web; and what comes out, that was put in.”
—Ralph Waldo Emerson (1803–1882)
“Just as the constant increase of entropy is the basic law of the universe, so it is the basic law of life to be ever more highly structured and to struggle against entropy.”
—Václav Havel (b. 1936)