Motivation
In information theory, the Kraft–McMillan theorem establishes that any directly decodable coding scheme for coding a message to identify one value xi out of a set of possibilities X can be seen as representing an implicit probability distribution q(xi) = 2−li over X, where li is the length of the code for xi in bits. Therefore, Cross entropy can be interpreted as the expected message-length per datum when a wrong distribution Q is assumed however the data actually follows a distribution P - that's why the expectation is taken over the probability distribution P and not Q.
Read more about this topic: Cross Entropy
Famous quotes containing the word motivation:
“Self-determination has to mean that the leader is your individual gut, and heart, and mind or we’re talking about power, again, and its rather well-known impurities. Who is really going to care whether you live or die and who is going to know the most intimate motivation for your laughter and your tears is the only person to be trusted to speak for you and to decide what you will or will not do.”
—June Jordan (b. 1939)