Units of Information - Primary Units

Primary Units

In 1928, Ralph Hartley observed a fundamental storage principle, which was further formalized by Claude Shannon in 1945: the information that can be stored in a system is proportional to the logarithm logb N of the number N of possible states of that system. Changing the basis of the logarithm from b to a different number c has the effect of multiplying the value of the logarithm by a fixed constant, namely logc N = (logc b) logb N. Therefore, the choice of the basis b determines the unit used to measure information. In particular, if b is a positive integer, then the unit is the amount of information that can be stored in a system with b possible states.

When b is 2, the unit is the "bit" (a contraction of binary digit). A system with 8 possible states, for example, can store up to log28 = 3 bits of information. Other units that have been named include:

  • Base b = 3: the unit is called "trit", and is equal to log2 3 (≈ 1.585) bits.
  • Base b = 10: the unit is called decimal digit, Hartley, ban, decit, or dit, and is equal to log2 10 (≈ 3.322) bits.
  • Base b = e, the base of natural logarithms: the unit is called a nat, nit, or nepit (from Neperian), and is worth log2 e (≈ 1.443) bits.

The trit, ban, and nat are rarely used to measure storage capacity; but the nat, in particular, is often used in information theory, because natural logarithms are sometimes easier to handle than logarithms in other bases.

Read more about this topic:  Units Of Information

Famous quotes containing the words primary and/or units:

    “A fact is a proposition of which the verification by an appeal to the primary sources of our knowledge or to experience is direct and simple. A theory, on the other hand, if true, has all the characteristics of a fact except that its verification is possible only by indirect, remote, and difficult means.”
    —Chauncey Wright (1830–1875)

    “Even in harmonious families there is this double life: the group life, which is the one we can observe in our neighbour’s household, and, underneath, another—secret and passionate and intense—which is the real life that stamps the faces and gives character to the voices of our friends. Always in his mind each member of these social units is escaping, running away, trying to break the net which circumstances and his own affections have woven about him.”
    —Willa Cather (1873–1947)