Kolmogorov Complexity - Definition

Definition

To define the Kolmogorov complexity, we must first specify a description language for strings. Such a description language can be based on any computer programming language, such as Lisp, Pascal, or Java Virtual Machine bytecode. If P is a program which outputs a string x, then P is a description of x. The length of the description is just the length of P as a character string, multiplied by the number of bits in a character (e.g. 7 for ASCII).

We could, alternatively, choose an encoding for Turing machines, where an encoding is a function which associates to each Turing Machine M a bitstring <M>. If M is a Turing Machine which, on input w, outputs string x, then the concatenated string <M> w is a description of x. For theoretical analysis, this approach is more suited for constructing detailed formal proofs and is generally preferred in the research literature. In this article, an informal approach is discussed.

Any string s has at least one description, namely the program:

function GenerateFixedString return s

If a description of s, d(s), is of minimal length (i.e. it uses the fewest number of characters), it is called a minimal description of s. Thus, the length of d(s) (i.e. the number of characters in the description) is the Kolmogorov complexity of s, written K(s). Symbolically,

We now consider how the choice of description language affects the value of K, and show that the effect of changing the description language is bounded.

Theorem: If K1 and K2 are the complexity functions relative to description languages L1 and L2, then there is a constant c – which depends only on the languages L1 and L2 chosen – such that

Proof: By symmetry, it suffices to prove that there is some constant c such that for all bitstrings s

Now, suppose there is a program in the language L1 which acts as an interpreter for L2:

function InterpretLanguage(string p)

where p is a program in L2. The interpreter is characterized by the following property:

Running InterpretLanguage on input p returns the result of running p.

Thus, if P is a program in L2 which is a minimal description of s, then InterpretLanguage(P) returns the string s. The length of this description of s is the sum of

  1. The length of the program InterpretLanguage, which we can take to be the constant c.
  2. The length of P which by definition is K2(s).

This proves the desired upper bound.

See also invariance theorem.

Read more about this topic:  Kolmogorov Complexity

Famous quotes containing the word definition:

    “The man who knows governments most completely is he who troubles himself least about a definition which shall give their essence. Enjoying an intimate acquaintance with all their particularities in turn, he would naturally regard an abstract conception in which these were unified as a thing more misleading than enlightening.”
    —William James (1842–1910)

    “The very definition of the real becomes: that of which it is possible to give an equivalent reproduction.... The real is not only what can be reproduced, but that which is always already reproduced. The hyperreal.”
    —Jean Baudrillard (b. 1929)

    “According to our social pyramid, all men who feel displaced racially, culturally, and/or because of economic hardships will turn on those whom they feel they can order and humiliate, usually women, children, and animals—just as they have been ordered and humiliated by those privileged few who are in power. However, this definition does not explain why there are privileged men who behave this way toward women.”
    —Ana Castillo (b. 1953)