Law of Large Numbers

In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.

For example, a single roll of a six-sided die produces one of the numbers 1, 2, 3, 4, 5, or 6, each with equal probability. Therefore, the expected value of a single die roll is

According to the law of large numbers, if a large number of six-sided dice are rolled, the average of their values (sometimes called the sample mean) is likely to be close to 3.5, with the accuracy increasing as more dice are rolled.

It follows from the law of large numbers that the empirical probability of success in a series of Bernoulli trials will converge to the theoretical probability. For a Bernoulli random variable, the expected value is the theoretical probability of success, and the average of n such variables (assuming they are independent and identically distributed (i.i.d.)) is precisely the relative frequency.

For example, a fair coin toss is a Bernoulli trial. When a fair coin is flipped once, the theoretical probability that the outcome will be heads is equal to 1/2. Therefore, according to the law of large numbers, the proportion of heads in a "large" number of coin flips "should be" roughly 1/2. In particular, the proportion of heads after n flips will almost surely converge to 1/2 as n approaches infinity.

Though the proportion of heads (and tails) approaches 1/2, almost surely the absolute (nominal) difference in the number of heads and tails will become large as the number of flips becomes large. That is, the probability that the absolute difference is a small number, approaches zero as the number of flips becomes large. Also, almost surely the ratio of the absolute difference to the number of flips will approach zero. Intuitively, expected absolute difference grows, but at a slower rate than the number of flips, as the number of flips grows.

The LLN is important because it "guarantees" stable long-term results for random events. For example, while a casino may lose money in a single spin of the roulette wheel, its earnings will tend towards a predictable percentage over a large number of spins. Any winning streak by a player will eventually be overcome by the parameters of the game. It is important to remember that the LLN only applies (as the name indicates) when a large number of observations are considered. There is no principle that a small number of observations will converge to the expected value or that a streak of one value will immediately be "balanced" by the others. See the Gambler's fallacy.

Read more about Law Of Large Numbers:  History, Forms, Proof

Famous quotes containing the words law of, law, large and/or numbers:

    Nor has science sufficient humanity, so long as the naturalist overlooks the wonderful congruity which subsists between man and the world; of which he is lord, not because he is the most subtile inhabitant, but because he is its head and heart, and finds something of himself in every great and small thing, in every mountain stratum, in every new law of color, fact of astronomy, or atmospheric influence which observation or analysis lay open.
    Ralph Waldo Emerson (1803–1882)

    A quality is something capable of being completely embodied. A law never can be embodied in its character as a law except by determining a habit. A quality is how something may or might have been. A law is how an endless future must continue to be.
    Charles Sanders Peirce (1839–1914)

    ... a large portion of those who demand woman suffrage are persons who have not been trained to reason, and are chiefly guided by their generous sensibilities.
    Catherine E. Beecher (1800–1878)

    The forward Youth that would appear
    Must now forsake his Muses dear,
    Nor in the Shadows sing
    His Numbers languishing.
    Andrew Marvell (1621–1678)