Probability Theory - Law of Large Numbers

Law of Large Numbers

Common intuition suggests that if a fair coin is tossed many times, then roughly half of the time it will turn up heads, and the other half it will turn up tails. Furthermore, the more often the coin is tossed, the more likely it should be that the ratio of the number of heads to the number of tails will approach unity. Modern probability provides a formal version of this intuitive idea, known as the law of large numbers. This law is remarkable because it is not assumed in the foundations of probability theory, but instead emerges out of these foundations as a theorem. Since it links theoretically derived probabilities to their actual frequency of occurrence in the real world, the law of large numbers is considered as a pillar in the history of statistical theory and has had widespread influence.

The law of large numbers (LLN) states that the sample average

of a sequence of independent and identically distributed random variables converges towards their common expectation, provided that the expectation of is finite.

It is in the different forms of convergence of random variables that separates the weak and the strong law of large numbers

\begin{array}{lll} \text{Weak law:} & \overline{X}_n \, \xrightarrow{P} \, \mu & \text{for } n \to \infty \\ \text{Strong law:} & \overline{X}_n \, \xrightarrow{\mathrm{a.\,s.}} \, \mu & \text{for } n \to \infty . \end{array}

It follows from the LLN that if an event of probability p is observed repeatedly during independent experiments, the ratio of the observed frequency of that event to the total number of repetitions converges towards p.

For example, if are independent Bernoulli random variables taking values 1 with probability p and 0 with probability 1-p, then for all i, so that converges to p almost surely.

Read more about this topic:  Probability Theory

Famous quotes containing the words law of, law, large and/or numbers:

    “The greatest step forward would be to see that everything factual is already theory. The blueness of the sky reveals the basic law of chromatics. Don’t look for anything behind the phenomena, they themselves are the doctrine.”
    —Johann Wolfgang Von Goethe (1749–1832)

    “America has always been a country of amateurs where the professional, that is to say, the man who claims authority as a member of an élite which knows the law in some field or other, is an object of distrust and resentment.”
    —W.H. (Wystan Hugh)

    “Switzerland is a small, steep country, much more up and down than sideways, and is all stuck over with large brown hotels built on the cuckoo clock style of architecture.”
    —Ernest Hemingway (1899–1961)

    “The principle of majority rule is the mildest form in which the force of numbers can be exercised. It is a pacific substitute for civil war in which the opposing armies are counted and the victory is awarded to the larger before any blood is shed. Except in the sacred tests of democracy and in the incantations of the orators, we hardly take the trouble to pretend that the rule of the majority is not at bottom a rule of force.”
    —Walter Lippmann (1889–1974)