Bernoulli Process - Formal Definition

Formal Definition

The Bernoulli process can be formalized in the language of probability spaces as a random sequence in of a single random variable that can take values of heads or tails.

Specifically, one considers the countably infinite direct product of copies of . It is common to examine either the one-sided set or the two-sided set . There is a natural topology on this space, called the product topology. The sets in this topology are finite sequences of coin flips, that is, finite-length strings of H and T, with the rest of (infinitely long) sequence taken as "don't care". These sets of finite sequences are referred to as cylinder sets in the product topology. The set of all such strings form a sigma algebra, specifically, a Borel algebra. This algebra is then commonly written as where the elements of are the finite-length sequences of coin flips (the cylinder sets). Note that the stress here is on finite length: the infinite-length sequences of coin-flips are excluded from the product topology; that this is a reasonable thing to do will become clear below.

If the chances of flipping heads or tails are given by the probabilities, then one can define a natural measure on the product space, given by (or by for the two-sided process). Given a cylinder set, that is, a specific sequence of coin flip results at times, the probability of observing this particular sequence is given by

where k is the number of times that H appears in the sequence, and n-k is the number of times that T appears in the sequence. There are several different kinds of notations for the above; a common one is to write

where each is a binary-valued random variable. It is common to write for . This probability P is commonly called the Bernoulli measure.

Note that the probability of any specific, infinitely long sequence of coin flips is exactly zero; this is because, for any . One says that any given infinite sequence has measure zero. Thus, infinite sequences of coin-flips are simply not needed to discuss the Bernoulli process, and it is for this reason that the product topology explicitly excludes them: it is the coarsest topology that allows the discussion of coin-flips. (Finer topologies, which do allow infinite sequences, can, in fact, lead to certain kinds of confusion and seeming paradoxes; see e.g. strong topology). Nevertheless, one can still say that some classes of infinite sequences of coin flips are far more likely than others, this is given by the asymptotic equipartition property.

To conclude the formal definition, a Bernoulli process is then given by the probability triple, as defined above.