Principle of Indifference - Application To Continuous Variables

Application To Continuous Variables

Applying the principle of indifference incorrectly can easily lead to nonsensical results, especially in the case of multivariate, continuous variables. A typical case of misuse is the following example.

• Suppose there is a cube hidden in a box. A label on the box says the cube has a side length between 3 and 5 cm.
• We don't know the actual side length, but we might assume that all values are equally likely and simply pick the mid-value of 4 cm.
• The information on the label allows us to calculate that the surface area of the cube is between 54 and 150 cm². We don't know the actual surface area, but we might assume that all values are equally likely and simply pick the mid-value of 102 cm².
• The information on the label allows us to calculate that the volume of the cube is between 27 and 125 cm3. We don't know the actual volume, but we might assume that all values are equally likely and simply pick the mid-value of 76 cm3.
• However, we have now reached the impossible conclusion that the cube has a side length of 4 cm, a surface area of 102 cm², and a volume of 76 cm3!

In this example, mutually contradictory estimates of the length, surface area, and volume of the cube arise because we have assumed three mutually contradictory distributions for these parameters: a uniform distribution for any one of the variables implies a non-uniform distribution for the other two. (The same paradox arises if we make it discrete: the side is either exactly 3 cm, 4 cm, or 5 cm, mutatis mutandis.) In general, the principle of indifference does not indicate which variable (e.g. in this case, length, surface area, or volume) is to have a uniform epistemic probability distribution.

Another classic example of this kind of misuse is Bertrand's paradox. Edwin T. Jaynes introduced the principle of transformation groups, which can yield an epistemic probability distribution for this problem. This generalises the principle of indifference, by saying that one is indifferent between equivalent problems rather than indifference between propositions. This still reduces to the "ordinary" principle of indifference when one considers a "permutation" of the labels as generating equivalent problems (i.e. using the permutation transformation group). To apply this to the above box example, we have three problems, with no reason to think one problem is "our problem" more than any other - we are indifferent between each. If we have no reason to favour one over the other, then our prior probabilities must be related by the rule for changing variables in continuous distributions. Let L be the length, and V be the volume. Then we must have

Which has a general solution: Where K is an arbitrary constant, determined by the range of L, in this case equal to:

To put this "to the test", we ask for the probability that the length is less than 4. This has probability of:

.

For the volume, this should be equal to the probability that the volume is less than 43 = 64. The pdf of the volume is

.

And then probability of volume less than 64 is

.

Thus we have achieve invariance with respect to volume and length. You can also show the same invariance with respect to surface area being less than 6(42) = 96. However, note that this probability assignment is not necessarily a "correct" one. For the exact distribution of lengths, volume, or surface area will depend on how the "experiment" is conducted. This probability assignment is very similar to the maximum entropy one, in that the frequency distribution corresponding to the above probability distribution is the most likely to be seen. So, if one was to go to N people individually and simply say "make me a box somewhere between 3 and 5 cm, or a volume between 27 and 125 cm, or a surface area between 54 and 150 cm", then unless there is a systematic influence on how they make the boxes (e.g. they form a group, and choose one particular method of making boxes), about 56% of the boxes will be less than 4 cm - and it will get very close to this amount very quickly. So, for large N, any deviation from this basically indicates the makers of the boxes were "systematic" in how the boxes were made.

The fundamental hypothesis of statistical physics, that any two microstates of a system with the same total energy are equally probable at equilibrium, is in a sense an example of the principle of indifference. However, when the microstates are described by continuous variables (such as positions and momenta), an additional physical basis is needed in order to explain under which parameterization the probability density will be uniform. Liouville's theorem justifies the use of canonically conjugate variables, such as positions and their conjugate momenta.