Multivariate Normal Distribution - Conditional Distributions

Conditional Distributions

If μ and Σ are partitioned as follows

with sizes

with sizes

then, the distribution of x₁ conditional on x₂ = a is multivariate normal (x₁|x₂ = a) ~ N(μ, Σ) where

and covariance matrix

This matrix is the Schur complement of Σ₂₂ in Σ. This means that to calculate the conditional covariance matrix, one inverts the overall covariance matrix, drops the rows and columns corresponding to the variables being conditioned upon, and then inverts back to get the conditional covariance matrix. Here is the generalized inverse of

Note that knowing that x₂ = a alters the variance, though the new variance does not depend on the specific value of a; perhaps more surprisingly, the mean is shifted by ; compare this with the situation of not knowing the value of a, in which case x₁ would have distribution .

An interesting fact derived in order to prove this result, is that the random vectors and are independent.

The matrix Σ₁₂Σ₂₂−1 is known as the matrix of regression coefficients.

In the bivariate case where x is partitioned into X₁ and X₂, the conditional distribution of X₁ given X₂ is

where is the correlation coefficient between X₁ and X₂.