Multivariate Normal Distribution - Conditional Distributions

Conditional Distributions

If μ and Σ are partitioned as follows

$\boldsymbol\mu = \begin{bmatrix} \boldsymbol\mu_1 \\ \boldsymbol\mu_2 \end{bmatrix} \quad$ with sizes

$\boldsymbol\Sigma = \begin{bmatrix} \boldsymbol\Sigma_{11} & \boldsymbol\Sigma_{12} \\ \boldsymbol\Sigma_{21} & \boldsymbol\Sigma_{22} \end{bmatrix} \quad$ with sizes

then, the distribution of x₁ conditional on x₂ = a is multivariate normal (x₁|x₂ = a) ~ N(μ, Σ) where

$\bar{\boldsymbol\mu} = \boldsymbol\mu_1 + \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \left( \mathbf{a} - \boldsymbol\mu_2 \right)$

and covariance matrix

$\overline{\boldsymbol\Sigma} = \boldsymbol\Sigma_{11} - \boldsymbol\Sigma_{12} \boldsymbol\Sigma_{22}^{-1} \boldsymbol\Sigma_{21}.$

This matrix is the Schur complement of Σ₂₂ in Σ. This means that to calculate the conditional covariance matrix, one inverts the overall covariance matrix, drops the rows and columns corresponding to the variables being conditioned upon, and then inverts back to get the conditional covariance matrix. Here is the generalized inverse of

Note that knowing that x₂ = a alters the variance, though the new variance does not depend on the specific value of a; perhaps more surprisingly, the mean is shifted by ; compare this with the situation of not knowing the value of a, in which case x₁ would have distribution .

An interesting fact derived in order to prove this result, is that the random vectors and are independent.

The matrix Σ₁₂Σ₂₂−1 is known as the matrix of regression coefficients.

In the bivariate case where x is partitioned into X₁ and X₂, the conditional distribution of X₁ given X₂ is

where is the correlation coefficient between X₁ and X₂.

Read more about this topic: Multivariate Normal Distribution

Famous quotes containing the word conditional:

“Computer mediation seems to bathe action in a more conditional light: perhaps it happened; perhaps it didn’t. Without the layered richness of direct sensory engagement, the symbolic medium seems thin, flat, and fragile.”
—Shoshana Zuboff (b. 1951)