Principal Component Analysis - Derivation of PCA Using The Covariance Method

Derivation of PCA Using The Covariance Method

Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.

We want to find a orthonormal transformation matrix P so that PX has a diagonal covariant matrix (i.e. PX is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming were unitary yields:

\begin{array}{rcl} \operatorname{var}(PX) &= &\mathbb{E}\\ &= &\mathbb{E}\\ &= &P~\mathbb{E}P^{\dagger}\\ &= &P~\operatorname{cov}(X)P^{-1}\\ \end{array}

Hence holds if and only if were diagonalisable by .

This is very constructive, as var(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.

Read more about this topic:  Principal Component Analysis

Famous quotes containing the word method:

    “in the absence of feet, “a method of conclusions”;
    “a knowledge of principles,”
    in the curious phenomenon of your occipital horn.”
    —Marianne Moore (1887–1972)