Principal Component Analysis - Derivation of PCA Using The Covariance Method

Derivation of PCA Using The Covariance Method

Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.

We want to find a orthonormal transformation matrix P so that PX has a diagonal covariant matrix (i.e. PX is a random vector with all its distinct components pairwise uncorrelated).

A quick computation assuming were unitary yields:

\begin{array}{rcl} \operatorname{var}(PX) &= &\mathbb{E}\\ &= &\mathbb{E}\\ &= &P~\mathbb{E}P^{\dagger}\\ &= &P~\operatorname{cov}(X)P^{-1}\\ \end{array}

Hence holds if and only if were diagonalisable by .

This is very constructive, as var(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.

Read more about this topic:  Principal Component Analysis

Famous quotes containing the word method:

    “You know, I have a method all my own. If you’ll notice, the coat came first, then the tie, then the shirt. Now, according to Hoyle, after that the pants should be next. There’s where I’m different. I go for the shoes next. First the right, then the left. After that, it’s every man for himself.”
    —Robert Riskin (1897–1955)