Derivation of PCA Using The Covariance Method
Let X be a d-dimensional random vector expressed as column vector. Without loss of generality, assume X has zero mean.
We want to find a orthonormal transformation matrix P so that PX has a diagonal covariant matrix (i.e. PX is a random vector with all its distinct components pairwise uncorrelated).
A quick computation assuming were unitary yields:
Hence holds if and only if were diagonalisable by .
This is very constructive, as var(X) is guaranteed to be a non-negative definite matrix and thus is guaranteed to be diagonalisable by some unitary matrix.
Read more about this topic: Principal Component Analysis
Famous quotes containing the word method:
“in the absence of feet, a method of conclusions;
a knowledge of principles,
in the curious phenomenon of your occipital horn.”
—Marianne Moore (18871972)