Singular Value Decomposition - Relation To Eigenvalue Decomposition

Relation To Eigenvalue Decomposition

The singular value decomposition is very general in the sense that it can be applied to any m × n matrix whereas eigenvalue decomposition can only be applied to certain classes of square matrices. Nevertheless, the two decompositions are related.

Given an SVD of M, as described above, the following two relations hold:

The right-hand sides of these relations describe the eigenvalue decompositions of the left-hand sides. Consequently:

• The columns of V (right-singular vectors) are eigenvectors of
• The columns of U (left-singular vectors) are eigenvectors of
• The non-zero elements of Σ (non-zero singular values) are the square roots of the non-zero eigenvalues of or

In the special case that M is a normal matrix, which by definition must be square, the spectral theorem says that it can be unitarily diagonalized using a basis of eigenvectors, so that it can be written for a unitary matrix U and a diagonal matrix D. When M is also positive semi-definite, the decomposition is also a singular value decomposition.

However, the eigenvalue decomposition and the singular value decomposition differ for all other matrices M: the eigenvalue decomposition is where U is not necessarily unitary and D is not necessarily positive semi-definite, while the SVD is where Σ is a diagonal positive semi-definite, and U and V are unitary matrices that are not necessarily related except through the matrix M.