Singular Value Decomposition - Relation To Eigenvalue Decomposition

Relation To Eigenvalue Decomposition

The singular value decomposition is very general in the sense that it can be applied to any m × n matrix whereas eigenvalue decomposition can only be applied to certain classes of square matrices. Nevertheless, the two decompositions are related.

Given an SVD of M, as described above, the following two relations hold:

M^{*} M = V \Sigma^{*} U^{*}\, U \Sigma V^{*} = V (\Sigma^{*} \Sigma) V^{*}\,
M M^{*} = U \Sigma V^{*} \, V \Sigma^{*} U^{*} = U (\Sigma \Sigma^{*}) U^{*}.\,

The right-hand sides of these relations describe the eigenvalue decompositions of the left-hand sides. Consequently:

  • The columns of V (right-singular vectors) are eigenvectors of
  • The columns of U (left-singular vectors) are eigenvectors of
  • The non-zero elements of Σ (non-zero singular values) are the square roots of the non-zero eigenvalues of or

In the special case that M is a normal matrix, which by definition must be square, the spectral theorem says that it can be unitarily diagonalized using a basis of eigenvectors, so that it can be written for a unitary matrix U and a diagonal matrix D. When M is also positive semi-definite, the decomposition is also a singular value decomposition.

However, the eigenvalue decomposition and the singular value decomposition differ for all other matrices M: the eigenvalue decomposition is where U is not necessarily unitary and D is not necessarily positive semi-definite, while the SVD is where Σ is a diagonal positive semi-definite, and U and V are unitary matrices that are not necessarily related except through the matrix M.

Read more about this topic:  Singular Value Decomposition

Famous quotes containing the words relation to and/or relation:

    “It would be disingenuous, however, not to point out that some things are considered as morally certain, that is, as having sufficient certainty for application to ordinary life, even though they may be uncertain in relation to the absolute power of God.”
    —René Descartes (1596–1650)

    “There is a constant in the average American imagination and taste, for which the past must be preserved and celebrated in full-scale authentic copy; a philosophy of immortality as duplication. It dominates the relation with the self, with the past, not infrequently with the present, always with History and, even, with the European tradition.”
    —Umberto Eco (b. 1932)