Singular Value Decomposition - Relation To Eigenvalue Decomposition

Relation To Eigenvalue Decomposition

The singular value decomposition is very general in the sense that it can be applied to any m × n matrix whereas eigenvalue decomposition can only be applied to certain classes of square matrices. Nevertheless, the two decompositions are related.

Given an SVD of M, as described above, the following two relations hold:

M^{*} M = V \Sigma^{*} U^{*}\, U \Sigma V^{*} = V (\Sigma^{*} \Sigma) V^{*}\,
M M^{*} = U \Sigma V^{*} \, V \Sigma^{*} U^{*} = U (\Sigma \Sigma^{*}) U^{*}.\,

The right-hand sides of these relations describe the eigenvalue decompositions of the left-hand sides. Consequently:

  • The columns of V (right-singular vectors) are eigenvectors of
  • The columns of U (left-singular vectors) are eigenvectors of
  • The non-zero elements of Σ (non-zero singular values) are the square roots of the non-zero eigenvalues of or

In the special case that M is a normal matrix, which by definition must be square, the spectral theorem says that it can be unitarily diagonalized using a basis of eigenvectors, so that it can be written for a unitary matrix U and a diagonal matrix D. When M is also positive semi-definite, the decomposition is also a singular value decomposition.

However, the eigenvalue decomposition and the singular value decomposition differ for all other matrices M: the eigenvalue decomposition is where U is not necessarily unitary and D is not necessarily positive semi-definite, while the SVD is where Σ is a diagonal positive semi-definite, and U and V are unitary matrices that are not necessarily related except through the matrix M.

Read more about this topic:  Singular Value Decomposition

Famous quotes containing the words relation to and/or relation:

    “We must get back into relation, vivid and nourishing relation to the cosmos and the universe. The way is through daily ritual, and is an affair of the individual and the household, a ritual of dawn and noon and sunset, the ritual of the kindling fire and pouring water, the ritual of the first breath, and the last.”
    —D.H. (David Herbert)

    “Parents ought, through their own behavior and the values by which they live, to provide direction for their children. But they need to rid themselves of the idea that there are surefire methods which, when well applied, will produce certain predictable results. Whatever we do with and for our children ought to flow from our understanding of and our feelings for the particular situation and the relation we wish to exist between us and our child.”
    —Bruno Bettelheim (20th century)