Singular Value Decomposition - Relation To Eigenvalue Decomposition

Relation To Eigenvalue Decomposition

The singular value decomposition is very general in the sense that it can be applied to any m × n matrix whereas eigenvalue decomposition can only be applied to certain classes of square matrices. Nevertheless, the two decompositions are related.

Given an SVD of M, as described above, the following two relations hold:

M^{*} M = V \Sigma^{*} U^{*}\, U \Sigma V^{*} = V (\Sigma^{*} \Sigma) V^{*}\,
M M^{*} = U \Sigma V^{*} \, V \Sigma^{*} U^{*} = U (\Sigma \Sigma^{*}) U^{*}.\,

The right-hand sides of these relations describe the eigenvalue decompositions of the left-hand sides. Consequently:

  • The columns of V (right-singular vectors) are eigenvectors of
  • The columns of U (left-singular vectors) are eigenvectors of
  • The non-zero elements of Σ (non-zero singular values) are the square roots of the non-zero eigenvalues of or

In the special case that M is a normal matrix, which by definition must be square, the spectral theorem says that it can be unitarily diagonalized using a basis of eigenvectors, so that it can be written for a unitary matrix U and a diagonal matrix D. When M is also positive semi-definite, the decomposition is also a singular value decomposition.

However, the eigenvalue decomposition and the singular value decomposition differ for all other matrices M: the eigenvalue decomposition is where U is not necessarily unitary and D is not necessarily positive semi-definite, while the SVD is where Σ is a diagonal positive semi-definite, and U and V are unitary matrices that are not necessarily related except through the matrix M.

Read more about this topic:  Singular Value Decomposition

Famous quotes containing the words relation to and/or relation:

    “A theory of the middle class: that it is not to be determined by its financial situation but rather by its relation to government. That is, one could shade down from an actual ruling or governing class to a class hopelessly out of relation to government, thinking of gov’t as beyond its control, of itself as wholly controlled by gov’t. Somewhere in between and in gradations is the group that has the sense that gov’t exists for it, and shapes its consciousness accordingly.”
    —Lionel Trilling (1905–1975)

    “In relation to God, we are like a thief who has burgled the house of a kindly householder and been allowed to keep some of the gold. From the point of view of the lawful owner this gold is a gift; From the point of view of the burglar it is a theft. He must go and give it back. It is the same with our existence. We have stolen a little of God’s being to make it ours. God has made us a gift of it. But we have stolen it. We must return it.”
    —Simone Weil (1909–1943)