singular value decomposition

Any real $m\times n$ matrix $A$ can be decomposed into

$$A=US{V}^T$$

where $U$ is an $m\times m$ orthogonal matrix, $V$ is an $n\times n$ orthogonal matrix, and $S$ is a unique $m\times n$ diagonal matrix with real, non-negative elements ${\sigma }_i$, $i=1,\mathrm{\dots },\min (m,n)$ , in descending order:

$${\sigma }_1\ge {\sigma }_2\ge \mathrm{\dots }\ge {\sigma }_{\min (m,n)}\ge 0$$

The ${\sigma }_i$ are the singular values of $A$ and the first $\min (m,n)$ columns of $U$ and $V$ are the left and right (respectively) singular vectors of $A$. $S$ has the form:

where $\mathrm{\Sigma }$ is a diagonal matrix with the diagonal elements ${\sigma }_1,{\sigma }_2,\mathrm{\dots },{\sigma }_{\min (m,n)}$. We assume now $m\ge n$. If , then

$${\sigma }_1\ge {\sigma }_2\ge \mathrm{\cdots }\ge {\sigma }_r>{\sigma }_{r+1}=\mathrm{\cdots }={\sigma }_n=0.$$

If ${\sigma }_r\ne 0$ and ${\sigma }_{r+1}=\mathrm{\cdots }={\sigma }_n=0$, then $r$ is the rank of $A$. In this case, $S$ becomes an $r\times r$ matrix, and $U$ and $V$ shrink accordingly. SVD can thus be used for rank determination.

The SVD provides a numerically robust solution to the least-squares problem. The matrix-algebraic phrasing of the least-squares solution $x$ is

$$x={(A^{T}A)}^{-1}{A}^{T}b$$

Then utilizing the SVD by making the replacement $A=US{V}^T$ we have

References

• Originally from The Data Analysis Briefbook (http://rkb.home.cern.ch/rkb/titleA.htmlhttp://rkb.home.cern.ch/rkb/titleA.html)

Title	singular value decomposition
Canonical name	SingularValueDecomposition
Date of creation	2013-03-22 12:05:17
Last modified on	2013-03-22 12:05:17
Owner	akrowne (2)
Last modified by	akrowne (2)
Numerical id	9
Author	akrowne (2)
Entry type	Definition
Classification	msc 15-00
Classification	msc 65-00
Synonym	SVD
Synonym	singular value
Synonym	singular vector
Related topic	Eigenvector
Related topic	Eigenvalue