singular value decomposition

Any real $m\times n$ matrix $A$ can be decomposed into

 $A=USV^{T}$

where $U$ is an $m\times m$ orthogonal matrix, $V$ is an $n\times n$ orthogonal matrix, and $S$ is a unique $m\times n$ diagonal matrix with real, non-negative elements $\sigma_{i}$, $i=1,\ldots,\min(m,n)$ , in descending order:

 $\sigma_{1}\geq\sigma_{2}\geq\dots\geq\sigma_{\min(m,n)}\geq 0$

The $\sigma_{i}$ are the singular values of $A$ and the first $\min(m,n)$ columns of $U$ and $V$ are the left and right (respectively) singular vectors of $A$. $S$ has the form:

 $\begin{bmatrix}\Sigma\\ 0\end{bmatrix}\operatorname{if}m\geq n\;\operatorname{and}\;\begin{bmatrix}% \Sigma&0\end{bmatrix}\operatorname{if}m

where $\Sigma$ is a diagonal matrix with the diagonal elements $\sigma_{1},\sigma_{2},\ldots,\sigma_{\min(m,n)}$. We assume now $m\geq n$. If $r=\operatorname{rank}(A) , then

 $\sigma_{1}\geq\sigma_{2}\geq\cdots\geq\sigma_{r}>\sigma_{r+1}=\cdots=\sigma_{n% }=0.$

If $\sigma_{r}\neq 0$ and $\sigma_{r+1}=\cdots=\sigma_{n}=0$, then $r$ is the rank of $A$. In this case, $S$ becomes an $r\times r$ matrix, and $U$ and $V$ shrink accordingly. SVD can thus be used for rank determination.

The SVD provides a numerically robust solution to the least-squares problem. The matrix-algebraic phrasing of the least-squares solution $x$ is

 $x=(A^{T}A)^{-1}A^{T}b$

Then utilizing the SVD by making the replacement $A=USV^{T}$ we have

 $x=V\begin{bmatrix}\Sigma^{-1}&0\end{bmatrix}U^{T}b.$

References

• Originally from The Data Analysis Briefbook (http://rkb.home.cern.ch/rkb/titleA.htmlhttp://rkb.home.cern.ch/rkb/titleA.html)

 Title singular value decomposition Canonical name SingularValueDecomposition Date of creation 2013-03-22 12:05:17 Last modified on 2013-03-22 12:05:17 Owner akrowne (2) Last modified by akrowne (2) Numerical id 9 Author akrowne (2) Entry type Definition Classification msc 15-00 Classification msc 65-00 Synonym SVD Synonym singular value Synonym singular vector Related topic Eigenvector Related topic Eigenvalue