Matrix Condition Number

The condition number for matrix inversion with respect to a matrix norm $\norm{\cdot}$ of a square matrix $A$ is defined by $$ \kappa(A) = \Vert A \Vert \Vert A^{-1} \Vert\,, $$ if $A$ is non-singular; and $\kappa(A) = +\infty$ if $A$ is singular.

The condition number is a measure of stability or sensitivity of a matrix (or the linear system it represents) to numerical operations. In other words, we may not be able to trust the results of computations on an ill-conditioned matrix.

Matrices with condition numbers near 1 are said to be well-conditioned. Matrices with condition numbers much greater than one (such as around $10^5$ for a $5 \times 5$ Hilbert matrix) are said to be ill-conditioned.

If $\kappa(A)$ is the condition number of $A$ then $\kappa(A)$ measures a sort of inverse distance from $A$ to the set of singular matrices, normalized by $\norm{A}$ Precisely, if $A$ is invertible, and $\norm{B - A} < \norm{A^{-1}}^{-1}$ then $B$ must also be invertible. On the other hand, in the case of the $2$ norm, there always exists a singular matrix $B$ such that $\norm{B-A}_2 = \norm{A^{-1}}_2^{-1}$ (so the distance estimate is sharp).

Bibliography

1

Golub and Van Loan. Matrix Computations, 3rd edition. Johns Hopkins University Press, 1996.