# Hessian matrix

Let $x\in {\mathbb{R}}^{n}$ and let $f:{\mathbb{R}}^{n}\to \mathbb{R}$ be a real-valued function having 2nd-order partial derivatives^{} in an open set $U$ containing $x$. The *Hessian matrix* of $f$ is the matrix of second partial derivatives evaluated at $x$:

$$\mathbf{H}(x):=\left[\begin{array}{cccc}\hfill \frac{{\partial}^{2}f}{\partial {x}_{1}^{2}}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{1}\partial {x}_{2}}\hfill & \hfill \mathrm{\dots}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{1}\partial {x}_{n}}\hfill \\ \hfill \frac{{\partial}^{2}f}{\partial {x}_{2}\partial {x}_{1}}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{2}^{2}}\hfill & \hfill \mathrm{\dots}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{2}\partial {x}_{n}}\hfill \\ \hfill \mathrm{\vdots}\hfill & \hfill \mathrm{\vdots}\hfill & \hfill \mathrm{\ddots}\hfill & \hfill \mathrm{\vdots}\hfill \\ \hfill \frac{{\partial}^{2}f}{\partial {x}_{n}\partial {x}_{1}}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{n}\partial {x}_{2}}\hfill & \hfill \mathrm{\dots}\hfill & \hfill \frac{{\partial}^{2}f}{\partial {x}_{n}^{2}}\hfill \end{array}\right].$$ | (1) |

If $f$ is in ${C}^{2}(U)$, $\mathbf{H}(x)$ is symmetric^{} (http://planetmath.org/SymmetricMatrix) because of the equality of mixed partials. Note that $\mathbf{H}(x)=\mathbf{J}(\nabla f)$, the Jacobian^{} of the gradient^{} of $f$.

Given a vector $\bm{v}\in {\mathbb{R}}^{n}$, the *Hessian* of $f$ at $\bm{v}$ is:

$$\mathbf{H}(x)(\bm{v}):=\frac{1}{2}\bm{v}\mathbf{H}(x){\bm{v}}^{\mathrm{T}}.$$ | (2) |

Here we view $\bm{v}$ as a $1$ by $n$ matrix so that ${\bm{v}}^{\mathrm{T}}$ is the transpose^{} of $\bm{v}$.

Remark. The Hessian of $f$ at $\bm{v}$ is a quadratic form^{}, since $\mathbf{H}(x)(r\bm{v})={r}^{2}\mathbf{H}(x)(\bm{v})$ for any $r\in \mathbb{R}$.

If $f$ is further assumed to be in ${C}^{2}(U)$, and $x$ is a critical point^{} of $f$ such that $\mathbf{H}(x)$ is positive definite^{} (http://planetmath.org/PositiveDefinite), then
$x$ is a strict local minimum of $f$.

This is not difficult to show. Since $\mathbf{H}(x)$ is positive definite (http://planetmath.org/PositiveDefinite), the Rayleigh-Ritz theorem shows that there is a $c>0$ such that for all $h\in {\mathbb{R}}^{n}$, ${h}^{T}\mathbf{H}(x)h\ge 2c{\parallel h\parallel}^{2}$. Thus by Taylor’s theorem (http://planetmath.org/TaylorPolynomialsInBanachSpaces) ( form)

$$f(x+h)=f(x)+\frac{1}{2}{h}^{T}\mathbf{H}(x)h+o({\parallel h\parallel}^{2})\ge c{\parallel h\parallel}^{2}+o({\parallel h\parallel}^{2}).$$ |

For small $\parallel h\parallel $ the first on the the second, so that both sides are positive for small $\parallel h\parallel $.

Title | Hessian matrix |

Canonical name | HessianMatrix |

Date of creation | 2013-03-22 12:59:41 |

Last modified on | 2013-03-22 12:59:41 |

Owner | cvalente (11260) |

Last modified by | cvalente (11260) |

Numerical id | 31 |

Author | cvalente (11260) |

Entry type | Definition |

Classification | msc 26B12 |

Related topic | Gradient |

Related topic | PartialDerivative |

Related topic | SymmetricMatrix |

Related topic | ComplexHessianMatrix |

Related topic | HessianForm |

Related topic | DirectionalDerivative |

Defines | Hessian |