|
Let $x \in \mathbb{R}^n$ and let $f\colon\mathbb{R}^n\to\mathbb{R}$ be a real-valued function having 2nd-order partial derivatives in an open set $U$ containing $x$ . The Hessian matrix of $f$ is the matrix of second partial derivatives evaluated at $x$ :
\begin{equation} \mathbf{H}(x):= \begin{bmatrix} \displaystyle{\frac{\partial^2 f}{\partial x_1^2}} & \displaystyle{\frac{\partial^2 f}{\partial x_1\partial x_2}} & \ldots & \displaystyle{\frac{\partial^2 f}{\partial x_1\partial x_n}} \\ \displaystyle{\frac{\partial^2 f}{\partial x_2\partial x_1}} & \displaystyle{\frac{\partial^2 f}{\partial x_2^2}} & \ldots & \displaystyle{\frac{\partial^2 f}{\partial x_2\partial x_n}} \\ \vdots & \vdots & \ddots & \vdots \\ \displaystyle{\frac{\partial^2 f}{\partial x_n\partial x_1}} & \displaystyle{\frac{\partial^2 f}{\partial x_n\partial x_2}} & \ldots & \displaystyle{\frac{\partial^2 f}{\partial x_n^2}} \end{bmatrix}. \end{equation} If $f$ is in $C^2(U)$ , $\mathbf{H}(x)$ is symmetric because of the equality of mixed partials. Note that $\mathbf{H}(x)=\mathbf{J}(\nabla f)$ , the Jacobian of the gradient of $f$ .
Given a vector $\boldsymbol{v}\in\mathbb{R}^n$ , the Hessian of $f$ at $\boldsymbol{v}$ is: \begin{equation} \mathbf{H}(x)(\boldsymbol{v}):=\frac{1}{2}\boldsymbol{v}\mathbf{H}(x)\boldsymbol{v}^{\operatorname{T}}. \end{equation}Here we view $\boldsymbol{v}$ as a $1$ by $n$ matrix so that $\boldsymbol{v}^{\operatorname{T}}$ is the transpose of $\boldsymbol{v}$ .
Remark. The Hessian of $f$ at $\boldsymbol{v}$ is a quadratic form, since $\mathbf{H}(x)(r\boldsymbol{v})=r^2\mathbf{H}(x)(\boldsymbol{v})$ for any $r\in\mathbb{R}$ .
If $f$ is further assumed to be in $C^2(U)$ , and $x$ is a critical point of $f$ such that $\mathbf{H}(x)$ is positive definite, then $x$ is a strict local minimum of $f$ .
This is not difficult to show. Since $\mathbf{H}(x)$ is positive definite, the Rayleigh-Ritz theorem shows that there is a $c > 0$ such that for all $h \in \mathbb{R}^n$ , $h^T\mathbf{H}(x)h \ge 2c \Vert h\Vert^2$ . Thus by Taylor's theorem (weaker form) $$ f(x + h ) = f(x) + \frac{1}{2} h^T\mathbf{H}(x)h + o(\Vert h \Vert^2) \ge c \Vert h \Vert^2 + o(\Vert h\Vert^2).$$ For small $\Vert h \Vert$ the first term on the right dominates the second, so that both sides are positive for small $\Vert h\Vert$ .
| 
