|
|
|
Viewing Version
18
of
'Hessian matrix'
|
[ view 'Hessian matrix'
|
back to history
]
| Title of object: |
Hessian matrix |
| Canonical Name: |
HessianMatrix |
| Type: |
Definition |
| Created on: |
2002-08-28 01:14:14 |
| Modified on: |
2006-08-05 16:08:57 |
| Classification: |
msc:26B12 |
| Keywords: |
second derivative |
| Defines: |
Hessian |
Preamble:
% this is the default PlanetMath preamble. as your knowledge
% of TeX increases, you will probably want to edit this, but
% it should be fine as is for beginners.
% almost certainly you want these
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsfonts}
% used for TeXing text within eps files
%\usepackage{psfrag}
% need this for including graphics (\includegraphics)
%\usepackage{graphicx}
% for neatly defining theorems and propositions
%\usepackage{amsthm}
% making logically defined graphics
%\usepackage{xypic}
% there are many more packages, add them here as you need them
% define commands here |
Content:
Let $f\colon\mathbb{R}^n\to\mathbb{R}$ be a real-valued function having 2nd-order partial derivatives. The \emph{Hessian matrix} of $f$ is the matrix of second partial derivatives:
\begin{equation}
\mathbf{H}(f):=
\begin{bmatrix}
\frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1\partial x_2} & \ldots & \frac{\partial^2 f}{\partial x_1\partial x_n}
\\ \frac{\partial^2 f}{\partial x_2\partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \ldots & \frac{\partial^2 f}{\partial x_2\partial x_n}
\\ \vdots & \vdots & \ddots & \vdots
\\ \frac{\partial^2 f}{\partial x_n\partial x_1} & \frac{\partial^2 f}{\partial x_n\partial x_2} & \ldots & \frac{\partial^2 f}{\partial x_n^2}
\end{bmatrix}
\end{equation}
If $f$ is in $C^2$, $\mathbf{H}(f)$ is \PMlinkname{symmetric}{SymmetricMatrix} because of the equality of mixed partials. Note that $\mathbf{H}(f)=\mathbf{J}(\nabla f)$, the Jacobian of the gradient of $f$.
Given a vector $\boldsymbol{v}\in\mathbb{R}^n$, the \emph{Hessian} of $f$ at $\boldsymbol{v}$ is:
\begin{equation}
\mathbf{H}(f)(\boldsymbol{v}):=\frac{1}{2}\boldsymbol{v}\mathbf{H}(f)\boldsymbol{v}^{\operatorname{T}}
\end{equation}
Here we view $\boldsymbol{v}$ as a $1$ by $n$ matrix so that $\boldsymbol{v}^{\operatorname{T}}$ is the transpose of $\boldsymbol{v}$.
\textbf{Remark}. The Hessian of $f$ at $\boldsymbol{v}$ is a quadratic form, since $\mathbf{H}(f)(r\boldsymbol{v})=r^2\mathbf{H}(f)(\boldsymbol{v})$ for any $r\in\mathbb{R}$.
If $f$ is further assumed to be in $C^2$, and $\boldsymbol{v}$ is a critical point of $f$ such that $\mathbf{H}(f)(\boldsymbol{v})$ is positive definite, then
$\boldsymbol{v}$ is a strict local minimum of $f$.
This is not difficult to show. Since $\mathbf{H}(f)(\boldsymbol{v})$ is positive definite, the Rayleigh-Ritz theorem shows that there is a $c > 0$ such that for all $w \in \mathbb{R}^n$
$w^T\mathbf{H}(f)(\boldsymbol{v})w \ge 2c \Vert w \Vert^2$. Thus by
\PMlinkname{Taylor's theorem}{TaylorPolynomialsInBanachSpaces} (\PMlinkescapetext{weaker} form)
$$
f(v + w ) = f(v) + \frac{1}{2} w^T\mathbf{H}(f)(\boldsymbol{v})w + o(\Vert w \Vert^2) \ge c \Vert w \Vert^2 + o(\Vert w \Vert^2).$$
For small $\Vert w \Vert$ the first term on the right dominates the second, so that both sides are positive for small $\Vert w\Vert$. |
|
|
|
|
|
