tests for local extrema in Lagrange multiplier method

Let $U$ be open in ${ℝ}^n$, and $f:U\to ℝ$, $g:U\to {ℝ}^m$ be twice continuously differentiable functions. Assume that $p\in U$ is a stationary point for $f$ on $M=g^{-1}(\{0\})$, and $\mathrm{D}g$ has full rank everywhere on $M$. Then we know that $p$ is the solution to the Lagrange multiplier system

$$\mathrm{D}f(p)=\lambda \cdot \mathrm{D}g(p),$$ (1)

for a Lagrange multiplier vector $\lambda =({\lambda }_1,\mathrm{\dots },{\lambda }_m)$.

Our aim is to develop an analogue of the second derivative test for the stationary point $p$.

The most straightforward way to proceed is to consider a coordinate chart $\alpha :V\to M$ for the manifold $M$, and consider the Hessian of the function $f\circ \alpha :V\to {ℝ}^n$ at $0={\alpha }^{-1}(p)$. This Hessian is in fact just the Hessian form of $f:M\to ℝ$ expressed in the coordinates of the chart $\alpha$. But the whole point of using Lagrange multipliers is to avoid calculating coordinate charts directly, so we find an equivalent expression for ${\mathrm{D}}^2(f\circ \alpha )(0)$ in terms of ${\mathrm{D}}^{2}f(p)$ without mentioning derivatives of $\alpha$.

To do this, we differentiate $f\circ \alpha$ twice using the chain rule and product rule. To reduce clutter, from now on we use the prime notation for derivatives rather than $\mathrm{D}$.

(2)

If we interpret ${(f\circ \alpha )}^{\prime \prime }(0)$ as a bilinear mapping of vectors $u,v\in {ℝ}^{n-m}$, then formula (2) really means

$${(f\circ \alpha )}^{\prime \prime }(0)\cdot (u,v)=f^{\prime \prime }(p)\cdot ({\alpha }^{\prime }(0)\cdot u,{\alpha }^{\prime }(0)\cdot v)+f^{\prime }(p)\cdot \left({\alpha }^{\prime \prime }(0)\cdot (u,v)\right).$$ (3)

To obtain the quadratic form, we set $v=u$; also we abbreviate the vector ${\alpha }^{\prime }(0)\cdot u$ by $h$, which belongs to the tangent space ${\mathrm{T}}_{p}M$ of $M$ at $p$. So,

$${(f\circ \alpha )}^{\prime \prime }(0)\cdot u^2=f^{\prime \prime }(p)\cdot h^2+f^{\prime }(p)\cdot \left({\alpha }^{\prime \prime }(0)\cdot u^2\right).$$ (4)

Naïvely, we might think that ${(f\circ \alpha )}^{\prime \prime }(0)$ is simply $f^{\prime \prime }(p)$ restricted to the tangent space ${\mathrm{T}}_{p}M$. This happens to be the first term in (4), but there is also an additional contribution by the second term involving ${\alpha }^{\prime \prime }(0)$; intuitively, ${\alpha }^{\prime \prime }(0)$ is the curvature of the surface (manifold) $M$, “changing the geometry” of the graph of $f$.

But the second term of (4) still involves $\alpha$. To eliminate it, we differentiate the equation $g\circ \alpha =0$ twice.

$$0={(g\circ \alpha )}^{\prime \prime }(0)=g^{\prime \prime }(p)\cdot {\alpha }^{\prime }(0)\cdot {\alpha }^{\prime }(0)+g^{\prime }(p)\cdot {\alpha }^{\prime \prime }(0).$$ (5)

(It is derived the same way as (2) but with $f$ replaced by $g$.) Now we can substitute (5) and (1) in (2) to eliminate the term $f^{\prime }(p)\cdot {\alpha }^{\prime \prime }(0)$:

(6)

or expressed as a quadratic form,

$${(f\circ \alpha )}^{\prime \prime }(0)\cdot u^2=f^{\prime \prime }(p)\cdot h^2-\lambda \cdot g^{\prime \prime }(p)\cdot h^2.$$ (7)

Thus, to understand the nature of the stationary point $p$, we can study the modified Hessian:

$$f^{\prime \prime }(p)-\lambda \cdot g^{\prime \prime }(p),\text{ restricted to }{\mathrm{T}}_{p}M\text{. }$$ (8)

For example, if this bilinear form is positive definite, then $p$ is a local minimum, and if it is negative definite, then $p$ is a local maximum, and so on. All the tests that apply to the usual Hessian in ${ℝ}^n$ apply to the modified Hessian (8).

In coordinates of ${ℝ}^n$, the modified Hessian (8) takes the form

$$\sum _{i=1}^n\sum _{j=1}^n\left({\frac{{\partial }^{2}f}{\partial x^i\partial x^j}|}_p-{\sum _{k=1}^m{\lambda }_k\frac{{\partial }^2{g}^k}{\partial x^i\partial x^j}|}_p\right)h^i{h}^j.$$ (9)

We emphasize that the vector $h$ can be restricted to lie in the tangent space ${\mathrm{T}}_{p}M$, when studying the stationary point $p$ of $f$ restricted to $M$.

In matrix form (9) can be written

$$B={\left[\begin{array}{c}\hfill \frac{{\partial }^{2}f}{\partial x^i\partial x^j}-\sum _{k=1}^m{\lambda }_k\frac{{\partial }^2{g}^k}{\partial x^i\partial x^j}\hfill \end{array}\right]}_{ij}.$$ (10)

But again, the test vector $h$ need only lie on ${\mathrm{T}}_{p}M$, so if we want to apply positive/negative definiteness tests for matrices, they should instead be applied to the projected or reduced Hessian:

$$Z^{\mathrm{t}}BZ$$ (11)

where the columns of the $n\times (n-m)$ matrix $Z$ form a basis for ${\mathrm{T}}_{p}M=\ker g^{\prime }(p)\subset {ℝ}^n$.

Title	tests for local extrema in Lagrange multiplier method
Canonical name	TestsForLocalExtremaInLagrangeMultiplierMethod
Date of creation	2013-03-22 15:28:52
Last modified on	2013-03-22 15:28:52
Owner	stevecheng (10074)
Last modified by	stevecheng (10074)
Numerical id	6
Author	stevecheng (10074)
Entry type	Result
Classification	msc 26B12
Classification	msc 49-00
Classification	msc 49K35
Related topic	HessianForm
Related topic	RelationsBetweenHessianMatrixAndLocalExtrema
Defines	projected Hessian
Defines	reduced Hessian