relation between positive function and its gradient when its Hessian matrix is bounded

Let $f:R^n\to R$ a positive function, twice differentiable everywhere. Furthermore, let ${\parallel {𝐇}_f(𝐱)\parallel }_2\le M,M>0$ $\forall 𝐱\in R^n$, where ${𝐇}_f(𝐱)$ is the Hessian matrix of $f(𝐱)$. Then, for any $𝐱\in R^n$,

$${\parallel \nabla f(𝐱)\parallel }_2\le \sqrt{2Mf(𝐱)}$$

Proof.

Let $𝐱,$ ${𝐱}_0\in R^n$ be arbitrary points. By positivity of $f(𝐱)$, writing Taylor expansion of $f(𝐱)$ with Lagrange error formula around ${𝐱}_0$, a point $𝐜\in R^n$ exists such that:

$0$ $\le$ $f(𝐱)$

$=$ $f({𝐱}_0)+\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)+{\displaystyle \frac{1}{2}}{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)$

$=$ $\left|f({𝐱}_0)+\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)+{\displaystyle \frac{1}{2}}{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)\right|$

$\le$ $f({𝐱}_0)+\left|\nabla f({𝐱}_0)\cdot (𝐱-{𝐱}_0)\right|+{\displaystyle \frac{1}{2}}\left|{(𝐱-{𝐱}_0)}^T\cdot {𝐇}_f(𝐜)\cdot (𝐱-{𝐱}_0)\right|$

$\le$ $f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+{\displaystyle \frac{1}{2}}{\parallel {𝐇}_f(𝐜)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2^2\text{ (by Cauchy-Schwartz inequality)}$

$\le$ $f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+{\displaystyle \frac{1}{2}}M{\parallel 𝐱-{𝐱}_0\parallel }_2^2$

The rightest side is a second degree polynomial in variable ${\parallel 𝐱-{𝐱}_0\parallel }_2$; for it to be positive for any choice of ${\parallel 𝐱-{𝐱}_0\parallel }_2$ (that is, for any choice of $𝐱$), the discriminant

$${\parallel \nabla f({𝐱}_0)\parallel }_2^2-4\cdot \frac{1}{2}Mf({𝐱}_0)$$

must be negative, whence the thesis. ∎

Note: The condition on the boundedness of the Hessian matrix is actually needed. In fact, in the Lagrange form remainder, the constant $𝐜$ depends upon the point $𝐱$. Thus, if we couldn’t rely on the condition ${\parallel {𝐇}_f(𝐱)\parallel }_2\le M$, we could only state $f({𝐱}_0)+{\parallel \nabla f({𝐱}_0)\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2+\frac{1}{2}{\parallel {𝐇}_f(𝐜(𝐱))\parallel }_2{\parallel 𝐱-{𝐱}_0\parallel }_2^2\ge 0$ which, not being a second degree polynomial, wouldn’t imply any particular further condition. Moreover, in the case $n=1$, the lemma assumes the simpler form: Let $f:ℝ\to ℝ$ a positive function, twice differentiable everywhere. Furthermore, let $f^{\prime \prime }(x)\le M,M>0$ $\forall x\in ℝ$. Then, for any $x\in ℝ$, $\left|f^{\prime }(x)\right|\le \sqrt{2Mf(x)}$.

Title	relation between positive function and its gradient when its Hessian matrix is bounded
Canonical name	RelationBetweenPositiveFunctionAndItsGradientWhenItsHessianMatrixIsBounded
Date of creation	2013-03-22 15:53:10
Last modified on	2013-03-22 15:53:10
Owner	Andrea Ambrosio (7332)
Last modified by	Andrea Ambrosio (7332)
Numerical id	16
Author	Andrea Ambrosio (7332)
Entry type	Theorem
Classification	msc 26D10