# relation between positive function and its gradient when its Hessian matrix is bounded

Let $f:R^{n}\rightarrow R$ a positive function, twice differentiable   everywhere. Furthermore, let $\left\|\mathbf{H}_{f}(\mathbf{x})\right\|_{2}\leq M,M>0$ $\forall\mathbf{x}\in R^{n}$, where $\mathbf{H}_{f}(\mathbf{x})$ is the Hessian matrix of $f(\mathbf{x})$. Then, for any $\mathbf{x}\in R^{n}$,

 $\left\|\nabla f(\mathbf{x})\right\|_{2}\leq\sqrt{2Mf(\mathbf{x})}$
###### Proof.

Let $\mathbf{x},$ $\mathbf{x}_{0}\in R^{n}$ be arbitrary points. By positivity of $f(\mathbf{x})$, writing Taylor expansion  of $f(\mathbf{x})$ with Lagrange error formula around $\mathbf{x}_{0}$, a point $\mathbf{c}\in R^{n}$ exists such that:

 $\displaystyle 0$ $\displaystyle\leq$ $\displaystyle f(\mathbf{x})$ $\displaystyle=$ $\displaystyle f(\mathbf{x}_{0})+\nabla f(\mathbf{x}_{0})\cdot(\mathbf{x}-% \mathbf{x}_{0})+\frac{1}{2}(\mathbf{x}-\mathbf{x}_{0})^{T}\cdot\mathbf{H}_{f}(% \mathbf{c})\cdot(\mathbf{x}-\mathbf{x}_{0})$ $\displaystyle=$ $\displaystyle\left|f(\mathbf{x}_{0})+\nabla f(\mathbf{x}_{0})\cdot(\mathbf{x}-% \mathbf{x}_{0})+\frac{1}{2}(\mathbf{x}-\mathbf{x}_{0})^{T}\cdot\mathbf{H}_{f}(% \mathbf{c})\cdot(\mathbf{x}-\mathbf{x}_{0})\right|$ $\displaystyle\leq$ $\displaystyle f(\mathbf{x}_{0})+\left|\nabla f(\mathbf{x}_{0})\cdot(\mathbf{x}% -\mathbf{x}_{0})\right|+\frac{1}{2}\left|(\mathbf{x}-\mathbf{x}_{0})^{T}\cdot% \mathbf{H}_{f}(\mathbf{c})\cdot(\mathbf{x}-\mathbf{x}_{0})\right|$ $\displaystyle\leq$ $\displaystyle f(\mathbf{x}_{0})+\left\|\nabla f(\mathbf{x}_{0})\right\|_{2}% \left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}+\frac{1}{2}\left\|\mathbf{H}_{f}(% \mathbf{c})\right\|_{2}\left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}^{2}\text{% \ \ \ \ \ (by Cauchy-Schwartz inequality)}$ $\displaystyle\leq$ $\displaystyle f(\mathbf{x}_{0})+\left\|\nabla f(\mathbf{x}_{0})\right\|_{2}% \left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}+\frac{1}{2}M\left\|\mathbf{x}-% \mathbf{x}_{0}\right\|_{2}^{2}$

The rightest side is a second degree polynomial in variable $\left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}$; for it to be positive for any choice of $\left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}$ (that is, for any choice of $\mathbf{x}$), the discriminant

 $\left\|\nabla f(\mathbf{x}_{0})\right\|_{2}^{2}-4\cdot\frac{1}{2}Mf(\mathbf{x}% _{0})$

must be negative, whence the thesis. ∎

Note: The condition on the boundedness of the Hessian matrix is actually needed. In fact, in the Lagrange form remainder, the constant $\mathbf{c}$ depends upon the point $\mathbf{x}$. Thus, if we couldn’t rely on the condition $\left\|\mathbf{H}_{f}(\mathbf{x})\right\|_{2}\leq M$, we could only state $f(\mathbf{x}_{0})+\left\|\nabla f(\mathbf{x}_{0})\right\|_{2}\left\|\mathbf{x}% -\mathbf{x}_{0}\right\|_{2}+\frac{1}{2}\left\|\mathbf{H}_{f}(\mathbf{c(\mathbf% {x})})\right\|_{2}\left\|\mathbf{x}-\mathbf{x}_{0}\right\|_{2}^{2}\geq 0$ which, not being a second degree polynomial, wouldn’t imply any particular further condition. Moreover, in the case $n=1$, the lemma assumes the simpler form: Let $f:\mathbb{R}\rightarrow\mathbb{R}$ a positive function, twice differentiable everywhere. Furthermore, let $f^{\prime\prime}(x)\leq M,M>0$ $\forall x\in\mathbb{R}$. Then, for any $x\in\mathbb{R}$, $\left|f^{\prime}(x)\right|\leq\sqrt{2Mf(x)}$.

Title relation between positive function and its gradient when its Hessian matrix is bounded RelationBetweenPositiveFunctionAndItsGradientWhenItsHessianMatrixIsBounded 2013-03-22 15:53:10 2013-03-22 15:53:10 Andrea Ambrosio (7332) Andrea Ambrosio (7332) 16 Andrea Ambrosio (7332) Theorem msc 26D10