|
Theorem:(Bennett inequality, 1962):
Let $\{X_{i}\}_{i=1}^{n}$ be a collection of independent random variables satisfying the conditions:
a) $E[X_{i}^{2}]<\infty $ $\forall i$ , so that one can write $% \sum_{i=1}^{n}E[X_{i}^{2}]=v^{2}$
b) $\Pr\left\{\left\vert X_{i}\right\vert \leq M\right\} =1$ $\forall i$ .
Then, for any $\varepsilon \geq 0$ , $$ \Pr\left\{ \sum_{i=1}^{n}\left( X_{i}-E[X_{i}]\right) >\varepsilon \right\} \leq \exp \left[ -\frac{v^{2}}{M^{2}}\theta \left( \frac{\varepsilon M}{v^{2}% }\right) \right] \leq \exp \left[ -\frac{\varepsilon }{2M}\ln \left( 1+\frac{% \varepsilon M}{v^{2}}\right) \right] $$ where $$ \theta \left( x\right) =\left( 1+x\right) \ln \left( 1+x\right) -x $$
Remark: Observing that $\left( 1+x\right) \ln \left( 1+x\right) -x\geq 9\left( 1+% \frac{x}{3}-\sqrt{1+\frac{2}{3}x}\right) \geq \frac{3x^{2}}{2\left( x+3\right) }$ $\ \ \forall x\geq 0$ , and plugging these expressions into the bound, one obtains immediately the Bernstein inequality under the hypotheses of boundness of random variables, as one might expect. However, Bernstein inequalities, although weaker, hold under far more general hypotheses than Bennett one.
|