martingale proof of Kolmogorov’s strong law for square integrable variables

We apply the martingale convergence theorem to prove the following result.

Theorem.

Let $X_{1},X_{2},\ldots$ be independent random variables such that $\sum_{n}\operatorname{Var}[X_{n}]/n^{2}<\infty$ . Then, setting

$S_{n}=\frac{1}{n}\sum_{k=1}^{n}(X_{k}-\mathbb{E}[X_{k}])$

we have $S_{n}\rightarrow 0$ as $n\rightarrow\infty$ , with probability one.

To prove this, we start by constructing a martingale,

$M_{n}=\sum_{k=1}^{n}\frac{X_{k}-\mathbb{E}[X_{k}]}{k}.$

If $\mathcal{F}_{n}$ is the $\sigma$ -algebra (http://planetmath.org/SigmaAlgebra) generated by $X_{1},\ldots X_{n}$ then

$\mathbb{E}[M_{n+1}\mid\mathcal{F}_{n}]=M_{n}+\frac{\mathbb{E}[X_{n+1}\mid% \mathcal{F}_{n}]-\mathbb{E}[X_{n+1}]}{n+1}=M_{n}.$

Here, the independence of $X_{n+1}$ and $\mathcal{F}_{n}$ has been used to imply that $\mathbb{E}[X_{n+1}\mid\mathcal{F}_{n}]=\mathbb{E}[X_{n+1}]$ . So, $M$ is a martingale with respect to the filtration $(\mathcal{F}_{n})_{n\in\mathbb{N}}$ .

Also, by the independence of the $X_{n}$ , the variance of $M_{n}$ is

$\operatorname{Var}[M_{n}]=\sum_{k=1}^{n}\operatorname{Var}[X_{k}/k]\leq\sum_{k% =1}^{\infty}\frac{\operatorname{Var}[X_{k}]}{k^{2}}<\infty.$

So, the inequality $\mathbb{E}[|M_{n}|]\leq\sqrt{\mathbb{E}[M_{n}^{2}]}=\sqrt{\operatorname{Var}[M% _{n}]}$ shows that $M$ is an $L^{1}$ -bounded martingale, and the martingale convergence theorem says that the limit $M_{\infty}=\lim_{n\rightarrow\infty}M_{n}$ exists and is finite, with probability one.

The strong law now follows from Kronecker’s lemma, which states that for sequences of real numbers $x_{1},x_{2},\ldots$ and $0<b_{1},b_{2},\ldots$ such that $b_{n}$ strictly increases to infinity and $\sum_{n}x_{n}/b_{n}$ converges to a finite limit, then $b_{n}^{-1}\sum_{k=1}^{n}x_{k}$ tends to $0$ as $n\rightarrow\infty$ . In our case, we take $x_{n}=X_{n}-\mathbb{E}[X_{n}]$ and $b_{n}=n$ to deduce that $n^{-1}\sum_{k=1}^{n}(X_{k}-\mathbb{E}[X_{k}])$ converges to zero with probability one.

References

1 David Williams, Probability with martingales, Cambridge Mathematical Textbooks, Cambridge University Press, 1991.
2 Olav Kallenberg, Foundations of modern probability, Second edition. Probability and its Applications. Springer-Verlag, 2002.

Title	martingale proof of Kolmogorov’s strong law for square integrable variables
Canonical name	MartingaleProofOfKolmogorovsStrongLawForSquareIntegrableVariables
Date of creation	2013-03-22 18:33:51
Last modified on	2013-03-22 18:33:51
Owner	gel (22282)
Last modified by	gel (22282)
Numerical id	4
Author	gel (22282)
Entry type	Proof
Classification	msc 60F15
Classification	msc 60G42
Related topic	MartingaleConvergenceTheorem
Related topic	KolmogorovsStrongLawOfLargeNumbers
Related topic	StrongLawOfLargeNumbers
Related topic	ProofOfKolmogorovsStrongLawForIIDRandomVariables