proof of Kolmogorov’s strong law for IID random variables

Kolmogorov’s strong law for square integrable random variables states that if $X_{1},X_{2},\ldots$ is a sequence of independent random variables with $\sum_{n}\operatorname{Var}[X_{n}]/n^{2}<\infty$ then $n^{-1}\sum_{k=1}^{n}(X_{k}-\mathbb{E}[X_{k}])$ converges to zero with probability one as $n\rightarrow\infty$ (see martingale proof of Kolmogorov’s strong law for square integrable variables). We show that the following version of the strong law for IID random variables follows from this.

Theorem (Kolmogorov).

Let $X_{1},X_{2},\ldots$ be independent and identically distributed random variables with $\mathbb{E}[|X_{n}|]<\infty$ . Then, $n^{-1}\sum_{k=1}^{n}(X_{k}-\mathbb{E}[X_{k}])\rightarrow 0$ as $n\rightarrow\infty$ , with probability one.

Note that here, the random variables $X_{n}$ are not necessarily square integrable. Let us set $\tilde{X}_{n}=X_{n}-\mathbb{E}[X_{n}]$ , so that $\tilde{X}_{n}$ are IID random variables with $\mathbb{E}[\tilde{X}_{n}]=0$ . Then, set

Y_{n}=\left\{\begin{array}[]{ll}\tilde{X}_{n},&\textrm{if }\left|\tilde{X}_{n}% \right|<n,\\ 0,&\textrm{otherwise}.\end{array}\right.

Using the fact that $X_{n}$ has the same distribution as $X_{1}$ gives

\begin{split}\displaystyle\sum_{n}\mathbb{E}[Y_{n}^{2}]/n^{2}&\displaystyle=% \sum_{n}\mathbb{E}\left[1_{\{|\tilde{X}_{n}|<n\}}n^{-2}\tilde{X}_{n}^{2}\right% ]\\ &\displaystyle=\sum_{n}\mathbb{E}\left[1_{\{|\tilde{X}_{1}|<n\}}n^{-2}\tilde{X% }_{1}^{2}\right]\\ &\displaystyle=\mathbb{E}\left[\sum_{n}1_{\{|\tilde{X}_{1}|<n\}}n^{-2}\tilde{X% }_{1}^{2}\right].\end{split}

(1)

Letting $N$ be the smallest integer greater than $|\tilde{X}_{1}|$ ,

\begin{split}\displaystyle\sum_{n}1_{\{|\tilde{X}_{1}|<n\}}n^{-2}&% \displaystyle\leq\sum_{n=N}^{\infty}\frac{4}{4n^{2}-1}=\sum_{n=N}^{\infty}% \left(\frac{2}{2n-1}-\frac{2}{2n+1}\right)\\ &\displaystyle=\frac{2}{2N-1}\leq\frac{2}{N}\leq\frac{2}{|\tilde{X}_{1}|}.\end% {split}

So, putting this into equation (1),

\sum_{n}\operatorname{Var}[Y_{n}]/n^{2}\leq\sum_{n}\mathbb{E}[Y_{n}^{2}]/n^{2}% \leq\mathbb{E}[2|\tilde{X}_{1}|]<\infty.

Therefore, $Y_{n}$ satisfies the required properties to apply the strong law for square integrable random variables,

n^{-1}\sum_{k=1}^{n}(Y_{k}-\mathbb{E}[Y_{k}])\rightarrow 0

(2)

as $n\rightarrow\infty$ , with probability one. Also,

\mathbb{E}[Y_{n}]=\mathbb{E}[Y_{n}-\tilde{X}_{n}]=-\mathbb{E}[1_{\{|\tilde{X}_% {n}|\geq n\}}\tilde{X}_{n}]=-\mathbb{E}[1_{\{|\tilde{X}_{1}|\geq n\}}\tilde{X}% _{1}]

converges to $0$ as $n\rightarrow\infty$ (by the dominated convergence theorem). So, the $\mathbb{E}[Y_{k}]$ terms in (2) vanish in the limit, giving

n^{-1}\sum_{k=1}^{n}Y_{k}\rightarrow 0

(3)

as $n\rightarrow\infty$ with probability one.

We finally note that

\mathbb{E}\left[\sum_{n}1_{\{\tilde{X}_{n}\not=Y_{n}\}}\right]=\mathbb{E}\left% [\sum_{n}1_{\{|\tilde{X}_{1}|\geq n\}}\right]\leq\mathbb{E}[|\tilde{X}_{1}|]<\infty,

so $\sum_{n}1_{\{\tilde{X}_{n}\not=Y_{n}\}}<\infty$ , and $\tilde{X}_{n}=Y_{n}$ for large $n$ (with probability one). So, $Y_{k}$ can be replaced by $\tilde{X}_{k}$ in (3), giving the result.

References

1 David Williams, Probability with martingales, Cambridge Mathematical Textbooks, Cambridge University Press, 1991.
2 Olav Kallenberg, Foundations of modern probability, Second edition. Probability and its Applications. Springer-Verlag, 2002.

Title	proof of Kolmogorov’s strong law for IID random variables
Canonical name	ProofOfKolmogorovsStrongLawForIIDRandomVariables
Date of creation	2013-03-22 18:33:57
Last modified on	2013-03-22 18:33:57
Owner	gel (22282)
Last modified by	gel (22282)
Numerical id	7
Author	gel (22282)
Entry type	Proof
Classification	msc 60F15
Related topic	KolmogorovsStrongLawOfLargeNumbers
Related topic	MartingaleProofOfKolmogorovsStrongLawForSquareIntegrableVariables