proof of Kolmogorov’s strong law for IID random variables


Kolmogorov’s strong law for square integrable random variablesMathworldPlanetmath states that if X1,X2, is a sequenceMathworldPlanetmath of independent random variables with nVar[Xn]/n2< then n-1k=1n(Xk-𝔼[Xk]) converges to zero with probability one as n (see martingaleMathworldPlanetmath proof of Kolmogorov’s strong law for square integrable variables). We show that the following version of the strong law for IID random variables follows from this.

Theorem (Kolmogorov).

Let X1,X2, be independent and identically distributed random variables with E[|Xn|]<. Then, n-1k=1n(Xk-E[Xk])0 as n, with probability one.

Note that here, the random variables Xn are not necessarily square integrable. Let us set X~n=Xn-𝔼[Xn], so that X~n are IID random variables with 𝔼[X~n]=0. Then, set

Yn={X~n,if |X~n|<n,0,otherwise.

Using the fact that Xn has the same distributionPlanetmathPlanetmath as X1 gives

n𝔼[Yn2]/n2=n𝔼[1{|X~n|<n}n-2X~n2]=n𝔼[1{|X~1|<n}n-2X~12]=𝔼[n1{|X~1|<n}n-2X~12]. (1)

Letting N be the smallest integer greater than |X~1|,

n1{|X~1|<n}n-2n=N44n2-1=n=N(22n-1-22n+1)=22N-12N2|X~1|.

So, putting this into equation (1),

nVar[Yn]/n2n𝔼[Yn2]/n2𝔼[2|X~1|]<.

Therefore, Yn satisfies the required properties to apply the strong law for square integrable random variables,

n-1k=1n(Yk-𝔼[Yk])0 (2)

as n, with probability one. Also,

𝔼[Yn]=𝔼[Yn-X~n]=-𝔼[1{|X~n|n}X~n]=-𝔼[1{|X~1|n}X~1]

converges to 0 as n (by the dominated convergence theorem). So, the 𝔼[Yk] terms in (2) vanish in the limit, giving

n-1k=1nYk0 (3)

as n with probability one.

We finally note that

𝔼[n1{X~nYn}]=𝔼[n1{|X~1|n}]𝔼[|X~1|]<,

so n1{X~nYn}<, and X~n=Yn for large n (with probability one). So, Yk can be replaced by X~k in (3), giving the result.

References

  • 1 David Williams, Probability with martingales, Cambridge Mathematical Textbooks, Cambridge University Press, 1991.
  • 2 Olav Kallenberg, Foundations of modern probability, Second edition. Probability and its Applications. Springer-Verlag, 2002.
Title proof of Kolmogorov’s strong law for IID random variables
Canonical name ProofOfKolmogorovsStrongLawForIIDRandomVariables
Date of creation 2013-03-22 18:33:57
Last modified on 2013-03-22 18:33:57
Owner gel (22282)
Last modified by gel (22282)
Numerical id 7
Author gel (22282)
Entry type Proof
Classification msc 60F15
Related topic KolmogorovsStrongLawOfLargeNumbers
Related topic MartingaleProofOfKolmogorovsStrongLawForSquareIntegrableVariables