Markov chain

Definition

We begin with a probability space $(\Omega,\mathcal{F},\mathbb{P})$ . Let $I$ be a countable set, $(X_{n}:n\in\mathbb{Z})$ be a collection of random variables taking values in $I$ , $\mathbf{T}=(t_{ij}:i,j\in I)$ be a stochastic matrix, and $\mathbf{\lambda}$ be a distribution. We call $(X_{n})_{n\geq 0}$ a Markov chain with initial distribution $\mathbf{\lambda}$ and transition matrix $\mathbf{T}$ if:

1.

$X_{0}$ has distribution $\mathbf{\lambda}$ .
2.

For $n\geq 0$ , $\mathbb{P}(X_{n+1}=i_{n+1}|X_{0}=i_{0},\ldots,X_{n}=i_{n})=t_{i_{n}i_{n+1}}$ .

That is, the next value of the chain depends only on the current value, not any previous values. This is often summed up in the pithy phrase, “Markov chains have no memory.”

As a special case of (2) we have that $\mathbb{P}(X_{n+1}=i|X_{n}=j)=t_{ij}$ whenever $i,j\in I$ . The values $t_{ij}$ are therefore called transition probabilities for the Markov chain.

Discussion

Markov chains are arguably the simplest examples of random processes. They come in discrete and continuous versions; the discrete version is presented above.

Title	Markov chain
Canonical name	MarkovChain
Date of creation	2013-03-22 12:37:32
Last modified on	2013-03-22 12:37:32
Owner	Mathprof (13753)
Last modified by	Mathprof (13753)
Numerical id	5
Author	Mathprof (13753)
Entry type	Definition
Classification	msc 60J10
Related topic	HittingTime
Related topic	MarkovChainsClassStructure
Related topic	MemorylessRandomVariable
Related topic	LeslieModel
Defines	Markov chain
Defines	transition matrix
Defines	transition probability