# Markov chain

## Definition

We begin with a probability space $(\Omega,\mathcal{F},\mathbb{P})$. Let $I$ be a countable set, $(X_{n}:n\in\mathbb{Z})$ be a collection of random variables taking values in $I$, $\mathbf{T}=(t_{ij}:i,j\in I)$ be a stochastic matrix, and $\mathbf{\lambda}$ be a distribution. We call $(X_{n})_{n\geq 0}$ a Markov chain with initial distribution $\mathbf{\lambda}$ and transition matrix $\mathbf{T}$ if:

1. 1.

$X_{0}$ has distribution $\mathbf{\lambda}$.

2. 2.

For $n\geq 0$, $\mathbb{P}(X_{n+1}=i_{n+1}|X_{0}=i_{0},\ldots,X_{n}=i_{n})=t_{i_{n}i_{n+1}}$.

That is, the next value of the chain depends only on the current value, not any previous values. This is often summed up in the pithy phrase, “Markov chains have no memory.”

As a special case of (2) we have that $\mathbb{P}(X_{n+1}=i|X_{n}=j)=t_{ij}$ whenever $i,j\in I$. The values $t_{ij}$ are therefore called transition probabilities for the Markov chain.

## Discussion

Markov chains are arguably the simplest examples of random processes. They come in discrete and continuous versions; the discrete version is presented above.

 Title Markov chain Canonical name MarkovChain Date of creation 2013-03-22 12:37:32 Last modified on 2013-03-22 12:37:32 Owner Mathprof (13753) Last modified by Mathprof (13753) Numerical id 5 Author Mathprof (13753) Entry type Definition Classification msc 60J10 Related topic HittingTime Related topic MarkovChainsClassStructure Related topic MemorylessRandomVariable Related topic LeslieModel Defines Markov chain Defines transition matrix Defines transition probability