PlanetMath: Markov chain

(more info)

Math for the people, by the people.

Main Menu

sections Encyclopædia
Papers
Books
Expositions

meta Requests (198)
Orphanage
Unclass'd
Unproven (490)
Corrections (7)
Classification

talkback Polls
Forums
Feedback
Bug Reports

downloads Snapshots
PM Book

information News
Docs
Wiki
ChangeLog
TODO List
Legalese
About

Entry average rating: No information on entry rating

Markov chain

(Definition)

Definition

We begin with a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ . Let

be a countable set, $(X_n: n \in \mathbb{Z})$ be a collection of random variables taking values in

, $\mathbf{T} = (t_{ij}: i,j \in I)$ be a stochastic matrix, and $\mathbf{\lambda}$ be a distribution. We call $(X_n)_{n\ge 0}$ a Markov chain with initial distribution $\mathbf{\lambda}$ and transition matrix $\mathbf{T}$ if:

has distribution $\mathbf{\lambda}$ .
For $n \ge 0$ , $\mathbb{P}(X_{n+1}=i_{n+1} \vert X_0 = i_0, \ldots, X_n = i_n) = t_{i_n i_{n+1}}$ .

That is, the next value of the chain depends only on the current value, not any previous values. This is often summed up in the pithy phrase, “Markov chains have no memory.”

As a special case of (2) we have that $\mathbb{P}(X_{n+1} = i \vert X_n = j) = t_{ij}$ whenever $i,j \in I$ . The values $t_{ij}$ are therefore called transition probabilities for the Markov chain.

Discussion

Markov chains are arguably the simplest examples of random processes. They come in discrete and continuous versions; the discrete version is presented above.

"Markov chain" is owned by Mathprof. [ full author list (2) | owner history (1) ]

(view preamble)

Also defines:

Markov chain, transition matrix, transition probability

Keywords:

random process

Attachments:: periodicity of a Markov chain (Definition) by CWoo
recurrence in a Markov chain (Definition) by CWoo

Log in to rate this entry.
(view current ratings)

Cross-references: continuous, discrete, random processes, current, chain, distribution, stochastic matrix, random variables, collection, countable, probability space
There are 13 references to this entry.

This is version 2 of Markov chain, born on 2002-04-29, modified 2006-12-31.
Object id is 2886, canonical name is MarkovChain.
Accessed 16492 times total.

Classification:

AMS MSC:

60J10 (Probability theory and stochastic processes :: Markov processes :: Markov chains with discrete parameter)

Pending Errata and Addenda

None.[ View all 2 ]

Discussion

forum policy

No messages.

Interact