|
|
|
|
Markov chain
|
(Definition)
|
|
We begin with a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ . Let $I$ be a countable set, $(X_n: n \in \integers)$ be a collection of random variables taking values in $I$ , $\mv{T} = (t_{ij}: i,j \in I)$ be a stochastic matrix, and $\mv{\lambda}$ be a distribution. We call $(X_n)_{n\ge 0}$ a Markov chain with initial distribution $\mv{\lambda}$ and transition matrix $\mv{T}$ if:
- $X_0$ has distribution $\mv{\lambda}$ .
- For $n \ge 0$ , $\mathbb{P}(X_{n+1}=i_{n+1} | X_0 = i_0, \ldots, X_n = i_n) = t_{i_n i_{n+1}}$ .
That is, the next value of the chain depends only on the current value, not any previous values. This is often summed up in the pithy phrase, ``Markov chains have no memory.''
As a special case of (2) we have that $\mathbb{P}(X_{n+1} = i | X_n = j) = t_{ij}$ whenever $i,j \in I$ . The values $t_{ij}$ are therefore called transition probabilities for the Markov chain.
Markov chains are arguably the simplest examples of random processes. They come in discrete and continuous versions; the discrete version is presented above.
|
"Markov chain" is owned by Mathprof. [ full author list (2) | owner history (1) ]
|
|
(view preamble | get metadata)
Cross-references: continuous, discrete, random processes, current, chain, distribution, stochastic matrix, random variables, collection, countable, probability space
There are 13 references to this entry.
This is version 2 of Markov chain, born on 2002-04-29, modified 2006-12-31.
Object id is 2886, canonical name is MarkovChain.
Accessed 19267 times total.
Classification:
| AMS MSC: | 60J10 (Probability theory and stochastic processes :: Markov processes :: Markov chains with discrete parameter) |
|
|
|
|
|
|
Pending Errata and Addenda
|
|
|
|
|
|
|
|
|
|
|
