# recurrence in a Markov chain

Let $\{{X}_{n}\}$ be a stationary (http://planetmath.org/StationaryProcess) Markov chain^{} and $I$ the state space^{}. Given $i,j\in I$ and any non-negative integer $n$, define a number ${F}_{ij}^{n}$ as follows:

$$ |

In other words, ${F}_{ij}^{n}$ is the probability that the process *first* reaches state $j$ at time $n$ from state $i$ at time $0$.

From the definition of ${F}_{ij}^{n}$, we see that the probability of the process reaching state $j$ *within and including* time $n$ from state $i$ at time $0$ is given by

$$\sum _{m=0}^{n}{F}_{ij}^{m}.$$ |

As $n\to \mathrm{\infty}$, we have the limiting probability of the process reaching $j$ *eventually* from the initial state of $i$ at $0$, which we denote by ${F}_{ij}$:

$${F}_{ij}:=\sum _{m=0}^{\mathrm{\infty}}{F}_{ij}^{m}.$$ |

Definitions. A state $i\in I$ is said to be *recurrent* or *persistent* if ${F}_{ii}=1$, and *transient* otherwise.

Given a recurrent state $i$, we can further classify it according to “how soon” the state $i$ returns after its initial appearance. Given ${F}_{ii}^{n}$, we can calculate the expected number of steps or transitions required to *return* to state $i$ by time $n$. This expectation is given by

$$\sum _{m=0}^{n}m{F}_{ii}^{m}.$$ |

When $n\to \mathrm{\infty}$, the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state $i$ *at all* from the beginning. We denote this figure by ${\mu}_{i}$:

$${\mu}_{i}:=\sum _{m=0}^{\mathrm{\infty}}m{F}_{ii}^{m}.$$ |

Definitions. A recurrent state $i\in I$ is said to be or *strongly ergodic* if $$, otherwise it is called *null* or *weakly ergodic*. If a stronly ergodic state is in addition^{} aperiodic (http://planetmath.org/PeriodicityOfAMarkovChain), then it is said to be an *ergodic state*.

Title | recurrence in a Markov chain |

Canonical name | RecurrenceInAMarkovChain |

Date of creation | 2013-03-22 16:24:43 |

Last modified on | 2013-03-22 16:24:43 |

Owner | CWoo (3771) |

Last modified by | CWoo (3771) |

Numerical id | 5 |

Author | CWoo (3771) |

Entry type | Definition |

Classification | msc 60J10 |

Synonym | null recurrent |

Synonym | positive recurrent |

Synonym | strongly ergodic |

Synonym | weakly ergodic |

Defines | recurrent state |

Defines | persistent state |

Defines | transient state |

Defines | null state |

Defines | positive state |

Defines | ergodic state |