# recurrence in a Markov chain

Let $\{X_{n}\}$ be a stationary (http://planetmath.org/StationaryProcess) Markov chain and $I$ the state space. Given $i,j\in I$ and any non-negative integer $n$, define a number $F_{ij}^{n}$ as follows:

 $F_{ij}^{n}:=\begin{cases}0&\text{if }n=0,\\ P(X_{n}=j\mbox{ and }X_{m}\neq j\mbox{ for }0

In other words, $F_{ij}^{n}$ is the probability that the process first reaches state $j$ at time $n$ from state $i$ at time $0$.

From the definition of $F_{ij}^{n}$, we see that the probability of the process reaching state $j$ within and including time $n$ from state $i$ at time $0$ is given by

 $\sum_{m=0}^{n}F_{ij}^{m}.$

As $n\to\infty$, we have the limiting probability of the process reaching $j$ eventually from the initial state of $i$ at $0$, which we denote by $F_{ij}$:

 $F_{ij}:=\sum_{m=0}^{\infty}F_{ij}^{m}.$

Definitions. A state $i\in I$ is said to be recurrent or persistent if $F_{ii}=1$, and transient otherwise.

Given a recurrent state $i$, we can further classify it according to “how soon” the state $i$ returns after its initial appearance. Given $F_{ii}^{n}$, we can calculate the expected number of steps or transitions required to return to state $i$ by time $n$. This expectation is given by

 $\sum_{m=0}^{n}mF_{ii}^{m}.$

When $n\to\infty$, the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state $i$ at all from the beginning. We denote this figure by $\mu_{i}$:

 $\mu_{i}:=\sum_{m=0}^{\infty}mF_{ii}^{m}.$

Definitions. A recurrent state $i\in I$ is said to be or strongly ergodic if $\mu_{i}<\infty$, otherwise it is called null or weakly ergodic. If a stronly ergodic state is in addition aperiodic (http://planetmath.org/PeriodicityOfAMarkovChain), then it is said to be an ergodic state.

 Title recurrence in a Markov chain Canonical name RecurrenceInAMarkovChain Date of creation 2013-03-22 16:24:43 Last modified on 2013-03-22 16:24:43 Owner CWoo (3771) Last modified by CWoo (3771) Numerical id 5 Author CWoo (3771) Entry type Definition Classification msc 60J10 Synonym null recurrent Synonym positive recurrent Synonym strongly ergodic Synonym weakly ergodic Defines recurrent state Defines persistent state Defines transient state Defines null state Defines positive state Defines ergodic state