Let $\lbrace X_n\rbrace$ be a stationary Markov chain and $I$ the state space. Given $i,j\in I$ and any non-negative integer $n$ , define a number $F_{ij}^n$ as follows:

From the definition of $F_{ij}^n$ , we see that the probability of the process reaching state $j$ within and including time $n$ from state $i$ at time $0$ is given by $$\sum_{m=0}^n F_{ij}^m.$$ As $n\to \infty$ , we have the limiting probability of the process reaching $j$ eventually from the initial state of $i$ at $0$ , which we denote by $F_{ij}$ : $$F_{ij}:=\sum_{m=0}^{\infty} F_{ij}^m.$$

Definitions. A state $i\in I$ is said to be recurrent or persistent if $F_{ii}=1$ , and transient otherwise.

Given a recurrent state $i$ , we can further classify it according to ``how soon'' the state $i$ returns after its initial appearance. Given $F_{ii}^n$ , we can calculate the expected number of steps or transitions required to return to state $i$ by time $n$ . This expectation is given by $$\sum_{m=0}^n m F_{ii}^m.$$ When $n\to \infty$ , the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state $i$ at all from the beginning. We denote this figure by $\mu_i$ : $$\mu_{i}:=\sum_{m=0}^{\infty} m F_{ii}^m.$$

Definitions. A recurrent state $i\in I$ is said to be positive or strongly ergodic if $\mu_i<\infty$ , otherwise it is called null or weakly ergodic. If a stronly ergodic state is in addition aperiodic, then it is said to be an ergodic state.