recurrence in a Markov chain
In other words, is the probability that the process first reaches state at time from state at time .
From the definition of , we see that the probability of the process reaching state within and including time from state at time is given by
As , we have the limiting probability of the process reaching eventually from the initial state of at , which we denote by :
Definitions. A state is said to be recurrent or persistent if , and transient otherwise.
Given a recurrent state , we can further classify it according to “how soon” the state returns after its initial appearance. Given , we can calculate the expected number of steps or transitions required to return to state by time . This expectation is given by
When , the above expression may or may not approach a limit. It is the expected number of transitions needed to return to state at all from the beginning. We denote this figure by :
Definitions. A recurrent state is said to be or strongly ergodic if , otherwise it is called null or weakly ergodic. If a stronly ergodic state is in addition aperiodic (http://planetmath.org/PeriodicityOfAMarkovChain), then it is said to be an ergodic state.
|Title||recurrence in a Markov chain|
|Date of creation||2013-03-22 16:24:43|
|Last modified on||2013-03-22 16:24:43|
|Last modified by||CWoo (3771)|