mean hitting time
Let be a Markov chain with transition probabilities where are states in an indexing set . Let be the hitting time of for a subset . That is, is the random variable of the time it takes for the to first reach a in .
Define the mean hitting time of given the starts in state to be
Remark. In this case, a solution is minimal if for any non negative solution we have for all .
If , then , which means (the is certain to be in a state in at step ).
If we condition on the first step:
So the satisfy the given equations.
Now suppose that is any non-negative solution to the equations. Then for we have . If , then
where is the probability that the chain does not enter in the first steps after the initial state .
is non negative by assumption, therefore so is the final term, and so
Since is arbitrary, by taking the limit , we have that
So for all and therefore is the minimal solution. ∎
|Title||mean hitting time|
|Date of creation||2013-03-22 14:20:12|
|Last modified on||2013-03-22 14:20:12|
|Last modified by||CWoo (3771)|
|Defines||mean hitting time|