mean hitting time
Let (Xn)n≥0 be a Markov chain with transition probabilities pij where i,j are states in an indexing set I. Let HA be the hitting time of (Xn)n≥0 for a subset A⊆I. That is, HA is the random variable of the time it takes for the to first reach a in A.
Define the mean hitting time of A given the starts in state i to be
kAi:= |
Remark. In this case, a solution is minimal if for any non negative solution we have for all .
Proof.
If , then , which means (the is certain to be in a state in at step ).
If we condition on the first step:
So the satisfy the given equations.
Now suppose that is any non-negative solution to the equations. Then for we have . If , then
where is the probability that the chain does not enter in the first steps after the initial state .
is non negative by assumption, therefore so is the final term, and so
Since is arbitrary, by taking the limit , we have that
So for all and therefore is the minimal solution. ∎
Title | mean hitting time |
---|---|
Canonical name | MeanHittingTime |
Date of creation | 2013-03-22 14:20:12 |
Last modified on | 2013-03-22 14:20:12 |
Owner | CWoo (3771) |
Last modified by | CWoo (3771) |
Numerical id | 24 |
Author | CWoo (3771) |
Entry type | Theorem |
Classification | msc 60J10 |
Related topic | HittingTime |
Defines | mean hitting time |