Stochastic Matrix - Example: The Cat and Mouse

Example: The Cat and Mouse

Suppose you have a timer and a row of five adjacent boxes, with a cat in the first box and a mouse in the fifth box at time zero. The cat and the mouse both jump to a random adjacent box when the timer advances. E.g. if the cat is in the second box and the mouse in the fourth one, the probability is one fourth that the cat will be in the first box and the mouse in the fifth after the timer advances. If the cat is in the first box and the mouse in the fifth one, the probability is one that the cat will be in box two and the mouse will be in box four after the timer advances. The cat eats the mouse if both end up in the same box, at which time the game ends. The random variable K gives the number of time steps the mouse stays in the game.

The Markov chain that represents this game contains the following five states:

• State 1: cat in the first box, mouse in the third box: (1, 3)
• State 2: cat in the first box, mouse in the fifth box: (1, 5)
• State 3: cat in the second box, mouse in the fourth box: (2, 4)
• State 4: cat in the third box, mouse in the fifth box: (3, 5)
• State 5: the cat ate the mouse and the game ended: F.

We use a stochastic matrix to represent the transition probabilities of this system,