Why introduce matrices and vectors when studying Markov chains?

Conditions

• The problem involves a Markov chain.
• The goal is to compute future states or manipulate transition probabilities.

Reasoning, step by step

1. Recognize that direct verbal or diagrammatic reasoning can become cumbersome for multiple steps.
2. Introduce vectors to encode the distribution of states at any given time.
3. Introduce matrices to organize the one-step transition probabilities.
4. Use matrix-vector multiplication to efficiently compute the state at future time steps.

Example

The speaker announces that they will use some formalisms of linear algebra, the idea of matrices, the idea of vectors and how to multiply those to make manipulations simpler when studying a Markov chain.

Common misconceptions

• Believing that matrices are strictly necessary to understand the basic concept of a Markov chain.
• Thinking that matrix multiplication is the only way to find the next state, ignoring that for a single step, reading the diagram is sufficient.

Watch the explanation

Explore next

• What is the initial state vector $S_0$ in this example?
• In this video, what do the rows and columns of the transition matrix represent?