Why does solving result in redundant equations for a Markov chain?
Conditions
- is a stochastic matrix (rows sum to 1).
- is a row vector of probabilities.
Reasoning, step by step
- Expand into scalar equations for each state.
- Observe that adding all these scalar equations together results in .
- Conclude that one equation is a linear combination of the others.
- Recognize that this leaves the system underdetermined, requiring an additional constraint like normalization () to find a unique solution.
Example
For the two-state chain, the equations and both simplify to . The speaker notes, "these two equations are exactly the same equation. They're redundant equations."
Common misconceptions
- Expecting the component equations from to provide independent constraints.
- Forgetting that the normalization condition is necessary to resolve the redundancy and find the exact probabilities.
Watch the explanation
Connected concepts
Explore next
Related questions
To find a stationary distribution for a Markov chain, one solves the system of linear equations given by , subject to the constraint that the probabilities in sum to 1.
Conditions: is the transition matrix of the Markov chain.; is a row vector of probabilities.
The stationary distribution is the row vector .
Conditions: The transition matrix is .; is a row vector satisfying and .
A stationary distribution (also called an invariant distribution) is a probability distribution row vector that remains unchanged when multiplied by the transition matrix of a Markov chain. This means that if the system starts in this distribution, it will remain in this distribution at all future time steps.
Conditions: is a row vector.; is the transition matrix of the Markov chain.; The elements of are non-negative and sum to 1.
We add the equation because is a valid probability distribution, meaning its elements must sum to 1. This provides the second independent equation needed to solve the underdetermined system created by the redundant balance equations.
Conditions: is a probability distribution over the states.; The state space has exactly two states, a and b.
By substituting into , we get . Combining the terms yields , and solving for gives .
Conditions: The balance equation gives , which rearranges to .; The normalization equation is .
Answers are generated from source material and independently checked. Consult the original video or creator if something is unclear.