Mathematical modeling / English

Markov Chains & Transition Matrices

Dr. Trefor Bazett · YouTube · 6:54

YouTube · Dr. Trefor BazettOriginal video

Selected moment · 4:05 · Chapter markers are time references, not playback progress.

IN THIS VIDEO

01Transition diagram for a two-state Markov chain
02Meaning of the 0.75 self-loop on state A
03Why use matrices and vectors for Markov chains

FROM THE LEARNING NOTES

P(\text{next state}=A \mid \text{current state}=A)=0.75

Meaning of the 0.75 self-loop on state A

SELECTED MOMENT

Numerical expansion of the first step

SUGGESTED PREPARATION

Basic probability: events, probabilities, and percentages
Ordered lists or tuples of numbers
Column vectors in linear algebra
Matrix-vector multiplication (announced for later use, not yet shown in this clip)
Basic probability
Column vectors
Matrix notation
Introductory idea of a state transition diagram
Matrix-vector multiplication
Basic notion of a Markov chain state vector
Reading transition probabilities from a state diagram
Basic matrix multiplication
Initial familiarity with Markov chains and state vectors

READ & KEEP

The explanation, unpacked.

Saves 0

Reviewed learning material · Video analysis · English

Read the full overview

This 120-second introductory segment presents a two-state Markov chain example. After a title card, a transition diagram with states A and B and probabilities 0.75, 0.25, 0.4, and 0.6 is shown while the speaker explains that arrows encode one-step transition probabilities. The lecture then introduces linear-algebra notation, displaying the initial state vector $S_0 = (1,0)^T$ and annotating its components as starting in A and not starting in B. The speaker emphasizes that this initial condition is asserted for the example and generalizes that a state vector has one component per state. This 120-second introductory lecture segment explains a two-state Markov chain using a transition diagram, state vectors, and a transition matrix. It first interprets $S_0=(1,0)^T$ and $S_1=(0.75,0.25)^T$ , then motivates an algebraic method for iterating states efficiently. The speaker constructs $P=\begin{pmatrix}0.75&0.4\\0.25&0.6\end{pmatrix}$ from the diagram under a column-start / row-end convention and ends by introducing the update rule $S_1=PS_0$ . This 120-second lecture segment explains how Markov-chain state vectors evolve by repeated multiplication with a transition matrix. The presenter first writes $S_1 = P S_0$ , then expands it numerically as $\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$ , identifying the operation as matrix-vector multiplication. He then advances to $S_2 = P S_1$ , computes $\begin{pmatrix}0.66\\0.34\end{pmatrix}$ , notes that this agrees with an earlier tree-diagram method, substitutes $S_1 = P S_0$ to get $S_2 = P^2 S_0$ , and ends with the general formula $S_n = P^n S_0$ . This 54-second clip is the closing summary of an introductory lesson on Markov chains and transition matrices. The presenter states the key formula $S_n = P^n S_0$ , explains that the nth state is found by multiplying the transition matrix n times by the initial state, and then outlines a three-step workflow for Markov problems: draw the transition diagram, convert it to matrix form, and compute future states by repeated matrix multiplication. A static two-state diagram with states A and B and transition probabilities 0.75, 0.25, 0.4, and 0.6 remains on the board throughout, but no numerical future-state calculation is performed in this segment.

Use the learning inspector for key ideas and moments, or open the reading tabs for the complete notes.

Chapters

0:00Title: Markov Chains 0:02Context and goal: using linear algebra for Markov chains 0:35Transition diagram and one-step probabilities 1:08Introducing the initial state vector

S_0

1:26Interpreting the components of

S_0

2:00Initial and one-step state vectors 2:37Why we need an efficient update method 3:02Building the transition matrix from the diagram 3:52Introducing the matrix update rule 4:00One-step update:

S_1 = P S_0

4:05Numerical expansion of the first step 4:17Assumed prerequisite: matrix-vector multiplication 4:46Why the formalism matters beyond step 1 4:55Two-step update:

S_2 = P S_1

5:10Numerical computation of

S_2

5:22Agreement with the tree-diagram method 5:44From

S_2 = P^2 S_0

S_n = P^n S_0

6:00Final formula for the nth state 6:18Three-step method for Markov problems 6:39Closing remarks

Learning script

Generated from the video's visuals and explanation; not verbatim speech.

The clip opens with the title "MARKOV CHAINS," signaling that the topic is the probabilistic modeling of state-to-state evolution.

The speaker frames the lesson as a continuation of an earlier introduction and says the goal is to bring in linear-algebra tools—matrices, vectors, and multiplication—to make Markov-chain calculations easier.

A concrete two-state example is then placed on screen. The diagram contains states A and B, and the speaker explains that a transition diagram records the possible states of a system together with the allowed moves between them.

Each arrow carries a number, and those numbers are interpreted as probabilities of moving from one state to another in a single step. The speaker explicitly reads the 0.75 self-loop on A as the probability that a system currently in A remains in A after one iteration.

To move from picture-based reasoning to symbolic reasoning, the lecture introduces notation for the starting condition of the system.

The formula $S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ appears, and the speaker names it the initial state vector. This is the first formal object in the clip that encodes the whole starting distribution in one column vector.

The components are then matched to states: the top entry corresponds to A and the bottom entry to B. The speaker also generalizes the pattern, noting that a system with states A, B, C would require a three-component vector instead.

Numerically, the entries 1 and 0 are read as probabilities: there is a 100% chance the process starts in A and a 0% chance it starts in B. The annotations "Initially at A" and "Initially not at B" make this mapping visually explicit.

Finally, the speaker stresses that this particular $S_0$ is an asserted initial condition for the example, not something forced by the transition probabilities. Other initial distributions are possible, provided the probabilities still sum to 1.

The segment opens on a two-state Markov chain diagram with states A and B and transition probabilities 0.75, 0.25, 0.4, and 0.6. Below it, the initial state vector is written as $S_0=\begin{pmatrix}1\\0\end{pmatrix}$ , meaning the system starts certainly in A and not in B.

The lecturer then introduces $S_1$ as the state after one iteration. On the board this is $S_1=\begin{pmatrix}0.75\\0.25\end{pmatrix}$ , and the explanation ties the top entry to the probability of being in A and the bottom entry to the probability of being in B.

Because the chain begins in A with probability 1, the one-step distribution is read directly from the arrows leaving A: there is a 75% chance of remaining in A and a 25% chance of moving to B. This gives the displayed $S_1$ without yet using matrix multiplication.

The lecture then shifts from interpretation to method. The speaker asks how to move systematically from $S_0$ to $S_1$ , and then onward to $S_2$ or even $S_{100}$ , making clear that direct diagram reading is not efficient for many iterations.

The proposed solution is to encode the transition rules algebraically using a matrix. The four probabilities from the diagram are collected into a single array, preparing the viewer for a compact update rule.

The matrix is written as $P=\begin{pmatrix}0.75&0.4\\0.25&0.6\end{pmatrix}$ . The lecturer explicitly adopts the convention that columns correspond to starting states and rows to ending states, so the entry 0.4 in row A, column B means $\Pr(B\to A)=0.4$ .

This array is named the transition matrix and abbreviated $P$ . With the state ordering fixed as A then B, the matrix now contains exactly the same one-step information as the diagram, but in a form suitable for repeated computation.

Finally, the board displays the update formula $S_1=PS_0$ . The clip ends just as the lecturer begins to explain that matrix multiplication is the mechanism for producing the next state vector from the current one.

The clip opens with a two-state Markov diagram and the symbolic rule $S_1 = P S_0$ , establishing that the next state vector is obtained by applying the transition matrix to the initial state vector.

The presenter then writes the numerical instance $\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$ , making clear that $S_0$ is the starting distribution and $S_1$ is the predicted one-step distribution.

He identifies the operation as matrix-vector multiplication and explicitly treats that skill as prerequisite background, pointing viewers to a separate introductory video rather than teaching it here.

At this stage he remarks that for only one step the matrix formalism does not yet add much, because $S_1$ could already be read from the diagram; the value of the algebra appears when iterating further.

The board advances to $S_2 = P S_1$ , showing that the same transition matrix is reused on the newly computed state vector to obtain the second-step distribution.

The numerical expansion becomes $\begin{pmatrix}0.66\\0.34\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}0.75\\0.25\end{pmatrix}$ , and the speaker states that carrying out the multiplication yields $S_2 = (0.66, 0.34)$ .

He compares this result with an earlier tree-diagram computation from a previous video, emphasizing that the matrix method reproduces the same answer while setting up a more scalable procedure.

Next he substitutes $S_1 = P S_0$ into $S_2 = P S_1$ to obtain $S_2 = P^2 S_0$ , revealing that two successive transitions correspond to applying the transition matrix twice.

The segment concludes by generalizing the pattern to the boxed formula $S_n = P^n S_0$ : the state after $n$ steps is found by raising the transition matrix to the $n$ th power and multiplying it by the initial state vector.

The clip opens on a static board with a two-state Markov diagram and the equations $S_2 = P S_1 = P^2 S_0$ and the boxed formula $S_n = P^n S_0$ . The speaker states that one multiplies by $n$ transition matrices, written $P^n$ , and identifies this as the final rule for manipulating Markov chains.

He then restates the meaning of the formula in words: the $n$ th state is obtained by taking the transition matrix and multiplying it $n$ times by the initial state vector. This directly links the displayed symbolic rule to the verbal explanation.

Next, the presenter summarizes a practical three-step procedure for Markov-process problems. First, write down the transition diagram so all the numerical transition values are collected. Second, convert that diagram into matrix form. Third, compute any future state by multiplying the matrix the required number of times.

The remaining seconds contain only closing remarks, with no new mathematics added. The board continues to show the same diagram and formulas while the speaker thanks viewers and invites comments and likes.

Knowledge cards

Transition diagram for a two-state Markov chain

A transition diagram shows the possible states of the system as nodes and the allowed one-step moves as directed arrows. In the example, the states are A and B. The number on each arrow is the probability of making that transition in one iteration. The displayed labels are $A\to A = 0.75$ , $A\to B = 0.25$ , $B\to A = 0.4$ , and $B\to B = 0.6$ .

Meaning of the 0.75 self-loop on state A

The 0.75 on the arrow from A back to A means that if the system is in state A now, then after one step it remains in state A with probability 75%. This is the speaker's worked interpretation of a single arrow label in the diagram.

P(\text{next state}=A \mid \text{current state}=A)=0.75

Why use matrices and vectors for Markov chains

The lecture announces that linear-algebra formalism will be used to simplify Markov-chain manipulations. Vectors can encode distributions over states, and matrices can organize transition rules, making repeated calculations cleaner than working only from the diagram.

Initial state vector $S_0$

The initial state vector records where the system starts. In this two-state example, $S_0$ is written as a column vector whose first component corresponds to state A and second component to state B. The displayed vector is $S_0 = (1, 0)^T$ .

S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}

Component order in a state vector

For the example shown, the top entry of the vector refers to state A and the bottom entry refers to state B. The speaker adds that if the system had more states, such as A, B, C, then the vector would have one component for each state.

Interpreting $S_0 = (1,0)^T$

The entries 1 and 0 are probabilities of the starting state. Thus the system begins in state A with probability 1 and in state B with probability 0. The on-screen annotations summarize this as "Initially at A" and "Initially not at B."

1 = P(\text{start in }A),\quad 0 = P(\text{start in }B)

The initial condition is asserted, not derived

The speaker explicitly says the choice $S_0 = (1,0)^T$ is being asserted for this example. It is not implied by the transition probabilities alone. Other initial distributions are possible, including cases where both states have nonzero starting probabilities that add to 1.

State vector $S_k$ in a two-state Markov chain

A state vector lists the probabilities of being in each state at time step $k$ . In this clip, the first component is the probability of state A and the second is the probability of state B. Thus $S_0=\begin{pmatrix}1\\0\end{pmatrix}$ means the system starts certainly in A, while $S_1=\begin{pmatrix}0.75\\0.25\end{pmatrix}$ means that after one iteration there is a 75% chance of being in A and a 25% chance of being in B.

S_k = \begin{pmatrix} \Pr(X_k=A) \\ \Pr(X_k=B) \end{pmatrix}

Reading $S_1$ directly from the transition diagram

When the initial state is purely A, the next-state probabilities are just the outgoing probabilities from A. The diagram shows $\Pr(A\to A)=0.75$ and $\Pr(A\to B)=0.25$ , so one iteration later the state vector is $\begin{pmatrix}0.75\\0.25\end{pmatrix}$ . This is a direct visual method that works for one step but becomes inefficient for many steps.

S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}

Why matrices are introduced for Markov chains

The lecturer motivates matrix notation by asking how to go from $S_1$ to $S_2$ , or from $S_2$ to $S_{100}$ , efficiently. Instead of re-reading the diagram at every stage, the transition rules are packaged into a matrix that can be applied repeatedly to state vectors.

Transition matrix convention used in this video

The transition matrix is built from the four diagram probabilities as $P=\begin{pmatrix}0.75&0.4\\0.25&0.6\end{pmatrix}$ . In this clip, columns represent starting states and rows represent ending states. Therefore row A, column B equals 0.4 because it encodes the probability of starting in B and ending in A.

P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6 \end{pmatrix}

One-step update rule $S_1=PS_0$

The final board equation states that the next state vector is obtained by multiplying the transition matrix by the current state vector. This is the algebraic replacement for reading the diagram manually. The clip introduces the rule but ends before carrying out the multiplication explicitly.

S_1 = P S_0

One-step Markov state update

The video defines the first evolved state vector by the matrix equation $S_1 = P S_0$ , where $P$ is the transition matrix and $S_0$ is the initial state vector. In the example, this becomes $\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$ .

S_1 = P S_0

Matrix-vector multiplication is the computational engine

The presenter explicitly names the operation used to move from one state vector to the next as matrix-vector multiplication. He assumes this as prior knowledge and says a separate introductory video covers it.

Why the formalism matters after step one

For $S_1$ alone, the speaker says the diagram is enough and the matrix formalism does not yet provide a major improvement. The benefit appears when computing later states such as $S_2$ and beyond.

Two-step Markov state update

The second state is computed by reusing the same transition matrix on the first state vector: $S_2 = P S_1$ . Numerically, the clip shows $\begin{pmatrix}0.66\\0.34\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}0.75\\0.25\end{pmatrix}$ .

S_2 = P S_1

Matrix method agrees with the tree-diagram method

The speaker notes that an earlier tree-diagram computation gave the same result for this example. The matrix approach is presented as equivalent in outcome but better suited for generalization.

From two steps to $P^2$

By substituting $S_1 = P S_0$ into $S_2 = P S_1$ , the video obtains $S_2 = P^2 S_0$ . This shows that two successive transitions correspond to applying the transition matrix twice.

S_2 = P S_1 = P^2 S_0

General n-step formula

The concluding boxed rule is $S_n = P^n S_0$ : to find the state after $n$ steps, start from the initial state vector and apply the transition matrix $n$ times.

S_n = P^n S_0

General formula for the nth state in a Markov chain

The video presents the central rule that the state after $n$ steps is found by applying the transition matrix $n$ times to the initial state vector. The board first shows the two-step identity $S_2 = P S_1 = P^2 S_0$ and then the boxed general formula $S_n = P^n S_0$ . In words, the speaker says the $n$ th state is the transition matrix multiplied $n$ times by the initial state.

S_n = P^n S_0

Three-step method for solving Markov chain problems

The presenter gives a compact workflow for Markov-process questions. Step 1 is to draw the transition diagram and collect all transition numbers. Step 2 is to rewrite that diagram in matrix form. Step 3 is to obtain any future state by repeated matrix multiplication, using the formula $S_n = P^n S_0$ .

Static two-state Markov diagram shown during the summary

Throughout the clip, the left side of the board displays a two-state example with states $A$ and $B$ . The visible transition labels are $0.75$ for the self-loop at $A$ , $0.25$ for $A \to B$ , $0.4$ for $B \to A$ , and $0.6$ for the self-loop at $B$ . The diagram is used as a visual reference while the speaker explains the general matrix-power method, but no numerical future-state computation is carried out in this segment.

Detailed learning notes

Explore conditions, steps and evidence. Supplementary explanations are labeled separately from content shown in the video.

Symbols · 36

MARKOV CHAINS

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Black title card with white handwritten-style text reading "MARKOV CHAINS".

Symbol

MARKOV CHAINS

Meaning

Title of the video topic.

Domain

video title

A

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Blue rounded rectangle labeled A in the transition diagram at upper left.
Audio
Observation
Speaker says the system might be in state A or state B and that the top component of the vector refers to state A.

Symbol

Meaning

One possible state of the Markov-chain system; also represented by the first/top component of the state vector.

Domain

state label

B

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Purple rounded rectangle labeled B in the transition diagram at upper left.
Audio
Observation
Speaker says the system might be in state A or state B and that the bottom component refers to state B.

Symbol

Meaning

The other possible state of the Markov-chain system; also represented by the second/bottom component of the state vector.

Domain

state label

0.75

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Self-loop arrow on state A labeled 0.75.
Audio
Observation
Speaker says the 0.75 on the arrow from state A to state A represents a 75% probability that state A transitions to state A in the next iteration.

Symbol

0.75

Meaning

Transition probability from state A to state A.

Domain

probability in [0,1]

0.25

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Arrow from A to B labeled 0.25.

Uncertainties

The speaker does not explicitly read this number aloud in the clip, but it is visible on the diagram.

Symbol

0.25

Meaning

Transition probability from state A to state B.

Domain

probability in [0,1]

0.4

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Arrow from B to A labeled 0.4.

Uncertainties

The speaker does not explicitly read this number aloud in the clip, but it is visible on the diagram.

Symbol

0.4

Meaning

Transition probability from state B to state A.

Domain

probability in [0,1]

0.6

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Self-loop arrow on state B labeled 0.6.

Uncertainties

The speaker does not explicitly read this number aloud in the clip, but it is visible on the diagram.

Symbol

0.6

Meaning

Transition probability from state B to state B.

Domain

probability in [0,1]

$S_0$

Clear evidence

Shown in the video

Evidence

Formula
Observation
Formula appears as $S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ .
Audio
Observation
Speaker introduces S naught as the initial state of the system and says it is a vector with a one and a zero.

Symbol

$S_0$

Meaning

Initial state vector encoding the starting probabilities of the Markov chain.

Domain

column vector of state probabilities

1

Clear evidence

Shown in the video

Evidence

Formula
Observation
Top entry of $S_0$ is 1.
Audio
Observation
Speaker says the top component refers to state A and that 1 means a 100% probability it starts in state A.
Diagram
Observation
Annotation "Initially at A" points to the top entry.

Symbol

Meaning

Probability that the system starts in state A.

Domain

probability value

0

Clear evidence

Shown in the video

Evidence

Formula
Observation
Bottom entry of $S_0$ is 0.
Audio
Observation
Speaker says the bottom component refers to state B and that 0 means a 0% probability it starts in state B.
Diagram
Observation
Annotation "Initially not at B" points to the bottom entry.

Symbol

Meaning

Probability that the system starts in state B.

Domain

probability value

Initial State

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Curved annotation label "Initial State" points to $S_0$ .

Symbol

Initial State

Meaning

Name given to the vector $S_0$ .

Domain

terminology

$S_0$

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ with labels "Initial State", "Initially at A", and "Initially not at B".
Audio
Observation
The speaker refers to $S_0$ as the initial state.

Symbol

$S_0$

Meaning

Initial state vector of a two-state Markov chain; first component corresponds to state A and second to state B.

Domain

Column vector with nonnegative entries summing to 1 in this example.

Knowledge points · 12

Transition diagram for a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says, "Imagine I have a transition diagram... A transition diagram lists the possible states of some system... there's a bunch of arrows between those states and the numbers on each arrow describe the probability."
Diagram
Observation
Diagram shows two states A and B with directed arrows and numerical labels 0.75, 0.25, 0.4, 0.6.

Definition

Explanation

A transition diagram represents the possible states of a system as nodes and the allowed transitions between states as directed arrows. The number attached to an arrow gives the probability of moving from the tail state to the head state in one step. In the displayed example, the states are A and B, with self-loops and cross-transitions labeled by probabilities.

Formula

Conditions

The diagram is being used for a Markov chain.
Each arrow label is interpreted as a one-step transition probability.

Use of linear algebra to study Markov chains

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says they will use "some formalisms of linear algebra, the idea of matrices, the idea of vectors and how to multiply those" to make manipulations simpler when studying a Markov chain.

Uncertainties

No explicit matrix formula is shown within this 120-second clip.

Method

Explanation

The lecture announces that matrices and vectors will be used as a formalism for manipulating Markov-chain calculations more simply than direct verbal or diagrammatic reasoning.

Formula

Conditions

Applies to computations involving Markov chains.
Assumes familiarity with matrices, vectors, and their multiplication.

Prerequisites

Transition diagram for a two-state Markov chain

Initial state vector $S_0$

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ .
Audio
Observation
Speaker calls S naught the initial state of the system, says it is a vector, explains that the top component refers to state A and the bottom to state B, and interprets 1 and 0 as 100% and 0% starting probabilities.
Diagram
Observation
Labels "Initial State", "Initially at A", and "Initially not at B" annotate the vector entries.

Definition

Explanation

The initial state vector $S_0$ encodes the starting distribution over the states of the Markov chain. For the two-state example, the first component corresponds to state A and the second component corresponds to state B. The displayed vector $S_0 = (1, 0)^T$ asserts that the system starts in state A with probability 1 and in state B with probability 0.

Formula

S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}

Conditions

The system has two states in the displayed example.
Components are ordered consistently with the states, here A then B.
The entries are probabilities describing the initial distribution.

Prerequisites

Transition diagram for a two-state Markov chain

State vector in a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $S_1 = \begin{pmatrix} 0.75 \\ 0.25 \end{pmatrix}$ are shown on the board.
Audio
Observation
The speaker explains that the top number refers to state A and the second number to state B.

Definition

Explanation

A state vector records the probabilities of being in each state at a given time step. In this video, the first component corresponds to state A and the second to state B. Thus $S_0=(1,0)^T$ means the system starts certainly in A, while $S_1=(0.75,0.25)^T$ means that after one iteration there is a 75% chance of being in A and a 25% chance of being in B.

Formula

S_k = \begin{pmatrix} \Pr(X_k=A) \\ \Pr(X_k=B) \end{pmatrix}

Conditions

Two-state chain with states A and B.
Entries are probabilities.
In the examples shown, the entries sum to 1.

Transition matrix for a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Formula
Observation
$P = \begin{pmatrix} 0.75 & 0.4 \\ 0.25 & 0.6 \end{pmatrix}$ is written on the board.
Audio
Observation
The speaker says columns are denoted by A and B and rows are denoted by A and B, then interprets 0.75 as start in A and end in A, and 0.4 as start in B and end in A.

Definition

Explanation

The transition matrix collects the one-step transition probabilities into an array. In this video's convention, columns represent the starting state and rows represent the ending state. Therefore the entry in row A, column A is $\Pr(A\to A)$ , row A, column B is $\Pr(B\to A)$ , row B, column A is $\Pr(A\to B)$ , and row B, column B is $\Pr(B\to B)$ .

Formula

P = \begin{pmatrix} 0.75 & 0.4 \\ 0.25 & 0.6 \end{pmatrix}

Conditions

Finite state space, here two states A and B.
Matrix entries are one-step transition probabilities.
This clip uses the column-start / row-end convention.

Prerequisites

State vector in a two-state Markov chain

One-step state update using the transition matrix

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $S_1 = P S_0$ .
Audio
Observation
The speaker says, "The formula to go from S naught to S one is just multiplying," introducing matrix algebra as the method for updating states.

Uncertainties

The full spoken sentence is cut off before the speaker explicitly says "by P".
The numerical verification of the product is not shown within this clip.

Formula

Explanation

The next state vector is obtained by multiplying the transition matrix by the current state vector. This converts the diagram-based reading of one-step behavior into an algebraic rule that can be iterated efficiently.

Formula

S_1 = P S_0

Conditions

Use the same state ordering in $P$ and $S_k$ .
In this clip, $P$ is 2-by-2 and $S_k$ is a 2-by-1 column vector.

Prerequisites

State vector in a two-state Markov chain
Transition matrix for a two-state Markov chain

One-step state update rule

Clear evidence

Shown in the video

Evidence

Formula
Observation
Equation $S_1 = P S_0$ is shown on screen.
Audio
Observation
Speaker explains that the next state is obtained by multiplying the transition matrix by the current state vector.

Formula

Explanation

The video presents the first-step evolution of a Markov chain as a matrix-vector product: the state vector at step 1 equals the transition matrix times the initial state vector.

Formula

S_1 = P S_0

Conditions

$P$ is the transition matrix.
$S_0$ is the initial state vector.
$S_1$ is the resulting state vector after one step.

Matrix-vector multiplication as the computational mechanism

Clear evidence

Shown in the video

Evidence

Formula
Observation
Expanded equation $\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$ is displayed.
Audio
Observation
Speaker says, "This is matrix vector multiplication."

Method

Explanation

The speaker identifies the operation used to compute the next state as matrix-vector multiplication, and states that performing it yields the predicted vector on the left.

Formula

Conditions

Requires familiarity with multiplying a matrix by a column vector.

Prerequisites

One-step state update rule

Two-step state update rule

Clear evidence

Shown in the video

Evidence

Formula
Observation
Equation $S_2 = P S_1$ appears on screen.
Audio
Observation
Speaker says the second state is found by taking the transition matrix and multiplying it now to the $S_1$ .

Formula

Explanation

The next iterate is obtained by applying the same transition matrix again, now to the previous state vector rather than to the initial vector.

Formula

S_2 = P S_1

Conditions

$S_1$ must already be known.
The same transition matrix $P$ is reused.

Prerequisites

One-step state update rule

General n-step state formula

Clear evidence

Shown in the video

Evidence

Formula
Observation
Boxed equation $S_n = P^n S_0$ appears near the end.
Audio
Observation
Speaker generalizes from $S_2 = P^2 S_0$ to the nth state.

Formula

Explanation

By iterating the same transition matrix, the state after $n$ steps is given by $P^n$ acting on the initial state vector.

Formula

S_n = P^n S_0

Conditions

$n$ is a step index.
$P$ is the transition matrix.
$S_0$ is the initial state vector.

Prerequisites

Two-step state update rule

General formula for the nth state in a Markov chain

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board displays $S_2 = P S_1 = P^2 S_0$ and the boxed formula $S_n = P^n S_0$ .
Audio
Observation
The speaker says, 'And then multiply by n transition matrices, P to the n,' and explains that the nth state is the transition matrix multiplied n times by the initial state.

Formula

Explanation

The clip presents the rule that the state after n steps is obtained by applying the transition matrix n times to the initial state vector. The displayed two-step identity $S_2 = P S_1 = P^2 S_0$ motivates the boxed general formula $S_n = P^n S_0$ .

Formula

S_n = P^n S_0

Conditions

P is the transition matrix.
$S_0$ is the initial state vector.
n is the number of transitions.

Three-step method for solving Markov process problems

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker lists three main steps: write down the transition diagram, put it into matrix form, and figure out any future state by multiplication of that matrix a sufficient number of times.

Method

Explanation

The video summarizes a practical workflow for Markov-chain problems: first draw the transition diagram to collect all numbers, then convert that diagram into a matrix, and finally compute future states by repeated matrix multiplication using the state-power formula.

Formula

Conditions

The problem concerns a Markov process or Markov chain.
A transition diagram and an initial state are available or can be constructed.

Prerequisites

General formula for the nth state in a Markov chain

Claims and conditions · 5

Arrow labels represent one-step transition probabilities

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker states that the numbers on each arrow describe the probability and gives the example that 0.75 on the arrow from A to A means a 75% probability that state A transitions to state A in the next iteration.
Diagram
Observation
Self-loop on A labeled 0.75.

Proposition

Statement

In the transition diagram, each arrow label gives the probability that the system moves from the source state to the target state in one iteration.

Hypotheses

The diagram is a transition diagram for a Markov chain.
The arrow connects a source state to a target state.

Quantifiers

For each labeled arrow in the displayed transition diagram.

State-vector components correspond to states in order

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says the top component refers to state A and the bottom component to state B, and adds that if there were more components such as A, B, C, then the vector would have three components.

Proposition

Statement

In the two-state example, the first component of the state vector corresponds to state A and the second to state B; with more states, the vector has one component per state.

Hypotheses

The system is represented by a state vector.
The ordering of states is fixed.

Quantifiers

For the displayed two-state example and the stated extension to more states.

The displayed $S_0$ is an asserted initial condition, not derived

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says the fact that it says 1 and 0 means there is a 100% probability it starts in state A and a 0% probability it starts in state B, and adds, "I've just asserted this."
Formula
Observation
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ .

Proposition

Statement

The vector $S_0 = (1, 0)^T$ is introduced as an assertion about the starting condition of the system, not as a consequence of the transition probabilities.

Hypotheses

The chosen initial condition is that the system starts in state A.

Quantifiers

For the specific example shown in this clip.

Matrices provide an efficient way to iterate Markov-chain states

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker asks how to go from $S_0$ to $S_1$ , then from $S_1$ to $S_2$ , or $S_2$ to $S_{100}$ , and says there is a process to do those manipulations efficiently using a matrix.

Proposition

Statement

Using a transition matrix gives an efficient algebraic process for moving from one state vector to later state vectors such as $S_2$ or $S_{100}$ , rather than re-reading the diagram each time.

Hypotheses

The Markov chain has fixed one-step transition probabilities.
State vectors are represented in a consistent order.

Quantifiers

For the two-state example shown, and by extension for repeated iterations of the same chain.

Matrix method agrees with earlier tree-diagram computation

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says that in the first video on Markov processes they did the computation via a tree diagram, and that using this matrix method gives the same answer as before.

Uncertainties

The referenced earlier tree-diagram computation is not shown in this clip.

Proposition

Statement

For this example, computing the state vector by matrix multiplication gives the same result as the earlier tree-diagram method.

Hypotheses

The same Markov chain and initial state are being considered.
The earlier tree-diagram method is valid for that computation.

Quantifiers

For the example discussed in the video.

Derivations and proofs · 6

Interpreting the 0.75 self-loop on state A

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker explains the meaning of the 0.75 label on the A-to-A arrow.
Diagram
Observation
Self-loop on A labeled 0.75.

Intuitive argument

Steps

Expression
$0.75$
Explanation
Read the numerical label on the arrow from state A back to state A.
Justification
The speaker states that numbers on arrows describe transition probabilities.
Shown in the video
Expression
$P(\text{next state}=A \mid \text{current state}=A)=0.75$
Explanation
Translate the label into a conditional one-step transition statement.
Justification
The speaker says this represents a 75% probability that state A in one iteration transitions to state A in the next.
Derived from the video

Conclusion

The 0.75 label means there is a 75% chance that a system currently in state A remains in state A after one step.

Deriving $S_1$ directly from the initial state and the diagram

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The transition diagram shows A with self-loop 0.75 and arrow to B with 0.25.
Formula
Observation
$S_0 = \begin{pmatrix}1\\0\end{pmatrix}$ and $S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$ are displayed.
Audio
Observation
The speaker says that if we start at $S_0$ and go one iteration into the future, we are then at $S_1$ , and that 0.75 is the chance of being back in state A after one iteration.

Intuitive argument

Steps

Expression
$S_0 = \begin{pmatrix}1\\0\end{pmatrix}$
Explanation
The system starts with probability 1 in state A and probability 0 in state B.
Justification
Directly read from the board labels "Initially at A" and "Initially not at B".
Shown in the video
Expression
$\Pr(A\to A)=0.75,\quad \Pr(A\to B)=0.25$
Explanation
From state A, the one-step probabilities are 0.75 to remain in A and 0.25 to move to B.
Justification
Read from the transition diagram arrows leaving A.
Shown in the video
Expression
$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$
Explanation
After one iteration, the probability of being in A is 0.75 and in B is 0.25.
Justification
Since the chain starts certainly in A, the next-state distribution is exactly the outgoing probabilities from A.
Shown in the video

Conclusion

For this example, $S_1$ can be read directly from the diagram because the initial state is purely A.

Constructing the transition matrix from the diagram

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The diagram shows four transition probabilities: 0.75, 0.25, 0.4, 0.6.
Formula
Observation
The board writes $P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6\end{pmatrix}$ with row and column labels A and B.
Audio
Observation
The speaker says to pull the four numbers together into a matrix and explains that columns denote starting states and rows denote ending states.

Visual argument

Steps

Expression
$\Pr(A\to A)=0.75$
Explanation
Place the probability of starting in A and ending in A in row A, column A.
Justification
The speaker identifies 0.75 as start in state A and end up in state A.
Shown in the video
Expression
$\Pr(A\to B)=0.25$
Explanation
Place the probability of starting in A and ending in B in row B, column A.
Justification
This follows the video's row-end / column-start convention and the arrow from A to B in the diagram.
Shown in the video
Expression
$\Pr(B\to A)=0.4$
Explanation
Place the probability of starting in B and ending in A in row A, column B.
Justification
The speaker explicitly says 0.4 is in the second column and represents starting in B and ending in A.
Shown in the video
Expression
$\Pr(B\to B)=0.6$
Explanation
Place the probability of starting in B and ending in B in row B, column B.
Justification
Read from the self-loop at B in the diagram.
Shown in the video
Expression
$P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6\end{pmatrix}$
Explanation
Collect the four entries into a 2-by-2 matrix.
Justification
This is the transition matrix written on the board.
Shown in the video

Conclusion

The four diagram probabilities assemble into the transition matrix $P$ under the column-start / row-end convention used in the clip.

Derivation of $S_2 = P^2 S_0$ by substitution

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says, "If I just look at this S two is P times S one, S one was P times S naught. So in other words what this is is P squared times S naught."
Formula
Observation
On-screen text updates from $S_2 = P S_1$ to $S_2 = P S_1 = P^2 S_0$ .

Proof

Steps

Expression
$S_2 = P S_1$
Explanation
Start from the two-step recurrence shown on screen.
Justification
Given in the video as the definition of the next state.
Shown in the video
Expression
$S_1 = P S_0$
Explanation
Replace $S_1$ using the one-step update rule.
Justification
Previously established in the video.
Shown in the video
Expression
$S_2 = P(P S_0) = P^2 S_0$
Explanation
Substitute and combine the two applications of $P$ .
Justification
Associativity of matrix multiplication and the definition of $P^2$ ; the final equality is shown on screen.
Shown in the video

Conclusion

The second state can be written directly as $S_2 = P^2 S_0$ .

Generalization from repeated multiplication to $S_n = P^n S_0$

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says that in general, to figure out the nth state, one starts at the initial state and applies the transition matrix repeatedly.
Formula
Observation
Boxed formula $S_n = P^n S_0$ appears on screen.

Uncertainties

The clip states the general pattern but does not provide a formal induction proof.

Intuitive argument

Steps

Expression
$S_1 = P S_0$
Explanation
One step uses $P$ once.
Justification
Shown earlier in the clip.
Shown in the video
Expression
$S_2 = P^2 S_0$
Explanation
Two steps use $P$ twice.
Justification
Derived immediately before the general statement.
Shown in the video
Expression
$S_n = P^n S_0$
Explanation
Continuing the same pattern gives the nth state.
Justification
Pattern extension stated by the speaker and displayed in the boxed formula.
Shown in the video

Conclusion

The nth state vector is obtained by applying the transition matrix $n$ times to the initial state vector.

Derivation of the nth-state formula from repeated one-step updates

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $S_2 = P S_1 = P^2 S_0$ and $S_n = P^n S_0$ .
Audio
Observation
The speaker says the nth state is the transition matrix multiplied n times by the initial state.

Intuitive argument

Steps

Expression
$S_2 = P S_1 = P^2 S_0$
Explanation
The displayed identity shows the second state obtained by applying P once to $S_1$ and equivalently applying P twice to $S_0$ .
Justification
Directly visible on the board.
Shown in the video
Expression
$S_n = P^n S_0$
Explanation
The boxed formula generalizes the same pattern to n steps.
Justification
Directly visible on the board and stated in the audio as multiplying by n transition matrices.
Shown in the video

Conclusion

The nth state vector is computed as $S_n = P^n S_0$ .

Worked examples · 5

Worked setup of a two-state Markov chain with initial state A

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Two-state transition diagram with A and B and probabilities 0.75, 0.25, 0.4, 0.6.
Formula
Observation
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ with annotations "Initial State", "Initially at A", and "Initially not at B".
Audio
Observation
Speaker uses this exact example to explain transition diagrams and the initial state vector.

Problem

Given a two-state Markov chain with states A and B and the displayed transition probabilities, encode the starting condition of the system using an initial state vector.

Given

States are A and B.
Transition probabilities shown are $A\to A = 0.75$ , $A\to B = 0.25$ , $B\to A = 0.4$ , $B\to B = 0.6$ .
The system is asserted to start in state A.

Goal

Write the initial state vector $S_0$ and interpret its components.

Steps

Expression
$\text{states} = \{A,B\}$
Explanation
Identify the state space from the transition diagram.
Justification
The speaker says a transition diagram lists the possible states of the system.
Shown in the video
Expression
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$
Explanation
Introduce the initial state vector for the example.
Justification
The formula is displayed on screen and named by the speaker as the initial state.
Shown in the video
Expression
$\text{top component} \leftrightarrow A,\quad \text{bottom component} \leftrightarrow B$
Explanation
Assign vector positions to states.
Justification
The speaker explicitly says the top component refers to state A and the bottom component to state B.
Shown in the video
Expression
$1 = P(\text{start in }A),\quad 0 = P(\text{start in }B)$
Explanation
Interpret the numerical entries as starting probabilities.
Justification
The speaker says 1 and 0 mean 100% probability of starting in A and 0% probability of starting in B.
Shown in the video

Answer

$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ , meaning the system starts in state A with probability 1 and in state B with probability 0.

Verification

The annotations "Initially at A" and "Initially not at B" match the entries 1 and 0, and the speaker states that these probabilities add to 1 in the general case.

Example: start in A and find the state after one iteration

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_0 = \begin{pmatrix}1\\0\end{pmatrix}$ and $S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$ are shown.
Diagram
Observation
The transition diagram shows $A\to A = 0.75$ and $A\to B = 0.25$ .
Audio
Observation
The speaker explains that starting at $S_0$ and going one iteration leads to $S_1$ , with 75% chance of being back in A.

Uncertainties

The clip does not show the explicit matrix multiplication arithmetic for this example.

Problem

Given a two-state Markov chain with states A and B, start in state A and determine the state vector after one iteration.

Given

$S_0 = \begin{pmatrix}1\\0\end{pmatrix}$
Transition probabilities from A: $\Pr(A\to A)=0.75$ , $\Pr(A\to B)=0.25$

Goal

Compute or identify $S_1$ .

Steps

Expression
$\text{Start certainly in }A$
Explanation
Because the first component of $S_0$ is 1 and the second is 0, the system begins in state A with certainty.
Justification
Read from the displayed initial state vector and its labels.
Shown in the video
Expression
$\Pr(\text{in }A\text{ after 1 step})=0.75,\quad \Pr(\text{in }B\text{ after 1 step})=0.25$
Explanation
The one-step probabilities from A become the probabilities of being in A and B after one iteration.
Justification
Taken from the transition diagram arrows leaving A.
Shown in the video
Expression
$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$
Explanation
Assemble these probabilities into the next state vector.
Justification
Displayed on the board and explained verbally by the speaker.
Shown in the video

Answer

$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$

Verification

The entries match the outgoing probabilities from A in the diagram, and they sum to 1.

Compute $S_1$ from $P$ and $S_0$

Clear evidence

Shown in the video

Evidence

Formula
Observation
Expanded equation $\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$ is shown.
Audio
Observation
Speaker says the actual information is $S_1 = (0.75, 0.25)$ , $P$ is that matrix, and $S_0 = (1,0)$ .

Problem

Use the transition matrix and initial state vector to compute the state after one step.

Given

$P = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}$
$S_0 = \begin{pmatrix}1\\0\end{pmatrix}$

Goal

Find $S_1$ .

Steps

Expression
$S_1 = P S_0$
Explanation
Apply the one-step update rule.
Justification
Formula shown on screen.
Shown in the video
Expression
$\begin{pmatrix}0.75\\0.25\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}1\\0\end{pmatrix}$
Explanation
Substitute the numerical matrix and initial vector.
Justification
Displayed explicitly in the expanded equation.
Shown in the video
Expression
$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$
Explanation
The product gives the predicted next-state vector.
Justification
Stated by the speaker as the result of the matrix-vector multiplication.
Shown in the video

Answer

$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$

Verification

The speaker notes that this matches the vector previously predicted from the diagram.

Compute $S_2$ from $P$ and $S_1$

Clear evidence

Shown in the video

Evidence

Formula
Observation
Expanded equation $\begin{pmatrix}0.66\\0.34\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}0.75\\0.25\end{pmatrix}$ is shown.
Audio
Observation
Speaker says that doing the matrix-vector multiplication gives $0.66, 0.34$ for $S_2$ .

Problem

Use the same transition matrix and the previously computed $S_1$ to find the state after two steps.

Given

$P = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}$
$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$

Goal

Find $S_2$ .

Steps

Expression
$S_2 = P S_1$
Explanation
Apply the two-step recurrence.
Justification
Formula shown on screen.
Shown in the video
Expression
$\begin{pmatrix}0.66\\0.34\end{pmatrix} = \begin{pmatrix}0.75 & 0.4\\ 0.25 & 0.6\end{pmatrix}\begin{pmatrix}0.75\\0.25\end{pmatrix}$
Explanation
Substitute the numerical values.
Justification
Displayed explicitly in the expanded equation.
Shown in the video
Expression
$S_2 = \begin{pmatrix}0.66\\0.34\end{pmatrix}$
Explanation
The product gives the second-step state vector.
Justification
Stated by the speaker.
Shown in the video

Answer

$S_2 = \begin{pmatrix}0.66\\0.34\end{pmatrix}$

Verification

The speaker says this agrees with the earlier tree-diagram computation from a previous video.

Two-state Markov diagram used as the visual example

Clear evidence

Shown in the video

Evidence

Diagram
Observation
A two-state diagram is shown with nodes A and B, self-loops labeled 0.75 at A and 0.6 at B, an arrow A to B labeled 0.25, and an arrow B to A labeled 0.4.
Audio
Observation
The speaker refers generally to writing down a transition diagram so you get all the numbers.

Uncertainties

No numerical future-state computation is actually performed in this clip.
The clip does not show the matrix entries assembled from these diagram labels.

Problem

A two-state Markov-chain diagram with states A and B is displayed while the lecturer explains how to compute future states.

Given

States A and B are shown.
Self-loop A to A is labeled 0.75.
Transition A to B is labeled 0.25.
Transition B to A is labeled 0.4.
Self-loop B to B is labeled 0.6.
The board also shows $S_2 = P S_1 = P^2 S_0$ and $S_n = P^n S_0$ .

Goal

Illustrate the setup for Markov-chain manipulation using a transition diagram and the state-power formula.

Steps

Expression
Explanation
The visual example provides a concrete transition diagram with numeric edge labels for states A and B.
Justification
Visible throughout the clip on the left side of the board.
Shown in the video
Expression
$S_n = P^n S_0$
Explanation
The lecturer pairs the diagram with the general formula for computing future states from the initial state.
Justification
Visible on the board and stated in the audio.
Shown in the video

Answer

The clip presents the diagram and formula but does not compute a specific numerical state vector.

Verification

No worked numerical verification is shown in this segment.

Visual events · 9

Opening title card

Clear evidence

Shown in the video

Evidence

Diagram
Observation
White handwritten-style title "MARKOV CHAINS" appears on a black background.

Objects

black background
white title text "MARKOV CHAINS"

Changes

Title text is displayed at the beginning of the clip.

Invariants

No mathematical notation beyond the title is shown yet.

Interpretation

The clip opens by naming the topic as Markov chains.

Two-state transition diagram remains on screen while the speaker explains it

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Upper-left diagram shows blue node A, purple node B, self-loop on A labeled 0.75, arrow $A\to B$ labeled 0.25, arrow $B\to A$ labeled 0.4, and self-loop on B labeled 0.6.
Audio
Observation
Speaker explains that the diagram lists possible states and that arrow numbers describe probabilities.

Objects

state node A
state node B
directed arrows
probability labels 0.75, 0.25, 0.4, 0.6

Changes

The speaker gestures toward the diagram while explaining states and transition probabilities.

Invariants

The diagram structure and labels stay fixed during this interval.

Interpretation

The visual establishes the state space and one-step transition probabilities for the example Markov chain.

Initial state vector appears

Clear evidence

Shown in the video

Evidence

Formula
Observation
Formula $S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ appears below the transition diagram.

Objects

formula $S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$

Changes

A new formula is added beneath the transition diagram.

Invariants

The transition diagram above remains unchanged.

Interpretation

The lecture shifts from describing transitions to encoding the starting condition as a vector.

Annotation names $S_0$ as the initial state

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Curved annotation "Initial State" points to $S_0$ .

Objects

label "Initial State"
arrow to $S_0$

Changes

A curved label is added pointing to the vector.

Invariants

The vector entries remain 1 and 0.

Interpretation

The visual identifies $S_0$ as the object representing the system's starting distribution.

Entrywise interpretation of the initial vector is annotated

Clear evidence

Shown in the video

Evidence

Diagram
Observation
Annotations "Initially at A" and "Initially not at B" appear, pointing respectively to the top and bottom entries of $S_0$ .

Objects

label "Initially at A"
label "Initially not at B"
vector entries 1 and 0

Changes

Two new labels are added, each pointing to one component of $S_0$ .

Invariants

The transition diagram and the formula itself remain unchanged.

Interpretation

The annotations map the vector components to the corresponding states and their starting probabilities.

Progressive board construction of the Markov-chain example

Clear evidence

Shown in the video

Evidence

Diagram
Observation
A blackboard-style layout shows a transition diagram at upper left and state vectors / matrix equations below it while the presenter stands at right.
Animation
Observation
Labels and formulas appear progressively: first $S_0$ , then $S_1$ , then the matrix $P$ , then $S_1 = P S_0$ .

Objects

Transition diagram with nodes A and B
State vectors $S_0$ and $S_1$
Transition matrix $P$
Equation $S_1 = P S_0$
Presenter gesturing beside the board

Changes

At the beginning, the diagram and $S_0$ are already visible.
Around 2 seconds, $S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$ appears below $S_0$ .
Around 18 seconds, explanatory labels "Probability of A" and "Probability of B" are added near $S_1$ .
Around 37 seconds, a yellow arrow with a question mark is drawn between $S_0$ and $S_1$ to emphasize the unknown transformation.
Around 71 seconds, the matrix entries are assembled into $P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6\end{pmatrix}$ .
Around 81 seconds, row and column labels A and B are added to clarify the matrix indexing.
Around 112 seconds, the equation $S_1 = P S_0$ is written.

Invariants

The underlying two-state diagram remains the same throughout the clip.
The state ordering stays A first and B second in the vectors and matrix labels.

Interpretation

The visual sequence motivates the shift from reading probabilities off a diagram to encoding them algebraically in a transition matrix and then stating the update rule $S_1 = P S_0$ .

Persistent two-state Markov diagram

Clear evidence

Shown in the video

Evidence

Diagram
Observation
A two-state diagram remains visible throughout: node A has a self-loop labeled 0.75, node B has a self-loop labeled 0.6, an arrow $A\to B$ is labeled 0.25, and an arrow $B\to A$ is labeled 0.4.

Objects

State node A
State node B
Self-loop at A labeled 0.75
Self-loop at B labeled 0.6
Arrow $A\to B$ labeled 0.25
Arrow $B\to A$ labeled 0.4

Changes

No change in the diagram during the clip.

Invariants

The transition probabilities encoded by the diagram remain fixed while the algebraic formulas are developed.

Interpretation

The diagram supplies the transition probabilities that are assembled into the matrix $P$ used in the equations.

Sequential buildup of state-evolution formulas

Clear evidence

Shown in the video

Evidence

Animation
Observation
Equations appear sequentially: first $S_1 = P S_0$ , then its numeric expansion, then $S_2 = P S_1$ , then its numeric expansion, then $S_2 = P S_1 = P^2 S_0$ , and finally the boxed $S_n = P^n S_0$ .

Objects

Symbolic one-step equation
Numeric one-step equation
Symbolic two-step equation
Numeric two-step equation
Substituted form with $P^2$
Boxed general formula

Changes

The board progresses from one-step notation to explicit numerical multiplication, then to two-step notation, then to substitution showing $P^2$ , and finally to the general $n$ -step formula.

Invariants

The same transition matrix $P$ is reused at each step.

Interpretation

The visual sequence demonstrates how repeated application of the same matrix produces higher-step state vectors and motivates the power notation $P^n$ .

Static Markov-chain board during spoken summary

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The left side of the frame continuously shows a two-state transition diagram with A and B and the equations $S_2 = P S_1 = P^2 S_0$ and boxed $S_n = P^n S_0$ .
Animation
Observation
The presenter gestures with his hands and points toward the board while speaking; the mathematical content on the board remains unchanged.

Objects

Blue node A
Purple node B
Self-loop at A labeled 0.75
Arrow from A to B labeled 0.25
Arrow from B to A labeled 0.4
Self-loop at B labeled 0.6
Equation $S_2 = P S_1 = P^2 S_0$
Boxed equation $S_n = P^n S_0$
Presenter gesturing on the right

Changes

The presenter changes hand gestures and occasionally points toward the formulas.
The board content itself does not change during the clip.

Invariants

The transition diagram remains visible throughout.
The equations $S_2 = P S_1 = P^2 S_0$ and $S_n = P^n S_0$ remain on screen throughout.

Interpretation

The visual layout reinforces the spoken summary: a concrete two-state Markov diagram is paired with the general matrix-power rule for future states.

Misconceptions · 6

Confusing the initial state with something forced by the transition probabilities

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says, "I've just asserted this," after interpreting the 1 and 0 entries of $S_0$ .

Misconception

One might think the vector $S_0 = (1,0)^T$ is determined by the transition diagram itself.

Clarification

The speaker explicitly says this initial condition is asserted for the example; other initial distributions are possible, including ones with two different probabilities that add up to 1.

Thinking a Markov-chain state vector always has only two entries

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker notes that if there were more components in the system, such as A, B, C, then the vector would have three components.

Misconception

Because the example shows a two-entry vector, one might infer that state vectors are limited to two components.

Clarification

The number of components equals the number of states; the two-entry form here is only because the example has states A and B.

Confusing the direction convention of a transition matrix

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker explicitly says columns are denoted by A and B and rows are denoted by A and B, then interprets 0.4 as starting in B and ending in A.
Formula
Observation
The displayed matrix is $P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6\end{pmatrix}$ , which matches the column-start / row-end convention.

Misconception

One may assume the first index always denotes the starting state, as in some textbooks' row-stochastic convention.

Clarification

In this clip, the convention is the opposite: columns represent the starting state and rows represent the ending state, so the entry in row A, column B is $\Pr(B\to A)=0.4$ .

Thinking the diagram alone is enough for many steps

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker notes that in this case $S_1$ could be found by looking at the diagram, but then asks how to go from $S_1$ to $S_2$ or $S_2$ to $S_{100}$ efficiently.

Misconception

Reading probabilities directly from the transition diagram is sufficient for all future states.

Clarification

The clip introduces matrix multiplication precisely because repeated updates such as $S_2$ or $S_{100}$ are more efficiently handled algebraically than by direct diagram inspection.

For one step, the diagram alone may suffice

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says, "Thus far we haven't really gotten a big improvement because you were capable of coming up with the S one just by looking at the diagram. We didn't need to do all this formalism."

Misconception

That the matrix formalism is already necessary just to compute $S_1$ .

Clarification

The video explicitly says that for the first step one could read off the answer from the diagram without the matrix machinery.

Matrix-vector multiplication is assumed background

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says that if you do not know matrix-vector multiplication, there is another introductory video, and he will assume that skill for the rest of this video.

Misconception

That this clip teaches matrix-vector multiplication from scratch.

Clarification

The speaker treats matrix-vector multiplication as prerequisite knowledge and directs viewers to a separate introductory video.

Concept relations · 11

Transition diagram for a two-state Markov chain → Initial state vector $S_0$

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker first explains the transition diagram and states A and B, then introduces $S_0$ whose components refer to those same states.
Diagram
Observation
The vector annotations point to A and B already defined in the diagram above.

Prerequisite

Explanation

Understanding which states exist and how they are ordered in the transition diagram is needed before interpreting the components of $S_0$ .

Use of linear algebra to study Markov chains → Transition diagram for a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Audio
Observation
Speaker says matrices and vectors will be used to simplify manipulations when studying a Markov chain.

Application

Explanation

The announced linear-algebra formalism is intended as a tool for analyzing the Markov-chain setup shown by the transition diagram.

Initial state vector $S_0$ → Transition diagram for a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_0$ is presented as a vector of starting probabilities.
Audio
Observation
Speaker generalizes that with more states the vector would have more components.

Special case

Explanation

For this two-state diagram, the general idea of a state vector specializes to a two-component column vector ordered by states A and B.

State vector in a two-state Markov chain → Transition matrix for a two-state Markov chain

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker first defines $S_0$ and $S_1$ , then builds the matrix $P$ to manipulate between them.
Formula
Observation
The board moves from state vectors to the matrix equation $S_1 = P S_0$ .

Prerequisite

Explanation

Understanding what a state vector represents is needed before interpreting how the transition matrix acts on it.

Transition matrix for a two-state Markov chain → One-step state update using the transition matrix

Clear evidence

Shown in the video

Evidence

Formula
Observation
After writing $P$ , the board immediately states $S_1 = P S_0$ .
Audio
Observation
The speaker says the formula to go from $S_0$ to $S_1$ is just multiplying, introducing matrix algebra as the mechanism.

Application

Explanation

The transition matrix is the object used in the one-step update formula.

Transition matrix for a two-state Markov chain → Constructing the transition matrix from the diagram

Clear evidence

Shown in the video

Evidence

Diagram
Observation
The four arrow labels in the diagram are extracted into the matrix $P$ .
Audio
Observation
The speaker says to forget the rest of the diagram for a moment and pull those four numbers together.

Contains

Explanation

The transition matrix packages the same information displayed visually in the two-state transition diagram.

One-step state update rule → Two-step state update rule

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board moves from $S_1 = P S_0$ to $S_2 = P S_1$ .
Audio
Observation
Speaker explains that the second state is found by multiplying the transition matrix by $S_1$ .

Generalizes

Explanation

The two-step rule extends the one-step update by reusing the same transition matrix on the newly obtained state vector.

Two-step state update rule → General n-step state formula

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_2 = P S_1 = P^2 S_0$ appears, followed by the boxed $S_n = P^n S_0$ .
Audio
Observation
Speaker derives $P^2 S_0$ and then states the general nth-state formula.

Generalizes

Explanation

Writing $S_2$ as $P^2 S_0$ reveals the pattern that leads to the general formula $S_n = P^n S_0$ .

One-step state update rule → Compute $S_1$ from $P$ and $S_0$

Clear evidence

Shown in the video

Evidence

Formula
Observation
Numeric expansions of $S_1$ and $S_2$ are shown directly beneath the symbolic formulas.
Audio
Observation
Speaker uses the numbers to show that the matrix method reproduces the expected state vectors.

Application

Explanation

The one-step formula is applied numerically in the first worked example.

Compute $S_2$ from $P$ and $S_1$ → General n-step state formula

Clear evidence

Shown in the video

Evidence

Formula
Observation
The numeric $S_2$ example is followed by the substitution $S_2 = P^2 S_0$ and then the boxed general formula.

Proof dependency

Explanation

The concrete two-step computation motivates and supports the leap to the general $n$ -step expression.

Three-step method for solving Markov process problems → General formula for the nth state in a Markov chain

Clear evidence

Derived from the video

Evidence

Audio
Observation
The speaker says future states can be found by multiplication of the matrix a sufficient number of times, which matches the boxed formula $S_n = P^n S_0$ .

Application

Explanation

The three-step problem-solving method applies the nth-state formula in its final step, where future states are computed by repeated matrix multiplication.

Find an answer · 19

What does a transition diagram represent in a Markov chain?

Clear evidence

Derived from the video

Evidence

Audio
Observation
Speaker defines a transition diagram and explains arrow labels as probabilities.

Knowledge points

Transition diagram for a two-state Markov chain

What does the 0.75 label on the arrow from A to A mean?

Clear evidence

Derived from the video

Evidence

Audio
Observation
Speaker interprets the 0.75 self-loop on A.
Diagram
Observation
Self-loop labeled 0.75 on state A.

Knowledge points

Transition diagram for a two-state Markov chain
Arrow labels represent one-step transition probabilities

Why introduce matrices and vectors when studying Markov chains?

Clear evidence

Derived from the video

Evidence

Audio
Observation
Speaker says linear algebra formalisms make Markov-chain manipulations simpler.

Knowledge points

Use of linear algebra to study Markov chains

What is the initial state vector $S_0$ in this example?

Clear evidence

Derived from the video

Evidence

Formula
Observation
$S_0 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ .
Audio
Observation
Speaker calls $S_0$ the initial state of the system.

Knowledge points

Initial state vector $S_0$

Which state does each component of $S_0$ correspond to?

Clear evidence

Derived from the video

Evidence

Audio
Observation
Speaker says the top component refers to state A and the bottom component to state B.
Diagram
Observation
Annotations "Initially at A" and "Initially not at B".

Knowledge points

Initial state vector $S_0$
State-vector components correspond to states in order

Is $S_0 = (1,0)^T$ the only possible initial state for this Markov chain?

Clear evidence

Derived from the video

Evidence

Audio
Observation
Speaker says "I've just asserted this" and notes other initial probabilities could add to 1.

Knowledge points

The displayed $S_0$ is an asserted initial condition, not derived
Confusing the initial state with something forced by the transition probabilities

What does $S_1$ mean in a Markov chain example?

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_1 = \begin{pmatrix}0.75\\0.25\end{pmatrix}$ is shown.
Audio
Observation
The speaker defines $S_1$ as the state after one iteration.

Knowledge points

State vector in a two-state Markov chain
Example: start in A and find the state after one iteration

Why introduce a matrix instead of just reading the transition diagram?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker asks how to go from $S_1$ to $S_2$ or $S_2$ to $S_{100}$ efficiently and answers that a matrix is used.

Knowledge points

Matrices provide an efficient way to iterate Markov-chain states
Thinking the diagram alone is enough for many steps

In this video, what do the rows and columns of the transition matrix represent?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker explains columns as starting states and rows as ending states.
Formula
Observation
$P = \begin{pmatrix}0.75 & 0.4 \\ 0.25 & 0.6\end{pmatrix}$ is displayed with $A/B$ labels.

Knowledge points

Transition matrix for a two-state Markov chain
Confusing the direction convention of a transition matrix

How do you compute the next state vector from the current one?

Clear evidence

Shown in the video

Evidence

Formula
Observation
The board shows $S_1 = P S_0$ .
Audio
Observation
The speaker says the formula to go from $S_0$ to $S_1$ is just multiplying.

Uncertainties

The clip ends before the multiplication is carried out numerically.

Knowledge points

One-step state update using the transition matrix

What does the 0.4 entry in the transition matrix mean here?

Clear evidence

Shown in the video

Evidence

Audio
Observation
The speaker says 0.4 means if I start in state B, then I will end up in state A.
Formula
Observation
0.4 appears in row A, column B of $P$ .

Knowledge points

Transition matrix for a two-state Markov chain
Confusing the direction convention of a transition matrix

How is the next state vector computed from the transition matrix and initial state?

Clear evidence

Shown in the video

Evidence

Formula
Observation
$S_1 = P S_0$ is displayed prominently.

Knowledge points

One-step state update rule
Compute $S_1$ from $P$ and $S_0$

Coverage and review notes

Covered · Opening title card naming the topic MARKOV CHAINS; no additional mathematical content beyond the title.

Covered · Speaker situates this as a follow-up video on Markov chains while the two-state transition diagram is already visible.

Covered · Speaker announces use of matrices, vectors, and multiplication from linear algebra to simplify Markov-chain manipulations.

Covered · Speaker references a computational difficulty from the introduction video that this lesson will solve; no new formula is shown in this clip.

Covered · Transition diagram is explained: states A and B, arrows between states, and arrow labels as probabilities, with the 0.75 self-loop interpreted explicitly.

Covered · Speaker announces introduction of notation to describe different things, leading into the initial-state vector.

Covered · Formula $S_0 = (1,0)^T$ appears and is annotated as Initial State, Initially at A, and Initially not at B; speaker explains component order, probability interpretation, and that the choice is asserted rather than derived.

Covered · The clip introduces the two-state diagram, the initial vector $S_0$ , and the one-step vector $S_1$ , with verbal explanation of their meanings.

Covered · The speaker motivates the need for an algebraic method to move efficiently from $S_0$ to $S_1$ , then to $S_2$ and beyond.

Covered · The four transition probabilities are assembled into the matrix $P$ , and the column-start / row-end convention is explained.

Covered · The board displays $S_1 = P S_0$ and the speaker begins to describe matrix multiplication as the update rule; the clip ends before the arithmetic is shown.

Covered · Introduces $S_1 = P S_0$ , expands it numerically, and explains that the operation is matrix-vector multiplication.

Covered · Moves to $S_2 = P S_1$ , computes the numerical second state, and compares the method with an earlier tree-diagram approach.

Covered · Substitutes to obtain $S_2 = P^2 S_0$ and generalizes to the boxed formula $S_n = P^n S_0$ .

Covered · Audio explains multiplying by n transition matrices and the nth state formula while the board shows $S_2 = P S_1 = P^2 S_0$ and $S_n = P^n S_0$ .

Covered · The speaker summarizes the three main steps for solving Markov process problems.

Covered · Closing remarks only: the speaker hopes viewers enjoyed the video, asks for comments and likes, and signs off; no new mathematical content is introduced.

Explore the knowledge in this video

Reviewed subject paths

Questions this video answers

Meet the concept

Meet the concept

Meet the concept

Understand why

Understand why

Meet the concept

Meet the concept

Meet the concept

Markov Chains & Transition Matrices

The explanation, unpacked.

Chapters

Learning script

Knowledge cards

Transition diagram for a two-state Markov chain

Meaning of the 0.75 self-loop on state A

Why use matrices and vectors for Markov chains

Initial state vector S0S_0S0​

Component order in a state vector

Interpreting S0=(1,0)TS_0 = (1,0)^TS0​=(1,0)T

The initial condition is asserted, not derived

State vector SkS_kSk​ in a two-state Markov chain

Reading S1S_1S1​ directly from the transition diagram

Why matrices are introduced for Markov chains

Transition matrix convention used in this video

One-step update rule S1=PS0S_1=PS_0S1​=PS0​

One-step Markov state update

Matrix-vector multiplication is the computational engine

Why the formalism matters after step one

Two-step Markov state update

Matrix method agrees with the tree-diagram method

From two steps to P2P^2P2

General n-step formula

General formula for the nth state in a Markov chain

Three-step method for solving Markov chain problems

Static two-state Markov diagram shown during the summary

Detailed learning notes

MARKOV CHAINS

A

B

0.75

0.25

0.4

0.6

S0S_0S0​

1

0

Initial State

S0S_0S0​

Transition diagram for a two-state Markov chain

Use of linear algebra to study Markov chains

Initial state vector S0S_0S0​

State vector in a two-state Markov chain

Transition matrix for a two-state Markov chain

One-step state update using the transition matrix

One-step state update rule

Matrix-vector multiplication as the computational mechanism

Two-step state update rule

General n-step state formula

General formula for the nth state in a Markov chain

Three-step method for solving Markov process problems

Arrow labels represent one-step transition probabilities

State-vector components correspond to states in order

The displayed S0S_0S0​ is an asserted initial condition, not derived

Matrices provide an efficient way to iterate Markov-chain states

Matrix method agrees with earlier tree-diagram computation

Interpreting the 0.75 self-loop on state A

Deriving S1S_1S1​ directly from the initial state and the diagram

Constructing the transition matrix from the diagram

Derivation of S2=P2S0S_2 = P^2 S_0S2​=P2S0​ by substitution

Generalization from repeated multiplication to Sn=PnS0S_n = P^n S_0Sn​=PnS0​

Derivation of the nth-state formula from repeated one-step updates

Worked setup of a two-state Markov chain with initial state A

Example: start in A and find the state after one iteration

Compute S1S_1S1​ from PPP and S0S_0S0​

Compute S2S_2S2​ from PPP and S1S_1S1​

Two-state Markov diagram used as the visual example

Opening title card

Two-state transition diagram remains on screen while the speaker explains it

Initial state vector appears

Annotation names S0S_0S0​ as the initial state

Entrywise interpretation of the initial vector is annotated

Progressive board construction of the Markov-chain example

Persistent two-state Markov diagram

Sequential buildup of state-evolution formulas

Static Markov-chain board during spoken summary

Confusing the initial state with something forced by the transition probabilities

Thinking a Markov-chain state vector always has only two entries

Confusing the direction convention of a transition matrix

Initial state vector $S_0$

Interpreting $S_0 = (1,0)^T$

State vector $S_k$ in a two-state Markov chain

Reading $S_1$ directly from the transition diagram

One-step update rule $S_1=PS_0$

From two steps to $P^2$

$S_0$

$S_0$

Initial state vector $S_0$

The displayed $S_0$ is an asserted initial condition, not derived

Deriving $S_1$ directly from the initial state and the diagram

Derivation of $S_2 = P^2 S_0$ by substitution

Generalization from repeated multiplication to $S_n = P^n S_0$

Compute $S_1$ from $P$ and $S_0$

Compute $S_2$ from $P$ and $S_1$

Annotation names $S_0$ as the initial state

Transition diagram for a two-state Markov chain → Initial state vector $S_0$

Initial state vector $S_0$ → Transition diagram for a two-state Markov chain

One-step state update rule → Compute $S_1$ from $P$ and $S_0$

Compute $S_2$ from $P$ and $S_1$ → General n-step state formula

What is the initial state vector $S_0$ in this example?

Which state does each component of $S_0$ correspond to?

Is $S_0 = (1,0)^T$ the only possible initial state for this Markov chain?

What does $S_1$ mean in a Markov chain example?