Introducing Markov Chains

1. Caption evidence
   Observation

   The multiple-choice screen labels option A as Averagemont.

2. Audio
   Observation

   The narrator names the five cities and includes Averagemont.

A

City label for Averagemont in the route graph and answer choices.

One of the five states/cities in the example.

1. Caption evidence
   Observation

   The multiple-choice screen labels option B as Bayesville.

2. Audio
   Observation

   The narrator names Bayesville as one of the five cities.

B

City label for Bayesville in the route graph and answer choices.

One of the five states/cities in the example.

1. Caption evidence
   Observation

   The multiple-choice screen labels option C as Continuopolis.

2. Audio
   Observation

   The narrator names Continuopolis as one of the five cities.

C

City label for Continuopolis in the route graph and answer choices.

One of the five states/cities in the example.

1. Caption evidence
   Observation

   The multiple-choice screen labels option D as Discretetown.

2. Audio
   Observation

   The narrator names Discretetown as one of the five cities.

D

City label for Discretetown in the route graph and answer choices.

One of the five states/cities in the example.

1. Caption evidence
   Observation

   The multiple-choice screen labels option E as East Vandermonde.

2. Audio
   Observation

   The narrator names East Vandermonde as one of the five cities.

E

City label for East Vandermonde in the route graph and answer choices.

One of the five states/cities in the example.

1. Caption evidence
   Observation

   The multiple-choice screen shows option F as About equal.

2. Audio
   Observation

   The narrator reads option F as about equal among the five cities.

F

Answer choice stating that long-run time is about equal among the five cities.

Multiple-choice option, not a city state.

1. Caption evidence
   Observation

   The multiple-choice screen shows option G as It depends.

2. Audio
   Observation

   The narrator reads option G as it depends on which city she starts out in.

G

Answer choice stating that the result depends on the starting city.

Multiple-choice option, not a city state.

1. Diagram
   Observation

   The letter A labels the green spinner sector and later the green bar in the city-frequency chart.

2. Diagram
   Observation

   At 01:52.000 the map labels Averagemont with the same green color used for A.

A

City label for Averagemont; also the state index shown on the spinner and bar chart.

State label in the Markov-chain example

1. Diagram
   Observation

   The letter B labels the blue spinner sector and later the blue bar in the city-frequency chart.

2. Diagram
   Observation

   At 01:52.000 the map labels Bayesville with the same blue color used for B.

B

City label for Bayesville; also the state index shown on the spinner and bar chart.

State label in the Markov-chain example

1. Diagram
   Observation

   The letter C labels the magenta spinner sector and later the magenta bar in the city-frequency chart.

2. Diagram
   Observation

   At 01:52.000 the map labels Continuopolis with the same magenta color used for C.

C

City label for Continuopolis; also the state index shown on the spinner and bar chart.

State label in the Markov-chain example

1. Diagram
   Observation

   The letter D labels the orange spinner sector and later the orange bar in the city-frequency chart.

2. Diagram
   Observation

   At 01:52.000 the map labels Discretown with the same orange color used for D.

D

City label for Discretown; also the state index shown on the spinner and bar chart.

State label in the Markov-chain example

1. Diagram
   Observation

   The letter E labels the purple spinner sector and later the purple bar in the city-frequency chart.

2. Diagram
   Observation

   At 01:52.000 the map labels East Vandermonde with the same purple color used for E.

E

City label for East Vandermonde; also the state index shown on the spinner and bar chart.

State label in the Markov-chain example

1. Audio
   Observation

   The title of the video is Introducing Markov Chains.

2. Audio
   Observation

   The narrator describes Ana moving from city to city by randomly selecting among available direct connecting flights with equal probabilities.

3. Diagram
   Observation

   A map of five cities is connected by lines representing direct flights.

This clip introduces a Markov-chain-style process by describing a traveler who repeatedly moves among a finite set of cities. At each step, the next city is chosen only from the currently available direct connections, and each available connection is selected with equal probability. The example frames the long-run question as which city the traveler spends the most time in.

1. Finite set of states, here five cities.

2. Transitions are allowed only along existing direct connections.

3. From the current state, all available next states are equally likely.

4. The question concerns the long run rather than a single trip.

1. Audio
   Observation

   The narrator lists the five cities: Averagemont, Bayesville, Continuopolis, Discretetown, and East Vandermonde.

2. Diagram
   Observation

   The map displays exactly these five labeled city icons.

The state space is the set of five cities in the company's network: Averagemont, Bayesville, Continuopolis, Discretetown, and East Vandermonde. Each city functions as a state in the stochastic process being discussed.

1. The process is observed at discrete stops.

2. Each stop is one of the five named cities.

1. Markov chain introduced through a random travel process

1. Audio
   Observation

   The narrator says she selects the next city randomly from all available connecting flights with equal probabilities.

2. Animation
   Observation

   A spinner is used to choose among available destinations.

The transition method shown is: from the current city, identify all directly connected cities, then choose one of them uniformly at random. This is the mechanism that turns the map into a random walk on a graph.

1. Only direct connections from the current city are eligible.

2. All eligible destinations have the same probability.

3. The choice is made independently at each stop once the current city is known.

1. State space of the example

1. Audio
   Observation

   The narrator says not all cities are connected via direct flights to each other.

2. Audio
   Observation

   The narrator gives the example that there is a direct flight from Averagemont to Bayesville but not from Averagemont to East Vandermonde.

3. Diagram
   Observation

   Lines appear between some pairs of cities but not others.

The example emphasizes that the transition structure is constrained by a network of direct routes. Some pairs of cities are adjacent and some are not, so the process is a random walk on a specific graph rather than unrestricted movement among all states.

1. Adjacency means a direct flight exists.

2. Non-adjacent pairs cannot be reached in one step.

1. Uniform transition rule among available neighbors

1. Audio
   Observation

   The narrator says Bayesville is a major hub and has direct connections to all the other cities.

2. Diagram
   Observation

   The map highlights Bayesville with lines extending to the other four cities.

Bayesville is singled out as the central hub of the network because it has direct connections to every other city. In graph terms, it has the largest visible degree in this example, which makes it structurally important for the long-run behavior question.

1. Bayesville is adjacent to Averagemont, Continuopolis, Discretetown, and East Vandermonde.

1. Allowed transitions form a graph, not a complete graph

1. Audio
   Observation

   Narrator: "The process that Ana is following is an important example of a Markov chain."

2. Animation
   Observation

   A yellow airplane moves along a route between labeled cities on a map.

The video identifies Ana's repeated random travel process as an example of a Markov chain. The concrete setup is a traveler who uses spinners at different cities to choose the next destination.

1. The process is sequential in time.

2. Each step is a move from one city to another.

3. The next destination is chosen randomly.

1. Audio
   Observation

   Narrator: "The Markov property means that at each time point, determining the next city that Ana will go to depends only on where she currently is and the randomness of her spinner."

2. Caption evidence
   Observation

   On-screen text: "Markov Property: Given the present, the future is independent of the past."

3. Animation
   Observation

   Multiple spinners are shown simultaneously, one per current city, emphasizing that the next move is selected from the present state's spinner.

The Markov property states that the next state depends only on the current state, not on earlier history. In this clip the narrator expresses it as memorylessness: once the present location is known, the past route is irrelevant for predicting the next city.

1. There is a notion of current state or present location.

2. One asks how the next state is determined.

3. The claim is about dependence structure, not about equal probabilities.

1. Markov chain as the model for Ana's travel process

1. Audio
   Observation

   Narrator: "This is a form of memorylessness. Given her current location, her past history is irrelevant for predicting where she will go next."

2. Animation
   Observation

   The traveler is shown in Terminal B after earlier arrivals, visually separating the current stop from prior trips.

The video restates the Markov property as memorylessness. This means the predictive rule for the next city uses only the current location and the local random mechanism, not the sequence of previous stops.

1. Applies to the next-step prediction in the process being modeled.

1. Markov property

1. Audio
   Observation

   Narrator: "To answer Ana's question, let's consider the following problem. In the long run, as Ana continues this random process, what percentage of her time will she spend in each city?"

2. Caption evidence
   Observation

   Earlier speech bubble text: "What fraction of my time will I end up spending in Bayesville?"

3. Diagram
   Observation

   A passport page records many dated visits to different cities.

Instead of asking where the traveler will be after one specific trip, the video reframes the problem as a long-run frequency question: over many steps, what fraction of time is spent in each state? This is the empirical route to discussing stationary distributions.

1. The process is continued for many steps.

2. One measures relative time or visit frequency by city.

1. Markov chain as the model for Ana's travel process

1. Audio
   Observation

   Narrator: "Let's plot this after various numbers of trips. We have one bar for each city, and the height of each bar indicates the percentage of time Ana has spent in that city overall."

2. Diagram
   Observation

   A five-bar chart appears with bars labeled A, B, C, D, E and a vertical axis marked 10%, 20%, 30%, 40%.

The clip introduces a bar chart whose bars correspond to cities A through E. Each bar height represents the cumulative percentage of time spent in that city after a given number of trips, so the chart tracks an empirical distribution over states as the simulation proceeds.

1. One bar per state.

2. Bar height is a percentage of total time or visits up to the current trip count.

1. Long-run occupancy question for a Markov chain

1. Audio
   Observation

   Narrator: "At first the plot was fluctuating a lot, but now it seems to have stabilized. The distribution it has stabilized to is called the stationary distribution of the Markov chain."

2. Caption evidence
   Observation

   On-screen title: "Stationary Distribution".

3. Animation
   Observation

   The bar heights change during the simulation and then settle into a stable pattern near 1000 trips.

The stationary distribution is presented as the limiting distribution of the empirical city frequencies after many trips. Visually, it is the stable set of bar heights that the simulation approaches once early fluctuations die out.

1. The process is run for many steps.

2. The empirical distribution is observed to stabilize.

1. Empirical distribution plotted by trip count
2. Markov chain as the model for Ana's travel process

1. Audio
   Observation

   Narrator: "We are showing one particular run of the simulation, but Ana's process will always converge to the same stationary distribution."

2. Animation
   Observation

   After the stabilized bar chart, the map reappears with routes among all five cities.

1. The clip states convergence but does not prove it or list the conditions under which it holds.

The video claims that although the displayed animation is only one simulated run, the underlying process converges to the same stationary distribution. This frames stationary distribution as a property of the chain rather than of one particular trajectory.

1. Refers to repeated long-run behavior of the same Markov chain.

1. Stationary distribution

1. Audio
   Observation

   The narrator asks, In the long run, which city do you think she will spend the most time in?

2. Caption evidence
   Observation

   The screen displays the same question and seven answer choices.

For Ana's random travel process on the five-city network, the clip poses the question of which city she will spend the most time in in the long run.

1. Ana repeatedly moves from city to city using direct connections only.

2. At each step she chooses uniformly among available next cities.

3. The comparison is over the long run.

Over the long run, among the five cities in the network.

1. Audio
   Observation

   The narrator states that not all of these cities are connected via direct flights to each other.

2. Audio
   Observation

   The narrator gives the explicit example that Averagemont connects directly to Bayesville but not to East Vandermonde.

The five-city network does not contain a direct flight between every pair of cities; for example, Averagemont is directly connected to Bayesville but not to East Vandermonde.

1. Cities are vertices of the route network.

2. A line between two cities represents a direct flight.

There exists at least one pair of cities without a direct connection.

1. Audio
   Observation

   The narrator says Bayesville is a major hub and has direct connections to all the other cities.

2. Diagram
   Observation

   The map shows Bayesville linked to the other four cities.

Bayesville has direct connections to all the other cities in the example network.

1. The network consists of the five named cities.

2. Direct connections are represented by lines on the map.

For every other city in the five-city set, there is a direct connection to Bayesville.

1. Audio
   Observation

   Narrator says the next city depends only on where Ana currently is and the randomness of her spinner.

2. Caption evidence
   Observation

   On-screen text: "Given the present, the future is independent of the past."

For Ana's travel process, the next city depends only on the current city and the local random choice mechanism, not on earlier cities visited.

1. The process is a Markov chain.

2. The current state is known.

At each time point, conditional on the present state, the future next-step distribution is independent of the past.

1. Audio
   Observation

   Narrator: "The distribution it has stabilized to is called the stationary distribution of the Markov chain."

2. Caption evidence
   Observation

   Title card: "Stationary Distribution".

1. The clip defines it operationally through stabilization of the empirical bar chart rather than through an algebraic equation.

The distribution to which the long-run empirical city frequencies stabilize is called the stationary distribution of the Markov chain.

1. The random process is continued for many trips.

2. The empirical distribution is tracked over time.

In the long run, after many trips.

1. Audio
   Observation

   Narrator: "We are showing one particular run of the simulation, but Ana's process will always converge to the same stationary distribution."

1. No proof or explicit regularity assumptions are given in this clip.

Although the animation shows one particular simulated run, the process converges to the same stationary distribution regardless of that sampled trajectory.

1. The same Markov chain is being considered.

2. The process is run long enough.

For the process shown, across runs in the long run.

1. Audio
   Observation

   The narrator says, “No matter which city she starts out in,” followed by the statement about the percentage of time Ana spends in a city.

1. The clip does not prove this claim or state the formal Markov-chain conditions under which it holds.

In the presented Markov-chain journey, the long-run percentage of time Ana spends in each city is independent of the city where she starts.

1. Ana follows the random-walk process depicted in the video.

2. The discussion concerns the long run.

3. The video does not explicitly state irreducibility or aperiodicity.

For any starting city among the cities in the network, the long-run percentage-of-time distribution is the same.

1. Audio
   Observation

   The narrator says the percentage of time Ana spends in a city is proportional to its number of connections.

2. Audio
   Observation

   The narrator uses Bayesville with four connections and East Vandermonde with one connection as the example.

1. The video states proportionality but does not show the normalization calculation.

For each city in the network, the stationary percentage of time spent there is proportional to that city’s number of connections.

1. The process is the network random walk shown in the video.

2. “Number of connections” means the node degree in the displayed graph.

3. The probabilities are interpreted in the long run.

For every city/node i in the displayed network, $\pi_i$ is proportional to $\deg(i)$.

1. Audio
   Observation

   The narrator says Bayesville has four connections and East Vandermonde only has one.

2. Audio
   Observation

   The narrator concludes that Ana visits Bayesville four times as often as East Vandermonde in the long run.

3. Formula
   Observation

   The bar chart shows approximately 33% for Bayesville and 8% for East Vandermonde.

1. The displayed percentages are rounded, so the exact ratio is not visually precise from the chart alone.

Because Bayesville has four connections and East Vandermonde has one, Ana’s long-run visit frequency/proportion of time in Bayesville is four times that in East Vandermonde.

1. The degree-proportional stationary rule applies.

2. Bayesville has degree 4.

3. East Vandermonde has degree 1.

For the two named cities Bayesville and East Vandermonde in the displayed network.

1. Audio
   Observation

   The narrator says, “The network structure is the key to determining the stationary distribution.”

2. Animation
   Observation

   The graph structure is visually paired with the stationary-distribution bars.

1. The clip does not define “network structure” beyond the displayed connections and degrees.

The stationary distribution is determined by the structure of the connection network.

1. The network is the five-city graph shown in the video.

2. The stationary distribution is the long-run percentage distribution discussed earlier.

For the Markov-chain network example shown in the clip.

1. Audio
   Observation

   The narrator says Today Ana is in Bayesville.

2. Animation
   Observation

   A spinner with four equal sectors labeled A, C, D, E is shown.

3. Audio
   Observation

   The narrator says The spinner lands on Averagemont, so that is her next stop.

4. Audio
   Observation

   The narrator then says After she finishes her inspection, it's time to choose the third stop.

1. The exact sector geometry is visually simplified; the clip presents it as equal-probability selection among available destinations.

1. Expression

   $$$$
   Explanation

   Start the process at Bayesville.

   Justification

   The narrator explicitly states that today Ana is in Bayesville.

   Shown in the video
2. Expression

   $$$$
   Explanation

   List the available next cities from Bayesville.

   Justification

   The narration says Bayesville has direct connections to all the other cities, and the spinner shows four destination sectors A, C, D, E.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Choose the next city uniformly at random from those four options.

   Justification

   The narrator says she uses a spinner to pick randomly from all available cities with equal probabilities.

   Shown in the video
4. Expression

   $$$$
   Explanation

   The realized outcome is Averagemont, so the second stop is Averagemont.

   Justification

   The narrator states that the spinner lands on Averagemont and identifies it as her next stop.

   Shown in the video
5. Expression

   $$$$
   Explanation

   After finishing in Averagemont, the process is ready to choose a third stop.

   Justification

   The narrator says it is time to choose the third stop after the inspection.

   Shown in the video

The clip demonstrates one concrete realization of the random process: Bayesville → Averagemont, with the next transition not yet resolved inside this segment.

1. Animation
   Observation

   The clip first shows a four-sector spinner, then a five-sector spinner, then a single-sector spinner, and finally multiple spinners arranged by city.

2. Audio
   Observation

   Narrator explains that Ana's process is a Markov chain and that the next city depends only on the current city and spinner randomness.

1. Expression

   $$$$
   Explanation

   The traveler arrives in a city and uses that city's spinner to choose the next destination.

   Justification

   Observed in the airport, map, and spinner animations.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Different cities have different spinners, including a one-option spinner for East Vandermonde that forces a return to Bayesville.

   Justification

   Shown visually by the spinner layouts and stated in narration about East Vandermonde.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Because the next move is generated from the spinner attached to the current city alone, earlier route history does not enter the next-step rule.

   Justification

   This is the narrator's explicit explanation of the Markov property.

   Shown in the video
4. Expression

   $$$$
   Explanation

   Therefore the travel process is presented as a Markov chain with memoryless transitions.

   Justification

   Direct conclusion stated by the narrator.

   Shown in the video

The city-specific random-choice setup motivates the Markov property: the future step depends only on the present state.

1. Animation
   Observation

   A bar chart updates from 10 trips to 1000 trips while the trip counter increases.

2. Audio
   Observation

   Narrator says the plot fluctuates at first and then stabilizes, and names the stabilized distribution the stationary distribution.

1. Expression

   $$$$
   Explanation

   Plot one bar per city and set each bar height to the percentage of total time spent in that city so far.

   Justification

   Explicitly described by the narrator and shown on screen.

   Shown in the video
2. Expression

   $$$$
   Explanation

   As the number of trips increases, the bar heights change from run to run and from step to step.

   Justification

   Visible in the animated chart from 10 trips onward.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Near 1000 trips the fluctuations become small and the bars appear settled.

   Justification

   Narrator: "At first the plot was fluctuating a lot, but now it seems to have stabilized."

   Shown in the video
4. Expression

   $$$$
   Explanation

   The stabilized profile of bar heights is named the stationary distribution.

   Justification

   Narrator definition plus on-screen title "Stationary Distribution".

   Shown in the video

The stationary distribution is introduced as the long-run limiting shape of the empirical visit-frequency chart.

1. Audio
   Observation

   The narrator says Bayesville has four connections and East Vandermonde only has one.

2. Audio
   Observation

   The narrator says Ana visits Bayesville four times as often as East Vandermonde in the long run.

3. Formula
   Observation

   The bar chart displays 33% for Bayesville and 8% for East Vandermonde.

1. The video does not write the intermediate normalized calculation on screen.

1. Expression

   $$\pi_i \propto \deg(i)$$
   Explanation

   Use the stated rule that stationary probability is proportional to number of connections.

   Justification

   Directly stated by the narrator.

   Shown in the video
2. Expression

   $$\deg(\text{Bayesville}) = 4,\quad \deg(\text{East Vandermonde}) = 1$$
   Explanation

   Identify the relevant node degrees from the narrated example and the map.

   Justification

   The narrator explicitly gives these connection counts.

   Shown in the video
3. Expression

   $$\frac{\pi_{\text{Bayesville}}}{\pi_{\text{East Vandermonde}}} = \frac{4}{1}$$
   Explanation

   Take the ratio of the proportional stationary probabilities.

   Justification

   If two quantities are proportional to 4 and 1 respectively, their ratio is 4 to 1.

   Derived from the video
4. Expression

   $$\pi_{\text{Bayesville}} \approx 33\%,\quad \pi_{\text{East Vandermonde}} \approx 8\%$$
   Explanation

   Compare the conclusion with the displayed stationary-distribution bars.

   Justification

   The bar chart labels Bayesville with 33% and East Vandermonde with 8%.

   Shown in the video

Bayesville’s long-run share is about four times East Vandermonde’s because its number of connections is four times as large.

1. Formula
   Observation

   The bar chart displays 25%, 33%, 17%, 17%, and 8% for A, B, C, D, and E.

2. Audio
   Observation

   The narrator says the percentages are proportional to the number of connections.

3. Diagram
   Observation

   The map shows Bayesville connected to four other cities and East Vandermonde connected only to Bayesville.

1. The individual degrees for A, C, and D are not explicitly spoken in this clip.

2. The displayed percentages are rounded, so the exact degree assignment is inferred from the chart and visible graph.

1. Expression

   $$\pi_i = \frac{\deg(i)}{\sum_j \deg(j)}$$
   Explanation

   Normalize the degree-proportional weights so that the stationary probabilities sum to 1.

   Justification

   Standard normalization of a probability distribution; the video states proportionality but does not write this formula.

   Supplementary explanation
2. Expression

   $$\sum_j \deg(j) = 3+4+2+2+1 = 12$$
   Explanation

   Use the degrees consistent with the displayed graph and percentages: Averagemont 3, Bayesville 4, Continuopolis 2, Discretown 2, East Vandermonde 1.

   Justification

   The audio explicitly gives Bayesville 4 and East Vandermonde 1; the remaining degrees are inferred from the visible graph and match the displayed percentages.

   Derived from the video
3. Expression

   $$\left(\frac{3}{12},\frac{4}{12},\frac{2}{12},\frac{2}{12},\frac{1}{12}\right)=\left(25\%,33\%,17\%,17\%,8\%\right)$$
   Explanation

   Compute the normalized probabilities and compare them with the bar-chart labels.

   Justification

   $3/12=25$%, $4/12$≈33%, $2/12$≈17%, and $1/12$≈8%, matching the displayed rounded values.

   Derived from the video

The displayed stationary distribution is consistent with normalizing node degrees by their total sum.

1. Audio
   Observation

   The narrator introduces Ana Markov as a quality inspector for Normal Distributors with plants in five cities.

2. Audio
   Observation

   The narrator explains that she flies randomly through airports with direct connecting flights and stamps her logbook after each inspection.

3. Diagram
   Observation

   A map and spinner visualize the route choices.

Given a five-city network of direct flights and a rule that Ana chooses uniformly among available next cities, determine which city she will spend the most time in in the long run.

1. Five cities: Averagemont, Bayesville, Continuopolis, Discretetown, East Vandermonde.

2. Ana moves only along direct connections.

3. At each stop she chooses uniformly among available next cities.

4. Bayesville is connected to all other cities.

5. Averagemont is directly connected to Bayesville but not to East Vandermonde.

Identify the city with the greatest long-run occupancy time.

1. Expression

   $$$$
   Explanation

   Model the cities as states and the direct flights as allowed transitions.

   Justification

   The narration and map define a finite set of locations connected by routes.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Use the uniform-neighbor rule to define the transition mechanism.

   Justification

   The narrator explicitly says the next city is selected randomly from all available connecting flights with equal probabilities.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Pose the long-run question rather than a single-step question.

   Justification

   The narrator asks which city she will spend the most time in in the long run.

   Shown in the video
4. Expression

   $$$$
   Explanation

   Illustrate one realized path segment: start in Bayesville, spin among A, C, D, E, and land on Averagemont.

   Justification

   The clip shows and narrates this exact sequence.

   Shown in the video

The clip does not provide the final answer within this segment; it only poses the multiple-choice question and begins an example trace.

Verification would require analyzing the full transition graph and its long-run behavior, which is not completed in this excerpt.

1. Audio
   Observation

   Narrator: "East Vandermonde is more remote and only has flights back to Bayesville, so she will definitely go back to Bayesville next."

2. Diagram
   Observation

   A single-sector blue spinner labeled B is shown beside the traveler.

3. Animation
   Observation

   A route line connects East Vandermonde back to Bayesville on the map.

What happens when Ana is in East Vandermonde and uses the local spinner?

1. East Vandermonde corresponds to state E.

2. The spinner for East Vandermonde has only one sector, labeled B.

3. Bayesville corresponds to state B.

Identify the next city forced by the local transition rule.

1. Expression

   $$$$
   Explanation

   Read the local spinner for East Vandermonde.

   Justification

   Shown visually as a one-sector spinner.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Since there is only one possible sector, the next destination is fixed.

   Justification

   Narrator explicitly says she will definitely go back to Bayesville next.

   Shown in the video
3. Expression

   $$$$
   Explanation

   Thus the transition from E is deterministically to B.

   Justification

   Combines the map labels with the spinner label B.

   Derived from the video

The next city is Bayesville (B).

The one-sector spinner and the narrated statement both indicate no alternative destination from East Vandermonde.

1. Caption evidence
   Observation

   Speech bubble text: "What fraction of my time will I end up spending in Bayesville?"

2. Audio
   Observation

   Ana asks: "If I keep doing this, I wonder, what fraction of my time will I end up spending in Bayesville?"

Ana wonders what fraction of her time she will spend in Bayesville if she keeps traveling by the spinner rule.

1. Ana repeatedly travels using city-specific spinners.

2. Bayesville is one of the cities in the network.

Express the question that leads to the stationary-distribution discussion.

1. Expression

   $$$$
   Explanation

   Recognize that the question is not about one trip but about accumulated time over repeated travel.

   Justification

   The wording "what fraction of my time" indicates a long-run proportion.

   Shown in the video
2. Expression

   $$$$
   Explanation

   The narrator then generalizes this to the percentage of time spent in each city in the long run.

   Justification

   Narrator explicitly reformulates Ana's question as the long-run problem.

   Shown in the video

The motivating question is the long-run fraction of time spent in Bayesville, later generalized to all cities.

The later bar chart directly measures percentage of time by city, matching the question posed.

1. Diagram
   Observation

   A passport page headed "INSPECTOR: Ana Markov" contains a Bayesville stamp reading "P(A|B) = P(B|A)".

2. Diagram
   Observation

   Other stamps show city names such as Discretown, Averagemont, Continuopolis, and East Vandermonde.

1. The clip does not explain the stamp formula; it functions as a visual pun tied to the city name Bayesville.

How does the video visualize Ana's travel history before introducing the long-run frequency chart?

1. A passport page is shown with multiple dated location stamps.

2. Some stamps include city names and symbols.

Show accumulated visit history across cities.

1. Expression

   $$$$
   Explanation

   The passport lists dated entries for several cities.

   Justification

   Visible on screen with dates such as $01/27/16$, $01/29/16$, $02/02/16$, and others.

   Shown in the video
2. Expression

   $$$$
   Explanation

   Bayesville appears multiple times, reinforcing that revisits are common in the process.

   Justification

   Several stamps are labeled Bayesville.

   Shown in the video
3. Expression

   $$$$
   Explanation

   This historical record motivates counting time by city rather than tracking a single route.

   Justification

   The next scene converts the travel record into a bar chart of percentages.

   Derived from the video

The passport page serves as a visual log of repeated city visits before the empirical distribution is plotted.

The subsequent chart aggregates exactly the kind of city-visit information recorded on the page.

1. Caption evidence
   Observation

   The quiz slide asks, “Which city do you think she will spend the most time in?”

2. Caption evidence
   Observation

   The options list A) Averagemont, B) Bayesville, C) Continuopolis, D) Discretown, E) East Vandermonde, F) About equal, G) It depends.

3. Animation
   Observation

   The answer B) Bayesville remains highlighted while the other options fade.

4. Audio
   Observation

   The narrator says Bayesville has four connections and East Vandermonde only has one, then says Ana visits Bayesville four times as often as East Vandermonde in the long run.

Which city will Ana spend the most time in?

1. The stationary distribution bar chart shows $A=25$%, $B=33$%, $C=17$%, $D=17$%, $E=8$%.

2. Bayesville has four connections.

3. East Vandermonde has one connection.

4. The narrator states that time percentage is proportional to number of connections.

Identify the city with the largest long-run percentage of time.

1. Expression

   $$\pi_B = 33\%$$
   Explanation

   Read Bayesville’s stationary percentage from the bar chart.

   Justification

   The bar labeled B is marked 33%.

   Shown in the video
2. Expression

   $$33\% > 25\% > 17\% = 17\% > 8\%$$
   Explanation

   Compare the displayed stationary percentages across the five cities.

   Justification

   The bar chart labels give the numerical ordering.

   Derived from the video
3. Expression

   $$\text{Answer: B) Bayesville}$$
   Explanation

   Select the city with the largest stationary percentage.

   Justification

   Bayesville has the highest displayed percentage and the largest stated connection count.

   Shown in the video

B) Bayesville

The answer is visually confirmed when option B remains highlighted and the other options fade; it also agrees with the degree-proportional rule because Bayesville has the largest number of connections.

1. Animation
   Observation

   HarvardX branding appears, followed by a bell-curve animation and the STAT110x title.

1. HarvardX logo

2. bell curve

3. STAT110x title

1. Logo fades in.

2. A character slides down a bell-shaped curve.

3. The course title appears beneath the curve.

1. No mathematical statement is asserted during this opening beyond visual branding.

This functions as course identification rather than content-bearing mathematics.

1. Diagram
   Observation

   A map shows five city icons and lines indicating direct connections.

2. Animation
   Observation

   A spinner with labeled sectors is used to choose destinations.

1. The exact degree of every non-Bayesville node is not fully enumerated in the audio; only selected adjacencies are explicitly stated.

1. Five city icons

2. connecting lines

3. spinner sectors labeled A, C, D, E

4. arrow pointer

1. City labels appear on the map.

2. Connection lines are drawn between some pairs of cities.

3. The spinner rotates and points to one sector.

1. The map represents allowed direct transitions.

2. The spinner represents equal-probability selection among available destinations.

The visuals encode a random walk on a graph: vertices are cities, edges are direct flights, and the spinner implements uniform neighbor selection.

1. Caption evidence
   Observation

   The screen displays: Which city do you think she will spend the most time in?

2. Caption evidence
   Observation

   Options A through G are listed, including About equal and It depends.

1. Question text

2. answer options A-G

1. The question text appears.

2. Answer options are revealed sequentially.

1. The question concerns long-run time allocation, not a single trip.

This screen formalizes the conceptual target of the segment: stationary or long-run behavior of the random process.

1. Diagram
   Observation

   Bayesville is highlighted with lines extending to the other four cities.

2. Audio
   Observation

   The narrator calls Bayesville a major hub with direct connections to all the other cities.

1. Bayesville icon

2. four outgoing connection lines

3. other city icons

1. Bayesville is visually emphasized.

2. Connections from Bayesville to all other cities are shown.

1. Bayesville remains the current state in this part of the example.

2. The highlighted edges represent allowed one-step transitions.

The visual stress on Bayesville foreshadows that node degree affects long-run visit frequency.

1. Animation
   Observation

   First a three-sector spinner with labels C, D, B is shown; then a four-sector spinner with A, C, D, E; then a one-sector spinner labeled B.

2. Diagram
   Observation

   Sector colors are consistent with later city colors: A green, B blue, C magenta, D orange, E purple.

1. The opening three-sector spinner is visible, but the clip does not verbally identify its city in this excerpt.

1. Three-sector spinner labeled C, D, B

2. Four-sector spinner labeled A, C, D, E

3. One-sector spinner labeled B

4. Traveler Ana

1. The spinner layout changes from multi-sector choices to a single forced outcome.

2. The traveler is shown reacting differently as the available destinations change.

1. Letters label destination sectors.

2. Colors correspond to specific cities later shown on the map.

The animation contrasts probabilistic choices with a deterministic transition, setting up state-dependent random rules.

1. Animation
   Observation

   A map shows labeled cities including Bayesville, East Vandermonde, Discretown, and later Averagemont and Continuopolis.

2. Diagram
   Observation

   Route lines connect cities; East Vandermonde is linked back toward Bayesville.

1. Bayesville

2. East Vandermonde

3. Discretown

4. Averagemont

5. Continuopolis

6. route lines

7. river

1. The camera reveals more of the network as the plane travels.

2. Route lines emphasize possible transitions between cities.

1. City names and colors remain consistent across scenes.

The map grounds the abstract spinner states in a travel network of cities.

1. Animation
   Observation

   Five spinners are displayed simultaneously in a diamond arrangement, each associated with a different city/state.

2. Caption evidence
   Observation

   The Markov Property text appears immediately afterward.

1. The exact mapping of every outer spinner to a named city is inferred from colors and labels rather than spoken one by one.

1. Central five-sector spinner

2. Outer spinners with two-, three-, and one-sector layouts

3. Labels A, B, C, D, E

1. All spinners rotate at once, showing independent local random mechanisms.

2. The composition emphasizes parallel transition rules rather than a single route.

1. Each spinner uses the same color-code for destinations.

2. The central spinner remains the five-state reference view.

This visual compresses the idea that each current state carries its own next-step distribution.

1. Diagram
   Observation

   A passport page titled "INSPECTOR: Ana Markov" fills the frame with ten stamp boxes, several filled with city stamps and dates.

1. Not every stamp date is equally legible in motion.

1. Passport page

2. city stamps

3. date fields

4. formula stamp P(A|B)=P(B|A)

1. Filled and empty stamp boxes convey a record of past travel and room for future visits.

1. The page header remains "INSPECTOR: Ana Markov".

2. City names recur across multiple stamps.

The page visualizes accumulated path history before the video switches to aggregate frequencies.

1. Animation
   Observation

   A bar chart with bars A, B, C, D, E updates as the trip counter rises from 10 trips to 1000 trips.

2. Diagram
   Observation

   The y-axis is labeled 10%, 20%, 30%, 40%.

1. Exact intermediate percentages at each trip count are not all readable.

1. Bars A, B, C, D, E

2. trip counter

3. percentage axis

1. Bar heights fluctuate at smaller trip counts.

2. By around 1000 trips the heights appear much steadier.

3. The title changes to "Stationary Distribution" after stabilization.

1. One bar always represents one city/state.

2. Color coding stays consistent with earlier maps and spinners.

The animation demonstrates convergence of sample frequencies toward a stable distribution.

1. Animation
   Observation

   After the stationary-distribution title, the full map returns with all five city labels and connecting routes.

1. Averagemont

2. Bayesville

3. Continuopolis

4. Discretown

5. East Vandermonde

6. route lines

7. airplane icon

1. The view shifts from abstract bar chart back to the geographic network.

1. The same five-city structure and color coding persist.

The ending ties the abstract stationary distribution back to the original travel process on the network.

1. Animation
   Observation

   A stylized map shows five named cities connected by black lines.

2. Animation
   Observation

   A yellow airplane icon moves along the connection from Discretown toward Bayesville.

3. Audio
   Observation

   The narrator says, “No matter which city she starts out in.”

1. Averagemont

2. Continuopolis

3. Bayesville

4. Discretown

5. East Vandermonde

6. black connection lines

7. yellow airplane icon

8. light blue river

1. The airplane travels along a network edge from Discretown toward Bayesville.

2. The map establishes the five-city network before the stationary-distribution chart appears.

1. The five city nodes and their connecting lines remain in fixed positions during the airplane movement.

2. The city names remain visible on the map.

The animation introduces Ana’s movement as a random walk on a fixed network of cities.

1. Diagram
   Observation

   A title reads “Stationary Distribution.”

2. Formula
   Observation

   Bars labeled A through E display 25%, 33%, 17%, 17%, and 8%.

3. Animation
   Observation

   Horizontal divider lines appear inside the tallest blue bar B.

1. Title “Stationary Distribution”

2. five colored bars labeled A, B, C, D, E

3. percentage labels 25%, 33%, 17%, 17%, 8%

4. horizontal divider lines in bar B

1. The scene changes from the map to a bar chart.

2. Percentage labels appear above the bars.

3. Divider lines subdivide the tallest Bayesville bar.

1. The bar order A, B, C, D, E remains fixed.

2. The colors correspond to the city colors used elsewhere in the video.

The chart gives the long-run percentage of time in each city and visually emphasizes that Bayesville’s share is the largest.

1. Audio
   Observation

   The narrator explicitly says not all cities are connected via direct flights to each other.

2. Audio
   Observation

   The narrator gives the counterexample that Averagemont connects to Bayesville but not to East Vandermonde.

One might assume the traveler can move directly between any pair of cities.

The clip states that direct connections are limited; some pairs of cities are not adjacent in one step.

1. Audio
   Observation

   The narrator asks specifically about the long run.

2. Animation
   Observation

   A single realized transition is later shown as an example.

A viewer might infer the answer from a single observed move such as Bayesville to Averagemont.

The question posed is about long-run time spent in each city, not the outcome of one random step.

1. Audio
   Observation

   The narration defines the Markov property in terms of dependence only on the present state, not equal chances.

2. Diagram
   Observation

   Spinners shown in the clip have unequal sector sizes and even a one-sector case.

A Markov chain might be mistaken as requiring all next destinations to be equally likely.

The clip's spinners show unequal and even deterministic transition structures; the Markov property concerns dependence on the present state, not uniformity of probabilities.

1. Audio
   Observation

   The narrator shifts from Ana's next-stop question to "In the long run ... what percentage of her time will she spend in each city?"

2. Animation
   Observation

   The stationary distribution is introduced only after a bar chart over many trips.

The stationary distribution could be mistaken for the answer to where the traveler will go on the next trip.

In this clip it is defined through long-run time fractions across many trips, not through a single next-state prediction.

1. Audio
   Observation

   The narrator begins with “No matter which city she starts out in,” then states the long-run percentage result.

1. The video does not spell out the misconception in words; it is inferred from the emphasis on independence from the starting city.

One might think that the city where Ana starts determines which city she spends the most time in during the long run.

The video states that the long-run percentage of time is independent of the starting city and is instead proportional to each city’s number of connections.

1. Formula
   Observation

   The stamp explicitly reads P(A|B) ≠ P(B|A).

1. The clip does not provide a verbal explanation or example for the inequality.

One might assume that reversing the conditioning events leaves a conditional probability unchanged.

The displayed inequality states that P(A|B) is not generally equal to P(B|A).

1. Formula
   Observation

   The stationary-distribution bars have unequal heights: 25%, 33%, 17%, 17%, and 8%.

2. Caption evidence
   Observation

   The quiz includes option F) About equal, which fades away when B) Bayesville is revealed.

1. The video does not explicitly say that “about equal” is wrong; the visual answer reveal implies it.

One might expect Ana to spend about equal time in all cities in the long run.

The bar chart shows unequal stationary percentages, and the answer reveal selects Bayesville rather than “About equal.”

1. Audio
   Observation

   The overall setup describes repeated random movement among cities.

2. Diagram
   Observation

   The map names the cities that serve as possible locations.

The general Markov-chain introduction contains the specific finite state space of five cities used in the example.

1. Audio
   Observation

   The next city is chosen from available connecting flights with equal probabilities.

2. Diagram
   Observation

   Lines on the map indicate which transitions are available.

The transition rule applies to the state space by specifying how the process moves from one city-state to another.

1. Diagram
   Observation

   The map shows a network of cities and direct routes.

2. Audio
   Observation

   The narrator discusses which cities are directly connected.

Understanding uniform selection among available neighbors requires first knowing the graph connectivity that determines which neighbors are available.

1. Audio
   Observation

   Bayesville is described as having direct connections to all other cities.

2. Diagram
   Observation

   The map highlights Bayesville's links to the rest of the network.

Bayesville is a special node in the connectivity structure because it is adjacent to every other city in the example.

1. Audio
   Observation

   The long-run question is asked before the hub property is fully illustrated.

2. Audio
   Observation

   The narrator later emphasizes Bayesville's hub status.

1. The causal explanation of why hub status affects long-run time is not completed in this segment.

Answering the long-run most-visited-city question depends on understanding structural features such as Bayesville's hub degree.

1. Audio
   Observation

   Narrator first calls Ana's process a Markov chain and then explains the Markov property governing it.

The Markov property is the defining feature used to explain why Ana's travel process is a Markov chain.

1. Audio
   Observation

   Narrator says, "This is a form of memorylessness," immediately after stating the Markov property.

In this clip, memorylessness is presented as a verbal restatement of the Markov property.

1. Audio
   Observation

   The narrator answers Ana's fraction-of-time question by plotting percentages after various numbers of trips.

The empirical bar chart is the method used to investigate the long-run fraction-of-time question.

1. Animation
   Observation

   The bar chart stabilizes and is then titled "Stationary Distribution".

2. Audio
   Observation

   Narrator names the stabilized distribution as the stationary distribution.

The stationary distribution is introduced as the limiting, stabilized version of the empirical distribution plotted over trips.

1. Audio
   Observation

   Narrator says the process will always converge to the same stationary distribution.

The convergence claim applies the notion of stationary distribution to repeated long-run behavior of the chain.

1. Audio
   Observation

   The narrator defines the stationary percentage as proportional to number of connections.

2. Diagram
   Observation

   The bar heights correspond to the network degrees discussed in the narration.

The degree-proportional rule is applied to compute or explain the stationary distribution’s long-run percentages.

1. Audio
   Observation

   The narrator says the network structure is the key to determining the stationary distribution.

2. Animation
   Observation

   The abstract graph is aligned with the stationary-distribution bars.

The statement about network structure gives the broader principle behind the specific stationary-distribution values shown earlier.

1. Audio
   Observation

   The narrator describes repeated random travel among cities using direct connections.

2. Diagram
   Observation

   A five-city map with connecting lines is shown.

1. Markov chain introduced through a random travel process
2. State space of the example
3. Uniform transition rule among available neighbors

1. Audio
   Observation

   The narrator says not all cities are directly connected and gives Averagemont to East Vandermonde as a non-example.

1. Allowed transitions form a graph, not a complete graph
2. Direct-flight network is not fully connected pairwise

1. Audio
   Observation

   The narrator asks which city she will spend the most time in in the long run.

2. Caption evidence
   Observation

   The same wording appears on the multiple-choice screen.

1. Long-run most-visited-city question
2. Confusing one realized trip with long-run frequency

1. Audio
   Observation

   The narrator says Bayesville is a major hub with direct connections to all other cities.

2. Diagram
   Observation

   The map highlights Bayesville's links to the rest of the network.

1. Bayesville as a high-degree hub
2. s0-cr-hub-to-question

1. Audio
   Observation

   The narrator says the next city is selected randomly from all available connecting flights with equal probabilities.

2. Animation
   Observation

   A spinner is used to choose among destinations.

1. Uniform transition rule among available neighbors
2. Graph visualization and random-selection device

1. Caption evidence
   Observation

   On-screen text explicitly names "Markov Property".

1. Markov property
2. Memorylessness interpretation of the Markov property

1. Audio
   Observation

   Narrator identifies Ana's process as a Markov chain and then explains the dependence rule.

1. Markov chain as the model for Ana's travel process
2. Markov property

1. Caption evidence
   Observation

   Title card reads "Stationary Distribution".

2. Audio
   Observation

   Narrator defines it as the distribution the plot stabilizes to.

1. Stationary distribution

1. Audio
   Observation

   Narrator shifts from next-stop talk to "In the long run ... what percentage of her time ..."

1. Long-run occupancy question for a Markov chain
2. Empirical distribution plotted by trip count
3. Stationary distribution

1. Audio
   Observation

   Narrator says East Vandermonde only has flights back to Bayesville.

2. Diagram
   Observation

   A one-sector spinner labeled B is shown.

1. Markov chain as the model for Ana's travel process
2. Markov property

1. Audio
   Observation

   Narrator explains that each bar height indicates the percentage of time Ana has spent in that city overall.

1. Empirical distribution plotted by trip count

1. Audio
   Observation

   Narrator says one run is shown but the process will always converge to the same stationary distribution.

1. Convergence to the same stationary distribution
2. Stationary distribution