Bayes theorem, the geometry of changing beliefs

Use the learning inspector for key ideas and moments, or open the reading tabs for the complete notes.

Chapters

Learning script

Generated from the video's visuals and explanation; not verbatim speech.

0:00Bayes' theorem is central to scientific discovery, machine learning, and even treasure hunting. Understanding it requires grasping three levels: what the symbols mean, why the formula works, and when to apply it. We begin with a famous cognitive bias experiment involving a character named Steve.

1:12Steve’s description may suggest a librarian. The illustrative setup considers only librarians and farmers, with twenty farmers per librarian. This is the example’s background assumption, not a current occupational statistic. Matching a description is not the same as a posterior probability.

3:15To resolve this, imagine a population of 210 people: 10 librarians and 200 farmers. Suppose 40% of librarians fit Steve's description, while only 10% of farmers do. This yields 4 matching librarians and 20 matching farmers. Out of the 24 total people who fit the description, only 4 are librarians. Thus, the probability Steve is a librarian given the evidence is approximately 16.7%, far lower than intuition suggests.

4:51We can formalize this calculation. Let H be the hypothesis 'Steve is a librarian' and E be the evidence 'fits the description'. The term $P(H) = 1/21$ is the prior probability. The term P(E|H) = 0.4 is the likelihood—how probable the evidence is if the hypothesis is true. Similarly, P(¬H)P(E|¬H) accounts for the alternative case where the hypothesis is false. The prior refers to this two-category example population, and conditioning requires $P(E)>0$.

6:20Combining these terms gives the full expansion of Bayes' theorem: P(H|E) equals the product of the prior and likelihood, divided by the sum of such products across all mutually exclusive hypotheses. The denominator represents the total probability of observing the evidence regardless of which hypothesis is true. This result, P(H|E), is called the posterior—the updated belief after incorporating new data.

10:00We begin by looking at the geometry behind Bayes' theorem. Consider the term $P(H)P(E|H)$ in the numerator. Geometrically, this represents the intersection of two events: the hypothesis $H$ being true and the evidence $E$ occurring. If we view the total sample space as a large square, the prior probability $P(H)$ defines the width of a vertical strip, and the likelihood $P(E|H)$ defines the height of a shaded region within that strip. Multiplying these dimensions gives us the exact area of the top-left rectangle. This area corresponds precisely to the joint probability—the chance that both the hypothesis and the evidence happen together. Visualizing probability as physical area helps ground abstract formulas in spatial reality.

10:24To understand human intuition better, let's step back from pure algebra. A powerful technique involves imagining a representative sample—a specific group of people—rather than dealing with abstract ratios. For instance, instead of thinking about percentages, imagine a population of 210 individuals divided into librarians and farmers. This approach aligns with findings from psychologists Daniel Kahneman and Amos Tversky. They discovered that framing statistical problems using concrete numbers of people significantly improves accuracy compared to using isolated probabilities. When we anchor our reasoning to a fixed sample size, complex conditional relationships become tangible counts, reducing cognitive load and minimizing common errors in probabilistic judgment.

10:39The Linda problem compares being a bank teller with being both a bank teller and active in the feminist movement. The second event is a subset of the first, so $P(A\cap B)\le P(A)$. A compelling narrative can distract from inclusion; strict inequality is not required in every case.

11:25Rephrasing the question as counts out of a hundred makes the subset relationship easier to see. The video reports a reduction in errors in the described experiment; it does not establish a zero-error guarantee for every person or problem. A subset’s count is at most the parent set’s count.

12:09While counting people works well for discrete events, real-world phenomena often involve continuous variables, such as weather patterns or measurement tolerances. In these cases, representing probabilities as grids of dots becomes cumbersome. Here, geometry offers a superior alternative. Imagine a bar chart where the width represents time or intensity, and the colored portion indicates the likelihood of rain. As conditions change continuously, the boundary between sunny and rainy outcomes shifts smoothly across the canvas. Unlike static icons, geometric areas naturally accommodate infinitesimal changes. Furthermore, sketching these regions on paper provides an immediate visual feedback loop, making it easier to manipulate equations and spot inconsistencies during manual calculation without relying solely on symbolic manipulation.

12:44Returning to Bayes' theorem itself, viewing it through the lens of proportions renders its meaning almost self-evident. The formula states that the posterior probability $P(H|E)$ equals the joint probability $P(H \cap E)$ divided by the marginal probability $P(E)$. Translated into plain language: First, restrict your attention only to those cases where the evidence $E$ has occurred. Then, among that filtered subgroup, calculate what fraction also satisfies the hypothesis $H$. Whether you measure this fraction by counting heads in a crowd or measuring pixels on a screen, the underlying arithmetic remains identical. Recognizing this structural simplicity transforms a daunting equation into a straightforward instruction for filtering data based on observed outcomes. Conditioning requires a positive evidence-region probability.

13:22Critics sometimes argue that applying base rates to individual profiles, like Steve the librarian/farmer puzzle, ignores contextual ambiguity. Who exactly is Steve? Is he drawn randomly from national census data, or is he someone personally known to the observer? These questions highlight that priors are subjective starting points influenced by background knowledge. Changing assumptions about the source population alters the baseline widths in our diagrams. Similarly, tweaking stereotypes affects the relative heights of likelihood rectangles. Yet regardless of where one places the dividing lines initially, the mechanism of updating stays constant. Evidence does not dictate absolute truth instantly; instead, it nudges existing beliefs along a spectrum. Mastering this dynamic interplay between flexible priors and hard data constitutes the essence of rational inference.

Knowledge cards