What is the key insight of dependent sampling in MCMC?

Conditions

• Each new sample is drawn from a probability distribution conditioned on the previous sample.
• The formula is $X_{n+1} \sim P(\cdot|X_n)$.
• Applies to complex probability distributions with multiple peaks and weird shapes.

Reasoning, step by step

1. Contrast independent sampling (isolated dots) with dependent sampling (connected chain).
2. Identify the mathematical formulation: $X_{n+1} \sim P(\cdot|X_n)$.
3. Explain that the next sample depends on the current sample.
4. Conclude that this dependence allows for efficient exploration of the target space.

Example

The video shows the formula $X_{n+1} \sim P(\cdot|X_n)$ and contrasts isolated cyan dots (independent) with green dots connected by yellow arrows (dependent).

Common misconceptions

• Thinking that dependent sampling throws away all information from previous attempts.
• Assuming that each sample is completely random and independent of the others.

Watch the explanation

Explore next

• What is a Markov chain and what is its memoryless property?
• Why is Accept-Reject Sampling inefficient for complex distributions?
• How does the Monte Carlo method estimate the expected value of a distribution?