How does the Monte Carlo method estimate the expected value of a distribution?

Conditions

• Requires the ability to draw random samples from the target distribution.
• Accuracy improves as the number of samples increases.
• $p(x)$ must be a valid probability density function.

Reasoning, step by step

1. Identify the target quantity to estimate, such as the expected value $E[X]$.
2. Draw independent random samples from the target distribution.
3. Compute the running average of the collected samples.
4. Observe the running average converging to the true expected value.

Example

The video shows green dots accumulating randomly along the x-axis of a bimodal distribution, with a running average box updating until it stabilizes near the true value line (-0.247).

Common misconceptions

• Assuming that a single random sample can accurately estimate the expected value.
• Believing that the Monte Carlo method requires solving the analytical integral $\int_{-\infty}^{\infty} x \cdot p(x) dx$ directly.

Watch the explanation

Explore next

• Why is pure random sampling inefficient in high dimensions, and how does MCMC solve this?
• What is the key insight of dependent sampling in MCMC?
• What is a Markov chain and what is its memoryless property?