A Markov chain is a mathematical system that undergoes transitions from one state to another according to certain probabilistic rules. Its defining characteristic is the memoryless property: the probability of transitioning to any particular state depends solely on the current state and time elapsed, not on the sequence of events that preceded it.
Conditions: The future state depends only on the present state.; Transition probabilities are defined between states.; The system is a Markov chain.
A Markov chain is a mathematical system that undergoes transitions from one state to another according to certain probabilistic rules. Its defining characteristic is the memoryless property: the probability of transitioning to any particular state depends solely on the current state and time elapsed, not on the sequence of events that preceded it.
Conditions: The future state depends only on the present state.; Transition probabilities are defined between states.; The system is a Markov chain.
The burn-in period consists of the initial samples generated by an MCMC algorithm before it has converged to its stationary distribution. These samples must be discarded because they are influenced by the arbitrary starting point and do not accurately represent the target distribution.
Conditions: Applies to MCMC sampling.; Requires discarding initial samples before using the rest for inference.; The chain starts from an arbitrary point.
The burn-in period consists of the initial samples generated by an MCMC algorithm before it has converged to its stationary distribution. These samples must be discarded because they are influenced by the arbitrary starting point and do not accurately represent the target distribution.
Conditions: Applies to MCMC sampling.; Requires discarding initial samples before using the rest for inference.; The chain starts from an arbitrary point.
Pure random sampling is inefficient in high dimensions because the volume of the space grows exponentially, making it nearly impossible to randomly hit regions of high probability. MCMC solves this by using dependent sampling via a Markov chain, which guides exploration and naturally concentrates samples in regions of high probability.
Conditions: The target distribution is defined over a high-dimensional space.; As dimensionality increases, random sampling becomes inefficient.; MCMC uses a Markov chain designed so that its stationary distribution is the target distribution.
Pure random sampling is inefficient in high dimensions because the volume of the space grows exponentially, making it nearly impossible to randomly hit regions of high probability. MCMC solves this by using dependent sampling via a Markov chain, which guides exploration and naturally concentrates samples in regions of high probability.
Conditions: The target distribution is defined over a high-dimensional space.; As dimensionality increases, random sampling becomes inefficient.; MCMC uses a Markov chain designed so that its stationary distribution is the target distribution.
The key insight is that instead of treating each sample independently, dependent sampling uses the information from the previous sample to determine the next one. This creates a chain of information that flows from sample to sample, allowing the sampler to intelligently explore high-probability regions.
Conditions: Each new sample is drawn from a probability distribution conditioned on the previous sample.; The formula is Xn+1∼P(⋅∣Xn).; Applies to complex probability distributions with multiple peaks and weird shapes.
The key insight is that instead of treating each sample independently, dependent sampling uses the information from the previous sample to determine the next one. This creates a chain of information that flows from sample to sample, allowing the sampler to intelligently explore high-probability regions.
Conditions: Each new sample is drawn from a probability distribution conditioned on the previous sample.; The formula is Xn+1∼P(⋅∣Xn).; Applies to complex probability distributions with multiple peaks and weird shapes.
The Monte Carlo method estimates the expected value by replacing difficult analytical integrals with the average of random samples. It draws independent random samples from the target distribution, computes their running average, and observes the average converging to the true expected value as the number of samples increases.
Conditions: Requires the ability to draw random samples from the target distribution.; Accuracy improves as the number of samples increases.; p(x) must be a valid probability density function.
The Monte Carlo method estimates the expected value by replacing difficult analytical integrals with the average of random samples. It draws independent random samples from the target distribution, computes their running average, and observes the average converging to the true expected value as the number of samples increases.
Conditions: Requires the ability to draw random samples from the target distribution.; Accuracy improves as the number of samples increases.; p(x) must be a valid probability density function.