Markov chain Monte Carlo (MCMC) is a family of algorithms for approximating expectations under a target probability distribution. An algorithm constructs a Markov chain whose transition rule preserves that target. Under additional conditions, empirical averages from an adequately exploring chain can approximate target expectations.

“Monte Carlo” refers to numerical estimation using random draws. “Markov chain” means the next-state distribution depends on the current state rather than the complete path. Consecutive MCMC draws are generally correlated, so 10,000 stored draws do not carry the information of 10,000 independent draws.

Why MCMC is useful—and fallible

In Bayesian inference, the posterior is proportional to the likelihood times the prior. MCMC is useful when this density can be evaluated up to a normalizing constant but direct sampling or integration is difficult. It is not automatically accurate: the model can be wrong, the posterior can be hard to explore, and finite runs can be biased or noisy.

Metropolis-Hastings

If the chain is at x, draw x′ from proposal density q(x′ | x). Accept it with probability

min(1, [π(x′) q(x | x′)] / [π(x) q(x′ | x)]),

where π is proportional to the target density. If rejected, the next state repeats x. “Always accept uphill, sometimes accept downhill” describes the symmetric-proposal Metropolis case, not the full algorithm. A valid proposal must permit adequate exploration; validity alone does not ensure useful mixing.

Gibbs, HMC, and NUTS

Gibbs sampling updates a variable or block from its full conditional distribution given all remaining variables. These conditionals may be univariate or multivariate and must be available and sampleable. Exact conditional updates are accepted, but strong dependence and multimodality can still produce slow exploration.

Hamiltonian Monte Carlo (HMC) uses gradients and simulated Hamiltonian dynamics to propose distant states with high acceptance. It requires differentiable target geometry and careful computation. The No-U-Turn Sampler (NUTS) adaptively chooses trajectory length; it is an HMC algorithm, not simply a generic Metropolis-Hastings variant.

Warmup is not a magic burn-in number

Modern HMC implementations use early iterations to adapt step size and a mass matrix. These warmup draws are usually excluded from posterior summaries because the transition process is adapting. There is no universal rule such as discarding 1,000 draws. Initialization, adaptation, posterior geometry, and the needed precision determine an adequate run.

Diagnostics to report

  • Independent chains: initialize multiple chains so failures to explore are easier to detect.
  • Rank-normalized R-hat: values near 1 support, but do not prove, consistent mixing. Investigate values above 1.01.
  • Bulk and tail ESS: estimate information for central and tail quantities after autocorrelation.
  • Monte Carlo standard error: compare simulation error with the precision required for each reported estimand.
  • Sampler diagnostics: inspect divergent transitions, maximum tree depth, energy behavior, and acceptance statistics for HMC/NUTS.
  • Visual checks: inspect rank plots and chain traces, while recognizing that plots alone cannot prove convergence.

More draws reduce Monte Carlo error only after chains explore the target adequately. They do not repair non-identifiability, separated modes, divergent trajectories, invalid code, or model misspecification.

Posterior and predictive checks

Simulate from the prior predictive distribution to detect implausible priors before fitting. After fitting, compare posterior predictive simulations with relevant features of observed data. These checks reveal model-data mismatch; they do not certify that the model is true.

Applications and boundaries

  • Hierarchical and latent-variable Bayesian models, including some Bayesian neural networks.
  • Topic models, where collapsed Gibbs is one option alongside variational and other inference methods.
  • Finance, where MCMC may infer uncertain parameters or states; forward Monte Carlo may then simulate losses. Not every scenario simulation is MCMC.
  • Epidemiology and phylogenetics, where results remain conditional on data, priors, likelihoods, model structure, and computational adequacy.

MCMC quantifies uncertainty conditional on a model; it does not make a black-box model transparent or guarantee calibrated decisions. Distinguish sampling error from modeling error. The bias-variance trade-off is a separate supervised-learning concept, and LLM evaluation metrics are not MCMC diagnostics.

Originally published August 14, 2025; technically reviewed and substantially updated September 4, 2026.