MCMC It is an algorithm for sampling from a random distribution based on a Markov chain, which approximates the posterior distribution of the parameter of interest by randomly sampling in the probability space.

The basic theory of MCMC is Markov process. In related algorithms, in order to sample on a specified distribution, we can simulate this process from any state according to the Markov process, and continuously perform state transitions until it finally converges to a stable distribution.

The overall idea is to use a stable distribution to replace the complex distribution, and use this to sample and fit to finally get the distribution of the complex sample.

Common MCMC methods: Metropolis-Hastings sampling, Gibbs sampling

Metropolis-Hastings sampling

1: Initialize the initial state of the Markov chain $latex {X\mathop{{}}\nolimits\_{{0}}\text{ }=\text{ }x\mathop{{}}\nolimits\_{{0}}}$latexX_0 = x_0

2: Sample the following process of $latex {t\text{ }=\text{ }0,\text{ }1,\text{ }2,\text{ }…}$latext = 0, 1, 2, … cycle

• At the $latex {t}$latext moment, the state of the Markov chain is$latex {X\mathop{{}}\nolimits\_{{t}}\text{ }=\text{ }x\mathop{{}}\nolimits\_{{t}}}$latexX_t = x_t , and the sampling$latex {y\text{ } \sim \text{ }q{ \left( {x \left| x\mathop{{}}\nolimits\_{{t}}\right. } \right) }}$latexy ∼ q(x​x_t)
• Sampling from a uniform distribution$latex {u\text{ } \sim \text{ }Uniform{ \left[ {0,1} \right] }}$latexu ∼ Uniform[0,1]
• If latex {u\text{ } < \text{ } \alpha { \left( {x\mathop{{}}\nolimits\_{{t}},y} \right) }\text{ }=\text{ }min{ \left\{ {\frac{{p{ \left( {y} \right) }q{ \left( {x\mathop{{}}\nolimits\_{{t}} \left| y\right. } \right) }}}{{p{ \left( {x\mathop{{}}\nolimits\_{{t}}} \right) }p{ \left( {y \left| {x\mathop{{}}\nolimits\_{{t}}\text{ } \to \text{ }y}, that is, $latex {X\mathop{{}}\nolimits\_{{t+1}}\text{ }=\text{ }y}$latexX_t+1 = y
• Otherwise, the transfer is not accepted, that is, $latex {X\mathop{{}}\nolimits\_{{t+1}}\text{ }=\text{ }x\mathop{{}}\nolimits\_{{t}}}$latexX_t+1 = x_t

Gibbs Sampling

1: Randomly initialize $latex {X\mathop{{}}\nolimits\_{{0}}\text{ }=\text{ }x\mathop{{}}\nolimits\_{{0}},\text{ }Y\mathop{{}}\nolimits\_{{0}}\text{ }=\text{ }y\mathop{{}}\nolimits\_{{0}}}$latexX_0 = x_0, Y_0 = y_0

2: Cyclic sampling of $latex {t\text{ }=\text{ }0,\text{ }1,\text{ }2,\text{ }…}$latext = 0, 1, 2, …

• $latex {y\mathop{{}}\nolimits\_{{t+1}}\text{ } \sim \text{ }p{ \left( {y \left| x\mathop{{}}\nolimits\_{{t}}\right. } \right) }}$latexy_t+1 ∼ p(y​x_t)
• $latex {x\mathop{{}}\nolimits\_{{t+1}}\text{ } \sim \text{ }p{ \left( {x \left| y\mathop{{}}\nolimits\_{{t+1}}\right. } \right) }}$latexx_t+1 ∼ p(x​y_t+1)

References

【1】Getting Started with MCMC

【2】A brief analysis of Markov chain Monte Carlo method