Links: Notes about MCMC on bounded domains

A note to self:

Recently I had an exercise task that required sampling a posterior distribution $p ( \theta \mid x) \propto p(x \mid \theta) p(\theta)$ where parameters were restricted $\theta > 0$.

How to do this?

See this SE question and answers.

1. Independence sampler on the bounded region.

2. Transform the variables to unbounded space ie sample from $\log(\theta)$. But if I’m not mistaken, this requires changes to target posterior distribution, and maybe also proposal? I should investigate.

3. Work out a proposal that is not independence sampler but still draws proposals within the bounds.

4. Simply reject the proposals outside the bounds. May not work well.

5. Extend the target outside the bounds, ignore all samples that are in the extended area. This sounds weird.

All of the above in the context of a regular MCMC. What about HMC?

Gelman et al. 2014 [1] discuss this on page 303. Instructions:

1. Ignore the value $\theta$ that stepped outside the restricted set, continue iteration from the previous legal value of $\theta$.

2. ‘Bouncing’: if the target density is not positive at some point of iteration, change the sign of the momentum.

3. Transformation. Again, one must carefully work out the correct posterior. (No mention about proposals.)

4. I guess other methods mentioned above might work as well.

References

[1] Gelman, Carlin, Stern, Dunson, Vehtari, Rubin. Bayesian Data Analysis, 3rd edition. 2014. CRC Press (Boca Raton, FL).

Internet references

stats.stackexchange.com/questions/73885/mcmc-on-a-bounded-parameter-space

xianblog.wordpress.com/tag/constrained-parameter-spaces/

Other notes about MCMC

Component-wise vs block-wise sampling

Convergence diagnostics.