MCMCprecision: Precision of discrete MCMC

The R package MCMCprecision estimates the precision of the posterior model probabilities in transdimensional Markov chain Monte Carlo methods (e.g., reversible jump MCMC or product-space MCMC). This is useful for applications of transdimensional MCMC such as model selection, mixtures with varying numbers of components, change-point detection, capture-recapture models, phylogenetic trees, variable selection, and for discrete parameters in MCMC output in general.

Installing MCMCprecision

MCMCprecision can be directly installed from CRAN:

install.packages("MCMCprecision")

Alternatively, the package can be installed from the GitHub repository (https://github.com/danheck/MCMCprecision) by pasting the following code to R (dependencies need to be installed manually):

### Dependencies:
# install.packages(c("combinat", "devtools","RcppProgress","RcppArmadillo", "RcppEigen"))
library(devtools)
install_github("danheck/MCMCprecision")

To compile C++ code, Windows requires Rtools and Mac Xcode Command Line Tools, respectively. Moreover, on Mac, it might be necessary to install the library gfortran manually by typing the following into the console (required to compile the package RcppArmadillo):

curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C /

Reference

• Heck, D. W., Overstall, A., Gronau, Q. F., & Wagenmakers, E. (2019). Quantifying uncertainty in transdimensional Markov chain Monte Carlo using discrete Markov models. Statistics & Computing, 29, 631–643. https://doi.org/10.1007/s11222-018-9828-0
  [Abstract] [BibTeX] [Preprint] [Data & R Scripts] [GitHub]
  

  Bayesian analysis often concerns an evaluation of models with different dimensionality as is necessary in, for example, model selection or mixture models. To facilitate this evaluation, transdimensional Markov chain Monte Carlo (MCMC) relies on sampling a discrete indexing variable to estimate the posterior model probabilities. However, little attention has been paid to the precision of these estimates. If only few switches occur between the models in the transdimensional MCMC output, precision may be low and assessment based on the assumption of independent samples misleading. Here, we propose a new method to estimate the precision based on the observed transition matrix of the model-indexing variable. Assuming a first order Markov model, the method samples from the posterior of the stationary distribution. This allows assessment of the uncertainty in the estimated posterior model probabilities, model ranks, and Bayes factors. Moreover, the method provides an estimate for the effective sample size of the MCMC output. In two model-selection examples, we show that the proposed approach provides a good assessment of the uncertainty associated with the estimated posterior model probabilities.

  @article{heck2019quantifying,
  title = {Quantifying Uncertainty in Transdimensional {{Markov}} Chain {{Monte Carlo}} Using Discrete {{Markov}} Models},
  author = {Heck, Daniel W and Overstall, Antony and Gronau, Quentin F and Wagenmakers, Eric-Jan},
  date = {2019},
  journaltitle = {Statistics \& Computing},
  volume = {29},
  pages = {631--643},
  doi = {10.1007/s11222-018-9828-0},
  abstract = {Bayesian analysis often concerns an evaluation of models with different dimensionality as is necessary in, for example, model selection or mixture models. To facilitate this evaluation, transdimensional Markov chain Monte Carlo (MCMC) relies on sampling a discrete indexing variable to estimate the posterior model probabilities. However, little attention has been paid to the precision of these estimates. If only few switches occur between the models in the transdimensional MCMC output, precision may be low and assessment based on the assumption of independent samples misleading. Here, we propose a new method to estimate the precision based on the observed transition matrix of the model-indexing variable. Assuming a first order Markov model, the method samples from the posterior of the stationary distribution. This allows assessment of the uncertainty in the estimated posterior model probabilities, model ranks, and Bayes factors. Moreover, the method provides an estimate for the effective sample size of the MCMC output. In two model-selection examples, we show that the proposed approach provides a good assessment of the uncertainty associated with the estimated posterior model probabilities.},
  arxivnumber = {1703.10364},
  github = {https://github.com/danheck/MCMCprecision},
  osf = {https://osf.io/kjrkz},
  keywords = {heckfirst,Polytope\_Sampling}
  }