metaBMA: Bayesian model-averaging for meta-analysis

Fixed-effects meta-analyses assume that the effect size d is identical in all studies. In contrast, random-effects meta-analyses assume that effects vary according to a normal distribution with mean d and standard deviation tau. Both models can be compared in a Bayesian framework by assuming specific prior distribution for d and tau. Given the posterior model probabilities, the evidence for or against an effect (i.e., whether d = 0) and the evidence for or against random effects can be evaluated (i.e., whether tau = 0). By using Bayesian model averaging (i.e., inclusion Bayes factors), both types of tests can be performed by marginalizing over the other question. Most importantly, this allows to test whether an effect exists while accounting for uncertainty whether study heterogeneity exists or not.

The Package metaBMA The R package

metaBMA is available on GitHub at https://github.com/danheck/metaBMA The most general functions in metaBMA are meta_bma and meta_default, which fit random- and fixed-effects models, compute the inclusion Bayes factor for the presence of an effect and the averaged posterior distribution of the mean effect d (which accounts for uncertainty regarding study heterogeneity). Moreover, meta_fixed and meta_random fit a single meta-analysis models. The model-specific posteriors for d can be averaged by bma and inclusion Bayes factors be computed by inclusion. Finally, the function prior facilitates the construction and visual inspection of prior distributions.

Installing metaBMA

metaBMA requires the software Stan. To install the latest stable release of metaBMA from CRAN, run:

install.packages("metaBMA")

Instructions on how to install the latest developer version can be found on the GitHub repository.

Citation

If you use metaBMA, please cite the software as follows:

  • Heck, D. W., Gronau, Q. F., & Wagenmakers, E. (2019). metaBMA: Bayesian model averaging for random- and fixed-effects meta-analysis. https://doi.org/10.32614/CRAN.package.metaBMA
    [BibTeX] [Preprint] 
    @book{heck2019metabma,
title = {{{metaBMA}}: {{Bayesian}} Model Averaging for Random- and Fixed-Effects Meta-Analysis},
author = {Heck, Daniel W and Gronau, Quentin F and Wagenmakers, Eric-Jan},
date = {2019},
doi = {10.32614/CRAN.package.metaBMA},
url = {https://CRAN.R-project.org/package=metaBMA},
keywords = {Bayesian meta-analysis}
}

The package was used in:

  • [PDF] Gronau, Q. F., Heck, D. W., Berkhout, S. W., Haaf, J. M., & Wagenmakers, E. (2021). A primer on Bayesian model-averaged meta-analysis. Advances in Methods and Practices in Psychological Science, 4, 1–19. https://doi.org/10.1177/25152459211031256
    [Abstract] [BibTeX] [Preprint] [Data & R Scripts]

    Meta-analysis is the predominant approach for quantitatively synthesizing a set of studies. If the studies themselves are of high quality, meta-analysis can provide valuable insights into the current scientific state of knowledge about a particular phenomenon. In psychological science, the most common approach is to conduct frequentist meta-analysis. In this primer, we discuss an alternative method, Bayesian model-averaged meta-analysis. This procedure combines the results of four Bayesian meta-analysis models: (1) fixed-effect null hypothesis, (2) fixed-effect alternative hypothesis, (3) random-effects null hypothesis, and (4) random-effects alternative hypothesis. These models are combined according to their plausibilities in light of the observed data to address the two key questions “Is the overall effect non-zero?” and “Is there between-study variability in effect size?”. Bayesian model-averaged meta-analysis therefore avoids the need to select either a fixed-effect or random-effects model and instead takes into account model uncertainty in a principled manner.

    @article{gronau2021primer,
title = {A Primer on {{Bayesian}} Model-Averaged Meta-Analysis},
author = {Gronau, Quentin F. and Heck, Daniel W and Berkhout, Sophie W. and Haaf, Julia M. and Wagenmakers, Eric-Jan},
date = {2021},
journaltitle = {Advances in Methods and Practices in Psychological Science},
volume = {4},
pages = {1--19},
doi = {10.1177/25152459211031256},
url = {https://psyarxiv.com/97qup},
abstract = {Meta-analysis is the predominant approach for quantitatively synthesizing a set of studies. If the studies themselves are of high quality, meta-analysis can provide valuable insights into the current scientific state of knowledge about a particular phenomenon. In psychological science, the most common approach is to conduct frequentist meta-analysis. In this primer, we discuss an alternative method, Bayesian model-averaged meta-analysis. This procedure combines the results of four Bayesian meta-analysis models: (1) fixed-effect null hypothesis, (2) fixed-effect alternative hypothesis, (3) random-effects null hypothesis, and (4) random-effects alternative hypothesis. These models are combined according to their plausibilities in light of the observed data to address the two key questions "Is the overall effect non-zero?" and "Is there between-study variability in effect size?". Bayesian model-averaged meta-analysis therefore avoids the need to select either a fixed-effect or random-effects model and instead takes into account model uncertainty in a principled manner.},
osf = {https://osf.io/npw5c},
keywords = {Bayesian meta-analysis}
}

  • [PDF] Gronau, Q. F., Van Erp, S., Heck, D. W., Cesario, J., Jonas, K. J., & Wagenmakers, E. (2017). A Bayesian model-averaged meta-analysis of the power pose effect with informed and default priors: the case of felt power. Comprehensive Results in Social Psychology, 2, 123–138. https://doi.org/10.1080/23743603.2017.1326760
    [Abstract] [BibTeX] [Data & R Scripts]

    Earlier work found that – compared to participants who adopted constrictive body postures – participants who adopted expansive body postures reported feeling more powerful, showed an increase in testosterone and a decrease in cortisol, and displayed an increased tolerance for risk. However, these power pose effects have recently come under considerable scrutiny. Here, we present a Bayesian meta-analysis of six preregistered studies from this special issue, focusing on the effect of power posing on felt power. Our analysis improves on standard classical meta-analyses in several ways. First and foremost, we considered only preregistered studies, eliminating concerns about publication bias. Second, the Bayesian approach enables us to quantify evidence for both the alternative and the null hypothesis. Third, we use Bayesian model-averaging to account for the uncertainty with respect to the choice for a fixed-effect model or a random-effect model. Fourth, based on a literature review, we obtained an empirically informed prior distribution for the between-study heterogeneity of effect sizes. This empirically informed prior can serve as a default choice not only for the investigation of the power pose effect but for effects in the field of psychology more generally. For effect size, we considered a default and an informed prior. Our meta-analysis yields very strong evidence for an effect of power posing on felt power. However, when the analysis is restricted to participants unfamiliar with the effect, the meta-analysis yields evidence that is only moderate.

    @article{gronau2017bayesian,
title = {A {{Bayesian}} Model-Averaged Meta-Analysis of the Power Pose Effect with Informed and Default Priors: The Case of Felt Power},
author = {Gronau, Quentin F. and Van Erp, Sara and Heck, Daniel W and Cesario, Joseph and Jonas, Kai J. and Wagenmakers, Eric-Jan},
date = {2017},
journaltitle = {Comprehensive Results in Social Psychology},
volume = {2},
pages = {123--138},
doi = {10.1080/23743603.2017.1326760},
abstract = {Earlier work found that – compared to participants who adopted constrictive body postures – participants who adopted expansive body postures reported feeling more powerful, showed an increase in testosterone and a decrease in cortisol, and displayed an increased tolerance for risk. However, these power pose effects have recently come under considerable scrutiny. Here, we present a Bayesian meta-analysis of six preregistered studies from this special issue, focusing on the effect of power posing on felt power. Our analysis improves on standard classical meta-analyses in several ways. First and foremost, we considered only preregistered studies, eliminating concerns about publication bias. Second, the Bayesian approach enables us to quantify evidence for both the alternative and the null hypothesis. Third, we use Bayesian model-averaging to account for the uncertainty with respect to the choice for a fixed-effect model or a random-effect model. Fourth, based on a literature review, we obtained an empirically informed prior distribution for the between-study heterogeneity of effect sizes. This empirically informed prior can serve as a default choice not only for the investigation of the power pose effect but for effects in the field of psychology more generally. For effect size, we considered a default and an informed prior. Our meta-analysis yields very strong evidence for an effect of power posing on felt power. However, when the analysis is restricted to participants unfamiliar with the effect, the meta-analysis yields evidence that is only moderate.},
osf = {https://osf.io/k5avt},
keywords = {Bayesian meta-analysis}
}