Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC



Case Study Comparing Bayesian and Frequentist Approaches for Multiple Treatment Comparisons

Research Report Mar 28, 2013
Download PDF files for this report here.

Page Contents

People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.

Structured Abstract


Bayesian statistical methods are increasingly popular as a tool for meta-analysis of clinical trial data involving both direct and indirect treatment comparisons. However, appropriate selection of prior distributions for unknown model parameters and checking of consistency assumptions required for feasible modeling remain particularly challenging. We compared Bayesian and traditional frequentist statistical methods for multiple treatment comparisons in the context of pharmacological treatments for female urinary incontinence (UI).


We searched major electronic bibliographic databases, U.S. Food and Drug Administration reviews, trial registries, and research grant databases up to November 2011 to find randomized studies published in English that examined drugs for urgency UI on continence, improvements in UI, and treatment discontinuation due to harms.

We fitted fixed and random effects models in frequentist and Bayesian frameworks. In a hierarchical model of eight treatments, we separately analyzed one safety and two efficacy outcomes. We produced Bayesian and frequentist treatment ranks and odds ratios (and associated measures of uncertainty) across all bivariate treatment comparisons. We also calculated the number needed to treat (NNT) to achieve continence or avoid harms from pooled absolute risk differences.


While frequentist and Bayesian analyses produced broadly comparable odds ratios of safety and efficacy, the Bayesian method's ability to deliver the probability that any treatment is best, or among the top two such treatments, offered a more meaningful clinical interpretation. In our study, two drugs emerged as attractive because while neither had any significant chance of being among the least safe drugs, both had greater than 50 percent chances of being among the top three drugs in terms of Best12 probability for one of the efficacy endpoints.


Bayesian methods are more flexible and their results more clinically interpretable but require more careful development and specialized software.

Journal Publications

Hong H, Carlin BP, Shamliyan TA, et al. Comparing Bayesian and frequentist approaches for multiple outcome mixed treatment comparisons. Med Decis Making. 2013 Apr 2. DOI: 10.1177/0272989X13481110.

Project Timeline

Bayesian Approaches to Indirect Comparisons

Sep 22, 2011
Topic Initiated
Sep 22, 2011
Jan 10, 2013
Mar 28, 2013
Research Report
Page last reviewed September 2019
Page originally created November 2017

Internet Citation: Research Report: Case Study Comparing Bayesian and Frequentist Approaches for Multiple Treatment Comparisons. Content last reviewed September 2019. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.

Select to copy citation