Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC

SHARE:

FacebookTwitterFacebookPrintShare

Methods Guide for Effectiveness and Comparative Effectiveness Reviews

COLLECTION February 23, 2018

Effectiveness and Comparative Effectiveness Reviews are systematic reviews of existing research examining the effectiveness, comparative effectiveness, and comparative harms of different healthcare interventions. They are intended to provide relevant evidence to inform real-world healthcare decisions for patients, providers, healthcare systems, and policymakers, among others. In an effort to improve the transparency, consistency, and scientific rigor of the work of the Effective Health Care (EHC) Program, through a collaborative effort, the Agency for Healthcare Research and Quality (AHRQ), the Scientific Resource Center, and the Evidence-based Practice Centers (EPCs) have developed a Methods Guide for Comparative Effectiveness Reviews. In addition to adherence to the methods described in the Methods Guide, EPC reports are also conducted in accordance with an established policy on financial and nonfinancial interests [PDF, 405 KB]. We intend that these documents will serve as a resource for our EPCs as well as for other investigators interested in conducting Comparative Effectiveness Reviews.

As further empiric evidence develops and our understanding of methods improves, updates are made and additional chapters are added, such that this Methods Guide is a living document. Please contact us with your comments and suggestions on the Methods Guide for Comparative Effectiveness Reviews and the Effective Health Care Program.

Given the interest in comparative effectiveness research and a desire to expand capacity to conduct Comparative Effectiveness Reviews, AHRQ has produced training modules to familiarize new investigators with the methodological framework and guidance outlined in AHRQ's Methods Guide for Effectiveness and Comparative Effectiveness Reviews. The AHRQ Training Modules for the Systematic Reviews Methods Guide is a series of 14 slide presentations that can be used by faculty engaged in educating investigators and clinicians interested in systematic review methods.

Systematic reviews on medical tests have unique challenges. To complement the EPC Methods Guide for Comparative Effectiveness Reviews, the EPC Program has developed a Methods Guide for Medical Test Reviews. This series of guidance papers applies the same principles for assessing treatments to the issues and challenges in assessing medical tests and highlights particular areas where the inherently different qualities of medical tests necessitate a different approach to systematic review.

Foreword: Comparing Medical Interventions: AHRQ and the Effective Health Care Program

Published 18 Nov 2008. Select for HTML or PDF (329 kb).
Other publication(s): Slutsky J, Atkins D, Chang S, et al. AHRQ Series Paper 1: Comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010; 63: 481-483.

Principles in Developing and Applying Guidance for Comparing Medical Interventions

Published 08 Aug 2009. Select for HTML or PDF (113 kb).
Other publication(s): Helfand M, Balshem H. AHRQ Series Paper 2: Principles for developing guidance: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010; 63: 484-490.

Identifying, Selecting and Refining Topics for Comparative Effectiveness Systematic Reviews

Published 27 Apr 2009. Select for HTML or PDF (589 KB).
Other publication(s): Whitlock E, Lopez S, Chang S, et al. AHRQ Series Paper 3: Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010; 63: 491-501.

Developing and Selecting Topic Nominations for Systematic Reviews

Published 19 Nov 2012. Select for HTML or PDF (743 KB).

The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program

Published 24 January 2013. Select for HTML or PDF (743 KB).
Other publication(s): Buckley DI, Ansari M, Butler M, et al. The Refinement of Topics for Systematic Reviews: Lessons and Recommendations From the Effective Health Care Program. J Clin Epidemiol. 2014; 67:425-432.

Finding Evidence for Comparing Medical Interventions

Published 06 Jan 2011. Select for HTML or PDF (357 KB).
Other publication(s): Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011 Nov;64(11):1168-77. PMID: 21684115.

Finding Grey Literature Evidence and Assessing for Outcome and Analysis Reporting Biases When Comparing Medical Interventions: AHRQ and the Effective Health Care Program

Published 18 Nov 2013. Select for HTML or PDF (364 KB).

Avoiding Bias in Selecting Studies

Published 20 February 2013. Select for HTML or PDF (201 KB).

Selecting Observational Studies for Comparing Medical Interventions

Updated chapter published 19 September 2022. Select for HTML or PDF (1 MB)
Other publication(s): Saldanha IJ, Adam GP, Bañez LL, et al. Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: Updated guidance from the Agency for Healthcare Research and Quality Effective Healthcare Program. J Clin Epidemiol. 2022 Sept 19. [Epub ahead of print.] PMID: 36245131.
Original chapter published 14 June 2010. Select for HTML or PDF (297 KB).
Other publication(s): Norris S, Atkins D, Bruening W, et al. Observational Studies in Systematic Reviews of Comparative Effectiveness. J Clin Epidemiol. 2010; 63. 2011 Nov;64(11):1178-86. Epub 2011 Jun 1. PMID: 21636246.

Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions

Updated chapter published 13 December 2017. Select for HTML or PDF (413 KB).
Original chapter published 08 March 2012. Select for HTML or PDF (325 KB).
Other publication(s): Viswanathan M, Patnode CD, Berkman ND, et al. recommendations for Assessing the Risk of Bias in Systematic Reviews of Health Care Interventions. Journal of Clinical Epidemiology. ePub ahead of print. DOI: https://dx.doi.org/10.1016/j.jclinepi.2017.12.004.

Assessing the Applicability of Studies when Comparing Medical Interventions

Published 05 Jan 2011. Select for HTML or PDF (340 KB).
Other publication(s): Atkins D, Chang SM, Gartlehner G, et al. Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2011 Nov;64(11):1198-207. PMID: 21463926.

Prioritization and Selection of Harms for Inclusion in Systematic Reviews

Published 2 February, 2018. Select for HTML or PDF (285 KB)

Assessing Harms when Comparing Medical Interventions

Published 18 Nov 2008. Select for HTML or PDF (524 KB).
Other publication(s): Chou R, Aronson N, Atkins D, et al. AHRQ Series Paper 4: Assessing harms when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010; 63: 502-512.

Quantitative Synthesis—An Update

Published 23 Feb 2018. Select for HTML or PDF (815 KB).

Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update

Updated chapter published 18 Nov 2013. Select for HTML or PDF (334 KB).
Other publication(s): Berkman ND, Lohr KN, Ansari MT, et al. Grading the strength of a body of evidence when assessing healthcare interventions: An EPC update. Journal of Clinical Epidemiology. 2015;68(11):1312–24. doi:10.1016/j.jclinepi.2014.11.023.
Original chapter published 05 Aug 2009. Select for HTML or PDF (193 KB).
Other publication(s): Owens D, Lohr K, Atkins D, et al. AHRQ Series Paper 5: Grading the strength of a body of evidence when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010; 63: 513-523.

Using Existing Systematic Reviews to Replace de Novo Processes in CERs

Published 05 Oct 2009. Select for HTML or PDF (77 KB).

Integrating Bodies of Evidence: Existing Systematic Reviews and Primary Studies

Published 25 February 2015. Select for HTML or PDF (226 KB).

Updating Comparative Effectiveness Reviews: Current Efforts in AHRQ's Effective Health Care Program

Published 05 Aug 2011. Select for HTML or PDF (213 KB).
Other publication(s): Tsertsvadze A, Maglione M, Chou R, et al. Updating comparative effectiveness reviews: current efforts in AHRQ's Effective Health Care Program. J Clin Epidemiol. 2011 Nov;64(11):1208-15. PMID: 21684114.

Guidance for the Conduct and Reporting of Modeling and Simulation Studies in the Context of Health Technology Assessment

Published 4 Oct 2016. Select for HTML or PDF (411 KB).
Other publication(s): Dahabreh I., Trikalinos T., Balk E., Wong J. Recommendations for the Conduct and Reporting of Modeling and Simulation Studies in Health Technology Assessment. Ann Intern Med. 2016 Oct;165(8):575-581. doi: 10.7326/M16-0161.

Page last reviewed October 2022
Page originally created February 2018

Internet Citation: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Content last reviewed October 2022. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.
https://effectivehealthcare.ahrq.gov/products/collections/cer-methods-guide

Select to copy citation