Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC

SHARE:

FacebookTwitterFacebookPrintShare

Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses

Research Report Sep 9, 2013
Download PDF files for this report here.

Page Contents

People using assistive technology may not be able to fully access information in these files. For additional assistance, please contact us.

Abstract

We propose a framework for organizing and describing the rationale behind methodological recommendations, and for communicating one's confidence in them. We start by defining the background context in which the recommendations are made. We distinguish recommendations that are testable (in that their likelihood to hold can be informed by theoretical arguments or empirical data) from nontestable ones, which represent beliefs or assumptions that are not verifiable. Nontestable statements can be justified, but their validity cannot be demonstrated. Testable statements can be assessed in terms of the adequacy of their evidentiary basis.

Both testable and nontestable statements can be evaluated regarding their feasibility of implementation, the expected impact of following them versus not, and their congruence with the desired characteristics of the background context. Considering these four dimensions, one can indicate one's confidence (along a continuum) about how closely a methods recommendation should be followed: some recommendations may be perceived and communicated as mandatory items (minimum standards), while others as highly desirable but not mandatory items. Finally, giving specific methods guidance for addressing difficult or ill-defined problems can be premature pending more research or clearer problem specification. In such cases, describing the problem and laying out attributes of a satisfactory resolution can serve until actionable guidance can be offered.

We view the proposed framework strictly as a communication tool to describe rationale for the recommendations to the intended audience and not as a device to deduce the “correctness” of a recommendation. Nonetheless, application of the framework can facilitate the latter, because methodologists can most effectively and honestly critique recommendations whose rationale is transparent. We anticipate that this initial instantiation of the framework for making methods recommendations will evolve.

Project Timeline

Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses

Nov 23, 2010
Topic Initiated
Sep 9, 2013
Research Report
Page last reviewed November 2017
Page originally created November 2017

Internet Citation: Research Report: Towards a Framework for Communicating Confidence in Methodological Recommendations for Systematic Reviews and Meta-Analyses. Content last reviewed November 2017. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.
https://effectivehealthcare.ahrq.gov/products/methodological-recommendations-framework-communications/research

Select to copy citation