This report describes the challenges faced by systematic reviewers in the Evidence-based Practice Center (EPC) program when reviewing behavior change interventions, and considers whether current guidance is specific enough to address these challenges.
A workgroup of members from EPCs, the Scientific Resource Center (SRC), and AHRQ was convened to describe systematic review methods for behavior change interventions in the EPC program and similar programs, and identify gaps in guidance. Complementary methods including a literature scan and key informant interviews were undertaken to reach the objectives. A literature scan was conducted to identify current guidance and methods literature on the inclusion of behavior change literature in systematic reviews. Interviews were held with thought leaders in the field to identify current practices and opinions on using behavior change literature. Workgroup members summarized information from the literature and interviews.
We identified specific challenges related to study selection (particularly in using single subject experimental designs), data extraction, risk of bias, strength of evidence, presentation and quantitative analysis. Challenges are particularly acute when fields are focused on identifying and describing heterogeneity in populations and treatment effects rather than identifying generalizable effects. Guidance is available, but the sources of guidance most used by the EPC program (EPC, Cochrane, and the National Institute for Health and Care Excellence) lack specificity in addressing these challenges.
Challenge of reviewing behavior change literature exists at each step of EPC systematic reviews. A larger discussion is needed about how EPC program methods can meet the evidentiary needs and questions of decisionmakers, consistent with the program's aim to provide accurate, independent, and scientifically rigorous information. Any methods approaches moving forward should address the specific needs of decision makers for using reviews of behavior change literature.