For research results to be useful and relevant to decisionmakers, the active involvement of stakeholders in the research process is critical. As such, stakeholder engagement is a fundamental aspect of comparative effectiveness research (CER), and the impact of CER depends on developing effective processes for the meaningful participation of stakeholders throughout the research continuum. A stakeholder in this context refers to anyone who plays a role in health care decisionmaking, including patients, clinicians, public and private policymakers and payers, and members of industry.
The centrality of stakeholder engagement to CER has been a guiding principle of the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care (EHC) Program since its inception. Section 1013 of the 2003 Medicare Prescription Drug, Improvement, and Modernization Act, which instructed AHRQ to conduct and support CER, mandated broad and ongoing consultation with relevant stakeholders.
In response to this mandate, AHRQ’s EHC Program has created multiple mechanisms for stakeholder input and relies on ongoing consultation with stakeholders to ensure that the program responds to the most pressing issues and produces information that is useful for health care decisionmakers. General stakeholder input is collected on an ongoing basis through topic nominations and public comment submitted via the EHC Web site. In recent years, AHRQ has specified stakeholder engagement as a core element of funding announcements, and new grantees are required to describe plans for involving stakeholders in their work. The EHC Stakeholder Group, a 20-member volunteer committee representing the full range of stakeholders, was established in 2007 and convened quarterly until July 2012 for the purpose of providing insights into stakeholder perspectives to increase the impact of the EHC Program. The John M. Eisenberg Center for Clinical Decisions and Communications Science, a component of the EHC Program responsible for translating research results into guides and tools that are useful to health care decisionmakers, regularly solicits stakeholder feedback on its products through focus groups and telephone interviews. In addition, AHRQ’s Community Forum initiative, initially funded by the American Recovery and Reinvestment Act of 2009, was created to examine and systematize stakeholder engagement in the EHC Program.
AHRQ created the Evidence-based Practice Centers (EPCs) in 1997 to produce comparative effectiveness or effectiveness reviews on medications, devices, and other health care services with the goal of helping patients, physicians, policymakers, and payers make better decisions about treatments. The EPCs solicit stakeholder input at multiple points in the review process, including topic generation, development, and refinement; protocol development; future research needs development; and translation and dissemination.
EPC researchers receive stakeholder input through open invitation during public posting and comment periods, as well as through direct contact with selected stakeholders. Consultation with key informants is critical at the topic development and refinement stages to ensure the relevance and usefulness of the questions being asked. Stakeholders also participate in Technical Expert Panels to provide input during research reviews and in identifying future research needs. Recognizing that different audiences have different needs from comparative effectiveness reviews, EPC process guidelines emphasize the importance of engaging a range of stakeholders, including clinicians, patients and consumers, researchers, government agencies, industry representatives, payers, and policymakers. However, due to issues of timeliness and cost, EPC researchers cannot engage all stakeholders at each step for every topic.5 Efficiently incorporating the most important perspectives where they are likely to have the greatest impact requires careful consideration of the evidence needs of different stakeholder groups.
Health care payers, defined here as any entity other than the patient that finances or reimburses the cost of health services, rely on comparative effectiveness reviews to help them make rational choices based on the available clinical evidence. When multiple treatments work for the same condition, information on the pros and cons of available alternatives enables payers to make more effective decisions that take into account potential variation in patient or provider treatment choices. The perspective of payers is therefore critical to the comparative effectiveness review process. Although payers have participated as key informants, there are a number of obstacles to consistently including this stakeholder group in EHC projects, such as identifying appropriate individuals to invite within often-complex organizations, and questions remain concerning the most appropriate time and method to capture their perspective. Furthermore, payers that have participated as stakeholders have often included individuals involved in other aspects of the EHC Program (e.g., the EHC Stakeholder Advisory Group) and have not represented the full spectrum of private and public health care payers.
Payers have conveyed a number of challenges related to the use of EHC reports for decisionmaking. At a 2011 meeting of the EHC Stakeholder Group and subsequent presentation at the EPC Directors meeting, private and public payers serving as stakeholders described the following challenges:
- Payers have to make policy decisions whether the evidence is good, bad, uncertain, or nonexistent. Therefore, it is problematic when researchers say, “We have no evidence.” Even in the case of very inadequate evidence, we do know some things (e.g., treatments that are unsafe, treatments that are safe but do not work).
- The causes of uncertainty need to be communicated more clearly.
- Information concerning the balance of benefits and harms needs to be more readily accessible.
As part of the Community Forum initiative, we sought to expand on previous payer input by soliciting feedback from a diverse panel of health care payers. The goals of the project were to broaden payer involvement in the EHC Program and develop recommendations regarding—
- Effective means of identifying, recruiting, and engaging payers
- Optimal points of engagement for payers in the EHC process
- Modifications or additions to EHC reports that would make them more useful for payers • Modifications or additions to summary guides produced by the Eisenberg Center that would make them more useful for payers
In this activity, we sought a better understanding of the evidence needs of health care payers, their preferences for the conduct and presentation of comparative effectiveness reviews, and strategies for expanding payer involvement in the EHC Program. The results are limited by the number and variety of payers involved in the activity, and those who agreed to participate may have a higher level of familiarity with the EHC Program than those who declined. Results are also limited by the focus on only two comparative effectiveness reviews. However, we believe many of the findings are generalizable to a larger payer audience and to other comparative effectiveness reviews.
Incorporating the input of multiple stakeholders, including payers, into the already tight timeline for comparative effectiveness reviews is an ongoing challenge for the EHC Program. Determining how payer input can be efficiently collected and when it is likely to have the greatest impact is critical. Provided EHC researchers are cognizant of issues that are routinely important to payers (e.g., cost, resource utilization, subpopulations, QOL, observational studies), it may not be essential to involve payers in every review. Researchers should consider the potential impact of review findings on payer decisionmaking in determining the optimal level of input. With regard to the optimal point of engagement, involving payers at the topic development and refinement stage is likely to increase the utility of comparative effectiveness reviews for coverage and policy decisions, particularly for topics that are of special interest to payers. When time and resources for obtaining payer input are limited, one strategy might be to target outreach to payer groups for input on Key Questions during the public comment period. For reviews already in process, payer input regarding the presentation of findings is likely to result in reports that are more useful for this stakeholder group. One possible approach is to present a summary of review findings to payers during the peer review phase, and solicit input regarding what information would be most useful to them and how it should be presented.
The need for timely information to address coverage decisions was a recurring theme in our interactions with the panel. Some payers expressed frustration at the length of time required for completion of AHRQ systematic reviews and raised the possibility of shortening the timeline. At the same time, they indicated that the value of AHRQ reviews is their thoroughness and rigor, and that other, commercial, services provide the quick turnaround they sometimes need. Accepting that the EHC Program cannot fill every evidence need of every stakeholder group, one conclusion is that it should focus on what it does best—producing comprehensive comparative effectiveness reviews that do not compromise methodological rigor in the interest of quick turnaround.
The potential role of other AHRQ products in addressing payers’ needs also warrants further consideration. For example, payers may not be aware that “topic suggestion dispositions” are posted for all nominated review topics. Payers seeking evidence for a particular topic may find it useful to identify, in addition to available reviews, reviews that were suggested but not conducted due to lack of available evidence. Disposition reports are available on the EHC Web site and are searchable by nomination year and keywords. The Healthcare Horizon Scanning System monitors emerging health care technologies and innovations and creates an inventory of those that have the highest potential for impact on clinical care, the health care system, patient outcomes, and costs. The payers we spoke with were not aware of the Horizon Scanning program, but may find this product valuable in anticipating future coverage requests and identifying topics that are the most relevant and technologies for which additional evidence will be needed.
Technical briefs are another AHRQ product with potential value to health care payers. In contrast to comparative effectiveness reviews, technical briefs address emerging topics for which a systematic review may be premature. The goal of a technical brief, as described by one EHC researcher, is to find out what we do not know about a particular topic and why we do not know it. This may involve taking a topic for which there is a lot of “hype” and grounding it in the reality of existing evidence. This goal is consistent with the need our panel expressed to understand what areas lack evidence. Moreover, the intended 6- to 9-month timeframe for the preparation of technical briefs is in keeping with payers’ need for rapid information, particularly when existing evidence for a topic is limited.
The need to increase general outreach to payers regarding the EHC Program is a key finding from this project. Although health care payers are important consumers of evidence, the payers on this panel had limited knowledge and awareness of AHRQ products. Direct outreach to individual payers (i.e., via email notifications) may not be ideal given the volume of communications they receive. Partnering with payer organizations to raise awareness about the range of AHRQ research products and their potential utility for payers is one potential strategy.