Skip Navigation
Department of Health and Human Services www.hhs.gov
 
 

Slide 1

Engaging Stakeholders in the Effective Health Care Program

Information and tools for researchers and investigators

Speaker notes: The Agency for Healthcare Research and Quality presents the following e-learning tool “Engaging Stakeholders in the Effective Health Care Program” to provide researchers and investigators with foundational information on, and tools for, engaging a broad range of stakeholders in the Effective Health Care Program.  The information presented is adapted from content developed by the Scientific Resource Center at the Oregon Evidence-based Practice Center.

Slide 2

Full Course Outline

  • Module I: AHRQ’s Vision for, Theory, and Principles of Stakeholder Engagement
  • Module II: Stakeholders and Stakeholder Engagement
  • Module III: Points of Engagement in the EHC Program
  • Module IV: Skills for Successful Engagement


Speaker notes: This is the first of four modules as part of this e-learning tool.

Slide 3

Module II

Stakeholders and Stakeholder Engagement

Slide 4

What to Expect in Module II

Module II will cover the following topics:

  • AHRQ’s definitions of stakeholder and stakeholder engagement
  • Types of stakeholders in the EHC Program
  • The roles of stakeholders in EHC Program research
  • Skills necessary for effectively engaging stakeholders
  • Strategies for effectively engaging stakeholders


Speaker notes: There are a myriad of ways to define stakeholder and stakeholder engagement. This module will review AHRQ’s definitions in relation to the EHC Program. This module will also discuss the expanding role of stakeholders in the EHC Program. This module is designed to assist investigators and research institutions to effectively engage stakeholders by matching the “right” stakeholders to the “right” points in the research process, and facilitating the engagement process by systematically involving stakeholders to increase knowledge exchange.

Slide 5

Module II Learning Objectives

After completing Module II, participants will be able to:

  • Define the term stakeholder within the context of the EHC Program
  • Define stakeholder engagement within the context of the EHC Program
  • Define the roles of different types of stakeholders in EHC Program research
  • Describe strategies for engaging stakeholders


Speaker notes: After completing Module II, participants will be able to:

  • Define the term stakeholder within the context of the EHC Program
  • Define stakeholder engagement within the context of the EHC Program
  • Identify differences and similarities between skills needed to engage stakeholders and conduct research
  • Describe methods of engaging stakeholders

Slide 6

Defining Stakeholders

EHC Program stakeholders are:

  • Individuals or organizations who have a personal or professional interest in the topic 
    • Any individual interested in the development or dissemination of EHC Program products
    • Any individual using EHC Program products to make health care decisions


Image: a series of photographs horizontally depicting people of different ages and genders, some in an office setting and others in a healthcare setting. Some of the people are patients and others are healthcare professionals.

Speaker notes: As Atkins (2007) asserts, “to a previous generation, a ‘medical decision maker’ would have been synonymous with a physician.” Understanding of the term “medical decision maker” has expanded and now includes “regulatory agencies, health plans and insurance companies, professional societies, formulary committees, private industry, quality organizations and others” (Atkins, 2007).

For the purposes of the EHC Program, stakeholders are defined as:

Individuals or organizations who have a personal or professional interest in the topic 
Conceivably, anyone could be an EHC Program stakeholder (O’Haire, 2011)

Slide 7

Types of Stakeholders

  • Manufacturers & Industry
  • Institutional health care providers
  • Clinicians
  • Policymakers
  • Insurers & payers
  • Researchers
  • Patient & Consumers


Each of the bullet points is written within a box and meant to represent a bucket of stakeholder type

Speaker notes: Stakeholders are people or groups – each with a unique perspective – who have an interest in health care decision making.  EHC Program stakeholders include: 

  • Consumers, patients, caregivers, and patient advocacy organizations
  • Clinicians and their professional associations
  • Health care institutions, such as hospital systems and medical clinics and their associations
  • Health care policymakers at the Federal, state and local levels
  • Health care researchers and research institutions
  • Health care industry representatives
  • Purchasers and payers, such as employers and public and private insurers

 Slide 8

Engagement Exercise 5: Defining Stakeholders

Which of the examples of individuals could be an EHC Program stakeholder?

    1. A State Governor
    2. A breast cancer survivor
    3. A retired medical professor
    4. A clinical expert in obstetrics
    5. All of the above


Image:  3 photographs overlap one another. The first photograph is of a businessman on his cellphone, the second is one of a mother and her two children, and the third is of a healthcare professional in a room with her patients.

Slide 9

Engagement Exercise 5: Defining Stakeholders
Answer: E

Which of the examples of individuals could be an EHC Program stakeholder?

    1. A State Governor
    2. A breast cancer survivor
    3. A retired medical professor
    4. A clinical expert in obstetrics
    5. All of the above

Speaker notes: Anyone who uses information from EHC Program research to make health care decisions for patients, him/herself, for family members or policy are stakeholders. Stakeholders also include individuals who nominate topics, helps develop research products, or disseminates research products.

Slide 10

Defining Stakeholder Engagement

Engagement is a process of giving voice to and involving (the bolded text is in gold) stakeholders in EHC Program related decision making and research. It serves the purposes of reciprocal learning and knowledge exchange (the bolded text is in gold) to improve health.

Image: A horizontal photograph of 7 different healthcare professionals

Speaker notes: Traditionally, involving stakeholders in research has meant communicating questions or problems in the beginning phases and communicating results and potential application at the end of that research (Burger, Gochfeld & Pletnikoff, 2009). These activities are sometimes referred to as knowledge transfer.

AHRQ, however, is actively moving to knowledge exchange processes that engage stakeholders in strategic ways throughout the research process. In the context of the EHC Program, engagement is a process of giving voice to and involving stakeholders in EHC Program related decision making and research. It serves the purposes of reciprocal learning and knowledge exchange to improve health. This ultimately requires moving from communicating questions and results to developing more interactive and in-depth relationships. This type of interaction is important to the reciprocal knowledge exchange AHRQ is trying to achieve.

One idea central to this level of involvement is that stakeholders have an ongoing relationship with the EHC Program, research institutions, and a topic.

Slide 11

Process of Stakeholder Engagement
Each of the bullets is encapsulated in a box from left to right, with arrows pointing between each box from left to right, meant to depict a process

  • Receive
    • Stakeholders are made aware of research and its availability to assist with their decision-making
  • React
    • Stakeholders provide input into priorities and resource use
  • Participate
    • Stakeholders influence priorities and resources
  • Engage
    • Stakeholders share in planning and action


Each bullet is written in one of the arrows

  • Website downloads, dissemination partnerships
  • Public comment on draft key questions, public nominate topics
  • Key informants and technical expert panels
  • Relationships with AHRQ TOOs and EPCs


Speaker notes: As the definition of stakeholder engagement connotes, working with decision-makers is not a one-time event. True engagement moves beyond one-way communication (knowledge transfer) to reciprocal knowledge interaction and exchange (Hashagen, 2002). This can be viewed as levels of stakeholder engagement:

1. "Receive" - stakeholders are made aware of research and its availability to assist with their decision-making; such as website or dissemination partnerships.
2. "React" - stakeholders provide input into priorities and resource use; such as public comment on draft documents.
3. "Participate" - stakeholders influence priorities and resources; by participating as key informants or on technical expert panels
4. "Engage" - stakeholders share in planning and action for research activities

As a result, it is important to develop specific strategies at each EHC Program phase to effectively achieve stakeholder engagement and build functional relationships that benefit EHC Program activities.

Adapted from Hashagens (2002). Models of community engagement. Scottish Community Development Centre.

Slide 12

Engagement Exercise 6: Levels of Engagement

The following slide provides examples of stakeholder involvement opportunities. Select the level of engagement that best describes each activity.

Each of the bullets is encapsulated in a box from left to right, with arrows pointing between each box from left to right, meant to depict a process

  • Receive
    • Stakeholders are made aware of research and its availability to assist with their decision-making
  • React
    • Stakeholders provide input into priorities and resource use
  • Participate
    • Stakeholders influence priorities and resources
  • Engage
    • Stakeholders share in planning and action

Slide 13

Engagement Exercise 6: Levels of Engagement
1) A  Federal agency posts a new study on their web page.
a) Receive    
b) React                   
c) Participate            
d) Engage
2) A  State agency solicits feedback on health care priorities from citizens through State-wide meetings. Participants vote on top priorities.
a) Receive
b) React                    
c) Participate            
d) Engage

3) A Congressperson hosts a town hall meeting to discuss health care reform.
a) Receive    
b) React                    
c) Participate            
d) Engage

4) A collaboration between government agencies and community organizations identifies the reintegration of drug users after incarceration as a community concern. The collaboration partners with representatives from local service providers, the city health organization, advocacy groups & residents to lead a study to develop strategies to improve the health of the population.
a) Receive    
b) React                    
c) Participate            
d) Engage

Slide 14

Engagement Exercise 6: Answers Levels of Engagement
1) A  Federal agency posts a new study on their web page.
a) Receive   
b) React                   
c) Participate            
d) Engage
2) A  State agency solicits feedback on health care priorities from citizens through State-wide meetings. Participants vote on top priorities.
a) Receive    
b) React       
c) Participate            
d) Engage
3) A Congressperson hosts a town hall meeting to discuss health care reform.
a) Receive    
b) React                    
c) Participate
d) Engage
4) A collaboration between government agencies and community organizations identifies the reintegration of drug users after incarceration as a community concern. The collaboration partners with representatives from local service providers, the city health organization, advocacy groups & residents to lead a study to develop strategies to improve the health of the population.
a) Receive    
b) React                    
c) Participate                      
d) Engage

Slide 15

Benefits of Stakeholder Engagement
There are many benefits to stakeholder engagement in research, including increased:

  • Clarity of research questions
  • Knowledge exchange
  • Relevancy and context
  • Application of evidence
  • Dissemination avenues

Image: a picture of different types of healthcare professionals interacting with patients.

Speaker notes: Engaging stakeholders in the research process is critical to facilitate the use of evidence in practice (Keown, Van Eerd & Irvin, 2008; Graham et al., 2006), and direct interactions with research increases this utilization (Lavis et al., 2002; Lomas, 2000). The benefits of engaging stakeholders in the research process include:

  • Increased clarity of research questions.
  • Increased knowledge exchange. Stakeholders are experts in their own field, and can offer complementary knowledge and expertise.
  • Increased relevancy and context. Stakeholders can provide real-world context that may guide research questions and result in more useable end products.
  • Increased application of evidence.
  • Increased dissemination avenues. Stakeholders can be one of the greatest assets to increased dissemination of research results.

Slide 16

Points of Stakeholders in EHC Program Research

  • Suggest a Research Topic
  • Provide Insights on Research Priorities
  • Refine Topics & Develop Key Questions
  • Help Develop Research Approach
  • Review Draft Research Findings
  • Shape & Inform Research Products & Tools
  • Distribute Products and Lead in Implementation


Each of the bullet points is written in a separate box meant to show buckets of different points in EHC program research

Speaker notes: A key element of the EHC Program has been the engagement of external stakeholders, those individuals and organizations outside the research institutions and the Agency who have an interest—personal or professional—in the findings and applications of EHC Program research. AHRQ continues to look for and expand opportunities to engage stakeholders.
Opportunities for involvement during the research process in the EHC Program include:

  • Suggest a research topic
  • Provide insights on research priorities
  • Refine the research topic and develop key questions
  • Help develop research approach
  • Review draft research findings
  • Shape and inform research products and tools
  • Distribute  research products and lead in implementation


AHRQ’s vision of stakeholder involvement includes the development of ongoing relationships between research institutions and stakeholders—particularly patients and patient groups and frontline clinicians. These collaborative relationships will, in turn, lead to increased topic nominations, topic prioritization, refinement and review, and ultimately to enhanced product dissemination. They will also assist in identifying  and prioritizing research needs that are important to real world decision-makers.

Slide 17

Roles of Stakeholders in EHC Research

  • Key Informant Interviews
    • Identify & clarify important gaps in evidence
    • Identify & clarify important intermediate and long term health outcomes
    • Identify key questions that relate to important outcomes
  • Technical Expert Panel
    • Provide  expert guidance on  clinical and  methodological issues
    • Provide expert guidance on evidence needed to inform decision-making
  • Stakeholder Discussions
    • Refine & prioritize research and help narrow the scope of the research topic and establish specific research questions.


Speaker notes: Stakeholders may be engaged throughout the research process in different ways; as key informants, as participants on Technical Expert Panels or as part of larger stakeholder group discussions.

One way is to systematically engage a broader group of stakeholders is as key informants to solicit diverse perspectives.  Input from key informants will assist with identifying and clarifying the important intermediate and long term health outcomes, as well as the key questions that relate to those outcomes.

Stakeholders serving on Technical Expert Panels help focus the development of research.  For a Research Review, this might include help to focus the literature search, identify criteria, and assist in the evaluation of available evidence.  For Original Research, stakeholders might advise on the treatment options to evaluate, study populations to include, and outcomes to assess.

Stakeholders may also be involved as part of multi-stakeholder discussions.  These discussions provide the opportunity for a diverse group of stakeholders (clinicians, patients & consumers, policy-makers, payers, etc) to come together to refine and prioritize research.  This intent of this process is to ensure research addresses the questions important to decision-makers, represents an accurate scope of issues, and produces the most valuable product.

Slide 18

Differing Roles of Stakeholders

  • Patients, Caregivers, Advocates, Policymakers, Frontline Clinicians, etc.
  • Researchers & Funders
  • Methods & Clinical Experts


Each of the bullets is written on a picture of a mechanical gear to show how they work together

Slide 19

Stakeholder Interests
Patients & Consumers

  • Health outcomes
  • Treatment options
  • Benefits & harms
  • Quality of life considerations


Image: a picture of a senior citizens couple

Clinicians

  • Clinical outcomes
  • Diagnostic & treatment options
  • Benefits & harms
  • Creating guidelines
  • Documenting standard care


Image: a picture of a physician

Policymakers

  • Benefit design
  • Guideline creation
  • Health outcomes
  • Cost effective population health


Image: a picture of a woman showing a thumbs up, and dressed in a business suit

Speaker notes: Stakeholder interests often differ. For example, patients & consumers may be most interested in treatment options, benefits and harms, and quality of life issues. Clinicians may be most interested in clinical outcomes, and diagnostic and treatment options for their patients. In contrast, policymakers are often concerned with using the evidence to assist with benefit design, guideline creation, health outcomes for specific populations, or to help them make other population-based health decisions that are effective and cost efficient.

By engaging multiple types of stakeholders and incorporating their varied interests, research products will be more relevant, useable, and appropriate for end-users.  Investigators should decide which perspectives are most relevant and appropriate to the topic, and at what point, or points in the research process each perspective is pertinent.

Slide 20

Challenge: Managing Bias

  • Suggest a Research Topic
    • Identify real-world questions for health & health care decisions
  • Provide Insights on Research Priorities
    • Provide input to develop and prioritize research gaps
  • Refine Topics & Develop Key Questions
    • Ensure key questions accurately describe decisional dilemmas  
  • Help Develop Research Approach
    • Provide input on methods, clinical relevance and appropriate outcomes to assess
  • Review Draft Research Findings
    • Provide input to improve quality and responsiveness of research reports
  • Shape & Inform Research Products & Tools
    • Help craft meaningful messages
  • Distribute Products and Lead in Implementation
    • Share information with others; take the lead in using evidence to inform health care practices


Each of the main bullet points is written within in a different box, showing the different buckets of points of engagement and stakeholder interests

Speaker notes: Engaging stakeholders throughout the research process increases relevancy, utilization, and dissemination. The key, however, is to match stakeholder interests with specific points in the research process. Not all stakeholder interests will help achieve the objective of each point in the process, nor will all points in the process be valuable to all stakeholders.

Before involving stakeholders, investigators should decide which perspectives are most relevant and appropriate to the topic, and at what point, or points in the research process each perspective is pertinent. Investigators should consider the overall objective of the research, and use it to guide a systematic approach to involving stakeholders. The approach should emphasize the importance of identifying the questions most salient to end-user stakeholders – particularly patients and their physicians or providers.

Investigators should also consider what stakeholders will gain from being involved in the research, and how their concerns will be addressed. Logistical issues such as work or child care schedules are also important to stakeholders who cannot participate as part of their normal job duties.

Slide 21

Strategies for Engaging Patients & Consumers

  • Explain value and benefit
  • Recognize expertise
  • Address time commitment and scheduling issues
  • Recognize additional challenges to participation, such as:
    • Child care
    • Work hours
    • Transportation
    • Medical constraints
  • Explicitly acknowledge interests
  • Acknowledge cultural differences
  • Provide overview of process


Image: 3 photographs overlapping each other. One is of a business man at his desk, the second of a boy with a backpack and the third of a busy highway.

Speaker notes: Patients & consumers can offer important perspectives in the research process. Patients & consumers can be identified locally or nationally. They should be recognized as experts in their disease or condition experience, and can provide invaluable context for research questions.

When asking patient & consumer representatives to participate in the research process it is important to explain the value they add, as well as the benefits to them. It is also helpful to recognize their expertise and unique experiences and knowledge, which can be assets to the research. Addressing issues, such as time commitments, early in the process will help to ensure a positive experience for all involved. Since many consumers will not participate as part of their job, consideration may need to be given for job and/or childcare responsibilities.

Patients & consumers can be identified for participation using a variety of venues and approaches, including:

  • Centralized patient advocacy groups (Consumers United for Evidence-based Healthcare, Research Advocacy Network, National Breast Cancer ProjectLEAD)
  • Government agencies that engage patient advocates (FDA patient representative program, Congressionally Directed Medical Research Programs, NIH)
  • Local or national advocacy organizations
  • Other research institutions

Slide 22

Strategies for Engaging Clinicians

  • Explain value and benefit
  • Recognize expertise
  • Provide process overview
  • Discuss time commitment and scheduling concerns
  • Explicitly acknowledge interests


Image: a horizontal series of photographs depicting first an eye exam, then various clinician and patient interactions

Speaker notes: Frontline clinicians working directly with patients offer unique insights into the context of diagnosis and treatment and the outcomes important to the patient/provider relationship. The clinician perspective is also important for relevant comparators and real-world decisions.

When asking clinicians to participate in the research process it is important to explain the value they add, as well as the benefits to them. It is also helpful to recognize their expertise and provide an overview of the research process and expectations for time commitment. Investigators may need to allow for and address issues related to the time providers’ participation may take away from seeing patients.

Clinicians can be identified for participation using a variety of venues and approaches, including:

  • Local clinics, hospitals or medical centers
  • National or local professional organizations
  • Universities

Slide 23

Strategies for Engaging Policymakers

  • Explain value and benefit
  • Recognize expertise
  • Address time constraints
  • Discuss strength and use of evidence
  • Explicitly acknowledge interests


Image:  3 photographs overlapping each other. One is of a health insurance report, the second of a policymaker on the phone, and the third of policymakers in a conference room.

Speaker notes: Policymakers place great value on and respect for the independence of research. They are often interested in applying research to population health decisions, including but not limited to benefit design and coverage. They are also often under pressure from patients, advocates, executives, and elected officials to make a specific decision, and to do so within a limited time frame. These constraints create a cultural reality for policymakers that can be quite different from that of researchers. Policymakers often hope for, or expect, rapid turnaround, just in time information, or “good enough” research to help address the dilemmas they face. It is important to understand and acknowledge these differences so that investigators can focus on and address the important interests and objectives of policymakers.

Many policymakers have also had clinical experience, although they may not be functioning in a clinical capacity when they interface with researchers. Policymakers can be identified for participation using a variety of venues and approaches, including:

  • National or large payer organizations
  • Networks or organizations
  • Foundations with a focus on policy

Slide 24

Engagement Exercise 7: Identifying & Recruiting Stakeholders
Imagine you are involved in research comparing multiple therapies for adults with advanced dementia. Consider the following:

  1. Which stakeholder perspectives would you involve?
  2. What perspectives and challenges will each bring?
  3. How will you identify each?
  4. How would the research plan be different as a result?

Slide 25

Summary of Module II

  • AHRQ’s definition of stakeholder is broad and inclusive
  • AHRQ’s definition of stakeholder engagement incorporates knowledge exchange
  • There are many opportunities to involve stakeholders in the EHC Program
  • Matching stakeholders and