Skip Navigation
Department of Health and Human Services www.hhs.gov
 
 

Slide 1

Engaging Stakeholders in the Effective Health Care Program

Information and tools for researchers and investigators

Speaker notes: The Agency for Healthcare Research and Quality presents the following e-learning tool “Engaging Stakeholders in the Effective Health Care Program” to provide researchers and investigators with foundational information on, and tools for, engaging a broad range of stakeholders in the Effective Health Care Program.  The information presented is adapted from content developed by the Scientific Resource Center at the Oregon Evidence-based Practice Center.

Slide 2

Full Course Outline

  • Module I: AHRQ’s Vision for, Theory, and Principles of Stakeholder Engagement
  • Module II: Stakeholders and Stakeholder Engagement
  • Module III: Points of Engagement in the EHC Program
  • Module IV: Skills for Successful Engagement


Speaker notes: This is the 4th module in a four-part e-learning tool series:

Slide 3

Module IV

Skills for Successful Engagement

Slide 4

What to Expect in Module IV

Module IV will cover the following topics:

  • Skills to engage stakeholders
  • Preparing investigators and stakeholders for involvement
  • Discussion/meeting facilitation basics
  • Addressing challenges


Speaker notes: Different stakeholders have different needs, expectations, and biases. They also have different perspectives and expertise that helps to ensure research is relevant and applicable. Challenges identified by researchers and AHRQ staff related to engaging stakeholders include identifying and preparing them, supporting their involvement, and communication for meaningful knowledge exchange. Advanced planning and preparation, clear roles and expectations, flexibility, and strong communication and facilitation skills are keys to ensuring efficient and effective stakeholder engagement.

Many of the skills necessary to effectively engage stakeholders may require an expansion of existing knowledge and abilities, but investigators and research institutions have resources, support, and tools available to assist with any aspect of stakeholder engagement. Module IV will cover the skills to engage stakeholders, preparing for involvement, facilitation basics, and ways to address specific challenges.

Slide 5

Module IV Learning Objectives

After completing Module IV participants will be able to:

  • Describe skills needed for stakeholder engagement
  • List at least five skills for successful facilitation
  • List at least three strategies for mitigating bias and conflict of interest
  • Provide at least two suggestions for effective evaluation and continuous quality improvement


Speaker notes: After completing Module IV participants will be able to:

  • Compare and contrast skills needed for stakeholder engagement and skills needed for research
  • List at least five skills for successful facilitation
  • List at least three strategies for mitigating bias and conflict of interest
  • Provide at least two suggestions for effective evaluation and continuous quality improvement

Slide 6

Skills for Engagement

  • Thoughtfully select stakeholders
  • Actively facilitate  stakeholder engagement activities
  • Carefully document input
  • Provide opportunities for stakeholders to evaluate their experiences


Image: a photograph of a diverse group of people standing together

Speaker notes: One task important in conducting comparative effectiveness research is to carefully and systematically elicit the questions and outcomes important to stakeholders.

Approaches to this thoughtful engagement can happen at several points in the process. For example:

  • Thoughtfully selecting stakeholders. Researchers should carefully identify and select participants so that stakeholders’ interests are aligned as closely as possible with the point of involvement, and objectives of the research.
  • Actively facilitate kick-off and key informant calls and provide opportunities for feedback or further communication beyond these calls to ensure all perspectives are heard.
  • Carefully document input and information provided by stakeholders in a systematic way, and organize it using the same framework as written or published data.
  • Provide opportunities for stakeholders to evaluate their experiences working with investigators and incorporate this feedback in future activities.

Slide 7

Skills for Successful Engagement

  • Ability to listen without judgment
  • Ability to put your own perspective aside
  • Ability to show respect for opinions of others, even when they disagree with you
  • Comfort dealing with conflict
  • Belief that groups working together are smarter than individuals working alone
  • Open to feedback about self
  • Openness


Speaker notes: Adapted from Cameron J and Gibson K (2001). Community Partnering Project: Community Researchers. (www.communityeconomies.org/training)

Stakeholder engagement is a complex process. It is important to be ready and willing to respond to various stakeholder needs to make the process accessible and relevant (Hashagen, 2002). Effectively engaging a diverse group of stakeholders requires specific skills, such as the ability to stay objective, translate technical information to non-research audiences, summarize and recap discussion points, work with diverse groups of people, actively listen, and to work as part of a team.

The skills needed to effectively engage stakeholders and apply their expertise leans heavily on communication, facilitation, and process. Engaging stakeholders may also require innovation, flexibility, and creativity. Some of these skills are similar to those typically used by investigators to complete research processes (such as synthesizing large amounts of information). Others, such as group process skills, may be different, or applied differently.

Slide 8

Skills for Successful Engagement (cont’)

  • Ability to balance participation of stakeholders with achieving results
  • Active listening
  • Flexibility
  • Ability to synthesize large amounts of diverse input
  • Ability to summarize
  • Knowledge of group processes
  • Ability to structure meetings to allow participants to focus on substance of issue


Each of the bullets is encapsulated in a box to represent a separate bucket of stakeholder activities.

Speaker notes: Stakeholders may be engaged at only one point, while others will interact with the Program at multiple points during the research process depending on the activity. Each point of engagement serves a different purpose, and provides distinct opportunities for involvement. The “art” of engaging stakeholders involves matching their interests with research objectives and the purpose of each Program phase.

AHRQ envisions ongoing relationships between its research institutions and stakeholders so that continued collaboration and knowledge exchange occurs throughout topic identification and nomination, topic refinement, research development and translation.

Slide 9

Introduction to Facilitation & Process Skills

  • Facilitation and process skills are:
    • Intentional and planned
    • Carried out before, during, and after a meeting
  • Groups perform more effectively when actively facilitated by someone whose focus is group process rather than content and decisions (Stewart, 2006)


Speaker notes: Effectively engaging a diverse group of stakeholders requires specific skills, in addition to those typically used to conduct research. These skills are similar to those found in consensus building processes, and are used by facilitators and mediators. “Facilitation” refers to the process of effectively engaging stakeholders in knowledge exchange; the skills, resources, logistics, advanced planning, and follow-up needed to successfully achieve meeting/discussion goals. Done effectively, facilitation is not a one-time event that occurs while someone leads a meeting or discussion.

Facilitator refers to the “role” someone is given to carry out effective processes and discussions. While it is important to have content or expert knowledge of the topic involved in discussions, in order to achieve the best outcomes, the facilitator’s role should focus on the discussion or group processes.

Slide 10

Facilitator Attributes

  • (Picture of a camera lens) Focused on process
  • (Picture of books on a pedestal) Knowledgeable about the topic
  • (Picture of a question mark) Comfortable questioning & probing
  • (Picture of business people conversing) Focused, alert & interested in the discussion
  • (Picture of a scale) Able to ensure all participants equally participate
  • (Picture of a telephone) Available to talk/debrief with participants


Speaker notes: The literature on facilitation and effective group process includes detail about preparation, skills, and role of the facilitator. Some basic guidance includes:

  • The facilitator should be focused on process, not content. However, the facilitator should be knowledgeable or skilled enough to question, probe, and approach all participants, consumers and experts alike.
  • The facilitator sets the tone for the discussion by example, adherence to ground rules, and by staying focused, alert, and interested in the discussion.
  • The facilitator uses a myriad of methods to ensure that all participants have an equal opportunity to contribute. This may mean using different techniques for different participants.
  • The facilitator is available to participants before group discussions to ensure they are prepared, and after to note any additional input and suggestions for process improvement.

Slide 11

Attention to Process

  • Attention to process is important:
    • Prior to meetings
    • During meetings
    • After meetings
  • Ensure  attention to logistics
  • Ensure avenues for input


Image: a photograph of four people in a conference room discussing a diagram on the white board

Speaker notes: Effective facilitation involves preparation before, during, and after meetings. Prior to meetings it is important to communicate effectively with all participants, and allow sufficient time and materials to prepare. Preparation may take different forms for different participants or different needs. It is also important to plan the processes that will be used to gather feedback.

During the meeting it is important that all participants have an opportunity to provide input. To accomplish this, it will be necessary to maintain the focus of the discussion and keep all participants on track. After the meeting, all participants should have the opportunity to review the meeting summary and other materials to ensure they accurately reflect their input. It is also important to provide an opportunity for meeting evaluation and improvement.

Slide 12

Facilitation & Logistics

To further facilitate involvement, planner should consider:

  • Meeting time
  • Composition of group
  • Reading level of materials
  • Incentives
  • Time commitment challenges, such as child care
  • Alternative forums
    • Individual interviews
    • Written testimony
    • Web-based discussions
    • Focus groups


Speaker notes: Throughout the meeting planning, attention should be paid to logistical arrangements. In order to encourage the best level of involvement, planners should consider time of day (during the workday may facilitate involvement of some stakeholders, while hampering the involvement of others), preparation materials, and whether incentives may be needed to encourage participation.

Finally, in order to allow the broadest participation possible, investigators should explore forums, other than meetings, that will allow for stakeholder input. Alternative forums might include individual interviews or written testimonies, web-based discussions, or focus groups held at already occurring meetings of consumers.

Slide 13

Engagement Exercise 8:

Facilitation Process

Each box below represents a facilitation process or activity, and each circle represents a phase in meeting planning. 
There are three circles. Each circle contains a phrase, in this order from left to right:

  • Before Meeting
  • During Meeting
  • After Meeting


Under the
three circles are six boxes, and each box contains a phrase, in this order from left to right:

  • Schedule & time of meeting
  • Introducing participants to group
  • Preparing materials
  • Participant incentives
  • Accuracy of meeting summary
  • Additional avenues for input

Slide 14

Engagement Exercise 8:

Facilitation Process
At which phase is it most important to consider each facilitation activity?

  • Before Meeting
  • During Meeting
  • After Meeting


In the first circle – Before Meeting, there are four boxes:

  • Schedule & time of meeting
  • Preparing materials
  • Participant incentives
  • Additional avenues for input


In the second circle – During Meeting, there is one box

  • Introducing participants to group


In the third circle – After Meeting, there is one box:

  • Accuracy of meeting summary

Slide 15

Skills to Address Challenges

  • Facilitation skills
  • Identifying stakeholders
  • Preparing stakeholders
  • Managing bias & conflict
  • Continuous improvement


Image: a picture of a sign against a grass field. The sign contains an image of a double sided arrow, with one side labeled failure and the other side labeled success

Speaker notes: Challenges identified to successful stakeholder engagement include:

  • Facilitation skills
  • Identifying stakeholders
  • Preparing stakeholders
  • Managing bias & conflict
  • Continuous improvement


The next set of slides will focus on these challenges and resources to address them.

Slide 16

Challenge: Identifying Stakeholders

  • Professional colleagues and networks
  • AHRQ staff
  • Consumer organizations
  • Policy networks and organizations
  • Literature searches
  • Web searches
  • Policy networks


Image: a photograph of a man and a woman, both professionally dressed, shaking hands

Speaker Notes: One of the challenges identified by researchers to engaging stakeholders is knowing how and where to locate them. Traditionally, research institutions have not had difficulty finding technical or professional stakeholders to participate in research product development. Locating patients and consumers or front-line clinicians has been more challenging. The EHC Program has several avenues to assist in locating and recruiting stakeholders. These include:

  • Professional colleagues and networks
  • Professional societies and associations
  • AHRQ staff
  • Patient and consumer organizations
  • Literature searches
  • Web searches
  • Policy networks and organizations


More specific information on locating stakeholders for each EHC Program point of involvement is contained in Module III.

Slide 17

Challenge: Engaging & Preparing Stakeholders

  • Address logistical concerns
  • Plan ahead
  • Provide written material well in advance
  • Clearly define roles, responsibilities and expectations in advance
  • Involve different types of stakeholders
  • Be open to alternative meeting formats


Image: a photograph of three professionally dressed people sitting at a conference table

Speaker notes: Once stakeholders have been identified it can be challenging to know the best approach for engaging them.  Particularly, a different approach may needed for patient, consumers and practicing physicians. (O’Haire, et al., 2011).
Some stakeholders may not be familiar with CER or other research processes, especially those not typically involved, such as consumers or policymakers. It will be helpful for these participants to have a general contextual understanding. However, it is not necessary for these stakeholders to become CER savvy or research conversant. The goal is to elicit and understand their perspectives, so that investigators can apply that information to the research process and protocol.

Possible strategies and tools for preparing stakeholders include:

  • Address logistical concerns. Since many stakeholders will not be able to participate as part of their job or other official duties, investigators need to allow for related logistical concerns, such as the need for advance-notice and scheduling, time of day, childcare or travel issues (if in person), and other considerations.
  • Plan the process, including notification, communication, facilitation techniques, and documentation, in advance.
  • Provide written materials, at basic levels of comprehension, in advance and in follow-up. Advance materials should be provided with sufficient time for stakeholders to ask clarifying questions.
  • Provide a clear and written, definition of roles and expectations related to involvement. Include what is not expected, as well as what is expected.
  • Consider involving multiple types of stakeholders, and appropriate processes for doing so. Individual interviews, focus groups, dedicated meetings, written testimonials, and other techniques, as well as involving stakeholders in diverse discussion groups may be used.

Slide 18

Challenge: Representativeness

  • No current requirement from AHRQ
  • Issues to consider:
    • One stakeholder is not representative
    • Different stakeholders of the same “type” have different perspectives
    • Identify explicit goals and objectives
    • Stakeholders with disparate views can result in balanced conversation


Image: a picture of a hand completing a puzzle

Speaker notes: There is currently no specific guidance on the issue of “representativeness.” Instead, the Effective Health Care Program acknowledges that each stakeholder has a valuable perspective; and the EHC Program is interested in the range of perspectives represented by a diversity of stakeholders. Working with one individual from a particular stakeholder group or type will only ensure that you have one perspective. One stakeholder will not be able to broadly represent the entire stakeholder group or type. Nor is she or he expected to do so.

In order to achieve a range of input, it is important to involve a diverse range of stakeholders. It is also important to clearly identify the goals and objectives of a particular research activity in order to determine what types of perspectives best match or address the goals.

It is additionally important to note that while representativeness is not likely to be achieved, selecting stakeholders with disparate views can help balance the discussions and increase the applicability of the final research product.

Slide 19

Challenge: Engaging & Preparing Stakeholders

Stakeholders are experts in:

  • Their own disease condition
  • Personal experience with disease or topic
  • Unique policy aspects of topic
  • Clinical concerns or applications


Image: there are three photographs overlapping each other. The bottom picture is of a businessman and his laptop, the second picture is of a healthcare professional, and the third is of a woman getting her blood pressure measured.

Speaker notes: AHRQ has high expectations for involving patients, consumers, frontline clinicians, and others who may not have traditionally been involved in research. Experience among investigators with engaging these types of stakeholders is varied. Some of the most important stakeholder engagement techniques may not involve applying specific skills, but rather shifting perspectives of investigators about “who” is the expert.

This shift also involves moving away from viewing stakeholder engagement as a “requirement” and toward stakeholder engagement as a step that will enable a better and more useable research product. It is also a shift toward viewing patients, consumers, and others as having important, expert knowledge.

Investigators with the most experience engaging a wide range of stakeholders often share the view that clinicians and investigators cannot know the perspective of living with a particular condition unless they actually experience it.Therefore, it is critical that investigators hear from actual patients and other decision-makers regarding the questions and outcomes of importance to them. 

Slide 20

Challenge: Managing Bias

  • Everyone is biased
  • Bias can represent:
    • Personal experience
    • Unique perspective
    • Specific context
  • Managing vs. eliminating bias:
    • Add more perspective to keep it balanced
    • Be proactive
    • Clearly define roles, responsibilities & expectations
    • Effective facilitation
    • Require disclosure
    • Document plan for managing bias


Image: a photograph of a businessman and woman in an arm wrestle

Speaker notes: All stakeholders have bias. The EHC Program is actively seeking to understand those biases in the form of stakeholder experience, perspective, and context. The role of the EHC Program investigator is to systematically review perspectives and biases to bolster research. Managing biases appropriately is important to keeping them from becoming disruptive conflicts during a meeting. This is similar to sorting through large amounts of written data to find appropriate, reliable, and scientifically sound evidence. Investigators also need to systematically sort through stakeholder voices to determine appropriate, relevant, and constructive perspectives. Stakeholder perspectives should be informative, not adjudicative to the research process.

Managing bias and conflicts of interest requires strategic handling on the part of investigators and AHRQ staff. Research institutions should consider establishing processes in advance to reduce potential for conflict and disruptive interactions during meetings. For example:

  • Ensure diverse involvement where all stakeholders have a “voice”
  • Establish written protocol regarding conflicts of interest
  • Be proactive in addressing conflicts of interest
  • Clearly define the roles, responsibilities, and expectations for each stakeholder in writing, and during meetings
  • Use effective facilitation to manage group process

Slide 21

Managing Bias & Conflicts

  • Distinguish biases vs. conflicts of interest vs. conflict
  • Establish and follow policies
  • Use facilitation techniques
  • Learn skills to manage bias
  • Keep it respectful


Speaker notes: All stakeholders have bias that the EHC Program seeks to understand as part of its systematic processes. It is important, however, to distinguish these biases from conflicts of interest, and also to learn the skill to manage biases when they results in conflicts during a meeting.

AHRQ has established policies regarding conflicts of interest. Investigators should follow these policies and make them available in writing to stakeholder participants.

Managing conflicts that arise during a meeting requires active and effective facilitation. Many potential conflicts can be anticipated and managed prior to a meeting. During a meeting, facilitators may choose a variety of techniques. If biases erupt into conflicts during a meeting investigators will have an opportunity to gather additional information about the topic that may be useful in informing the research. However, it is critical that participants remain respectful during any conflicts. The role of the facilitator in this setting is not to resolve conflicts, but to learn from them, keep stakeholders engaged, and ensure the process is respectful.

Slide 22

Challenge: Continuous Improvement

  • Review processes
  • Encourage feedback
  • Incorporate suggestions


Image: a picture of two arrows- one pointing up, and one pointing down, and three figures

Speaker notes: The level of and opportunities for stakeholder involvement in the EHC Program have continued to expand. AHRQ is interested in understanding and learning from the experience of investigators and research institutions. Eliciting feedback from stakeholders throughout their involvement is essential to achieving continuous improvement. Feedback topics should include:

  • Facilitator knowledge/ability
  • Meeting materials
  • Overall meeting effectiveness
  • What was effective and/or beneficial about the meeting
  • What was ineffective and/or frustrating about the meeting
  • Changes to improve the meeting’s effectiveness


Adapted from “Basic Facilitation Skills (2002) by The Human Leadership and Development Division of the American Society for Quality, The Association for Quality and Participation, The International Association of Facilitators.

Slide 23

Summary of Module IV

  • Engaging stakeholders in research is rewarding, yet challenging
  • Preparation and planning can help mitigate challenges
  • Be proactive and prepare stakeholders for their role
  • Facilitation is a process that requires specific skills


Speaker notes: Engaging stakeholders can be a rewarding, yet challenging and time consuming endeavor. Different stakeholders have different needs, expectations, and biases. They also have different perspectives and expertise that is critical to ensuring research is relevant and applicable. Advanced planning and preparation, clear roles and expectations, flexibility, and strong communication and facilitation skills are keys to ensuring efficient and effective stakeholder engagement.

Slide 24

Module IV Quiz

To help you review the information presented in Module IV, please consider completing the following quiz.

Slide 25

Module IV Quiz:

Question 1
One of the challenges to stakeholder engagement is engaging and preparing stakeholders for involvement in research. Of the options below, which activities can assist in this process?

  1. Providing contextual information before the first meeting
  2. Telling stakeholders what the research will say
  3. Clearly defining roles, responsibilities, and expectations
  4. Involving only one type of stakeholder
  5. Providing written materials in advance
  6. Having flexible meeting times that address logistical concerns

Slide 26

Module IV Quiz:

Question 1: ANSWER

One of the challenges to stakeholder engagement is engaging and preparing stakeholders for involvement in research. Of the options below, which activities can assist in this process?

  1. Providing contextual information before the first meeting
  2. Telling stakeholders what the research will say
  3. Clearly defining roles, responsibilities, and expectations
  4. Involving only one type of stakeholder
  5. Providing written materials in advance
  6. Having flexible meeting times that address logistical concerns

Slide 27

Module IV Quiz:

Question 2

Imagine you have found a stakeholder that is willing and able to be involved in research processes. One way to support their involvement is:

  1. Recognize stakeholders as experts in their own experience/perspective/condition
  2. Allow stakeholders to remain silent throughout the research process
  3. Always have stakeholders defer to clinical and methodological experts
  4. Only allow discussion and input to occur during scheduled meetings

Slide 28

Module IV Quiz:

Question 2: ANSWER

Imagine you have found a stakeholder that is willing and able to be involved in research processes. One way to support their involvement is:

  1. Recognize stakeholders as experts in their own experience/perspective/condition
  2. Allow stakeholders to remain silent throughout the research process
  3. Always have stakeholders defer to clinical and methodological experts
  4. Only allow discussion and input to occur during scheduled meetings

Slide 29

Module IV Quiz:

Question 3

Facilitation is:

  1. A one-time event during a meeting or discussion
  2. Focused on content and decisions
  3. A process focused on group dynamics and processes
  4. A superfluous skill

Slide 30

Module IV Quiz:

Question 3: ANSWER

Facilitation is:

  1. A one-time event during a meeting or discussion
  2. Focused on content and decisions
  3. A process focused on group dynamics and processes
  4. A superfluous skill

Slide 31

Module IV Quiz:

Question 4

Indicate which of the following skills are important for facilitation

  1. Being focused and alert
  2. Having little or no knowledge about the topic
  3. Focusing on content and decisions
  4. Being process oriented
  5. Being flexible and available

Slide 32

Module IV Quiz:

Question 4: ANSWER

Indicate which of the following skills are important for facilitation

  1. Being focused and alert
  2. Having little or no knowledge about the topic
  3. Focusing on content and decisions
  4. Being process oriented
  5. Being flexible and available

Slide 33

Works Cited
Stewart, J. (2006). High-performing (and threshold) competencies for group facilitators. Journal of Change Management, 6(4): 417-439.

The Human Leadership and Development Division of the American Society for Quality, The Association for Quality and Participation & The International Association of Facilitators (2002).Basic Facilitation Skills.