Skip Navigation
Department of Health and Human Services www.hhs.gov
 
 

Slide 1

Engaging Stakeholders in the Effective Health Care Program
Information and tools for researchers and investigators

Speaker notes: The Agency for Healthcare Research and Quality presents the following e-learning tool “Engaging Stakeholders in the Effective Health Care Program” to provide researchers and investigators with foundational information on, and tools for, engaging a broad range of stakeholders in the Effective Health Care Program.  The information presented is adapted from content developed by the Scientific Resource Center at the Oregon Evidence-based Practice Center.

Slide 2

Full Course Outline

  • Module I: AHRQ’s Vision for, Theory, and Principles of Stakeholder Engagement
  • Module II: Stakeholders and Stakeholder Engagement
  • Module III: Points of Engagement in the EHC Program
  • Module IV: Skills for Successful Engagement

Speaker notes: This is the first of four modules as part of this e-learning tool.

Slide 3

Module I
AHRQ’s Vision for Stakeholder Engagement

Slide 4

What to Expect in Module I

Module I will cover the following topics:

  • AHRQ’s vision for stakeholder engagement in the EHC Program
  • The theory and evidence behind stakeholder engagement in comparative effectiveness research
  • The alignment of principles of comparative effectiveness research (CER) and stakeholder engagement

Speaker notes: AHRQ is continually evolving and expanding opportunities for stakeholder involvement to fulfill its vision for stakeholder engagement and knowledge exchange. AHRQ’s vision for stakeholder engagement is grounded in theory, principle, and practice. AHRQ’s vision aims to increase topic nominations, improve product development to meet end-user needs, and expand dissemination of research products. This module will provide information regarding AHRQ’s vision and expectations for, and theory and principles of stakeholder engagement in the EHC Program, as well as the alignment of these principles with the principles of comparative effectiveness research (CER).

Slide 5

Module I Learning Objectives

After completing Module I, participants will be able to:

  • Identify AHRQ’s vision for stakeholder engagement
  • Describe the theoretical basis for engaging stakeholders
  • List at least three benefits of engaging stakeholders in research
  • Identify three differences between knowledge transfer and knowledge exchange

Speaker notes: After completing Module I, participants will be able to:

  • Identify AHRQ’s vision for stakeholder engagement
  • Describe the theoretical basis for engaging stakeholders
  • List at least three benefits of engaging stakeholders in research
  • Identify three differences between knowledge transfer and knowledge exchange

Slide 6

Stakeholder Engagement in Research

“Awareness of the difference between the views of members of the public and those of researchers and professionals implies a pressing need to include the public in setting research agendas at the early stages in the research process . . . . Involving members of the public at this stage may be key to real empowerment as it is at these stages of research development that there is opportunity to influence.”
(Gooberman-Hill & Horwood, 2008)

Speaker notes: Traditionally, involving stakeholders in research has meant communicating questions or problems in the beginning phases and communicating results at the end of that research (Burger, Gochfeld & Pletnikoff, 2009). AHRQ, however, is committed to processes that engage stakeholders throughout the research process and at strategic points.

Slide 7

Defining  Stakeholder Engagement

Stakeholders are defined as individuals or organizations who have a personal or professional interest in the topic.

Stakeholder engagement is a process of giving voice to and involving stakeholders in EHC Program related decision making and research. It serves the purposes of reciprocal learning and knowledge exchange to improve health.

Speaker notes: Stakeholders are defined as individuals or organizations who have a personal or professional interest in the topic (O’Haire, 2011). Engagement in the EHC Program is a process of giving voice to and involving stakeholders in Program related decision making and research. It serves the purpose of reciprocal learning and knowledge exchange to improve health. This ultimately requires transitioning from knowledge transfer to knowledge exchange in order to develop more interactive and in depth relationships. The concepts of knowledge transfer and knowledge exchange will be discussed later in this module.

Slide 8

AHRQ’s Vision for Stakeholder Engagement

“The Agency for Healthcare Research and Quality (AHRQ) firmly believes that involving all stakeholders in the research enterprise from the beginning improves the end product and facilitates the diffusion and implementation of the findings by getting early buy in from users. Involving all stakeholders also helps to ensure that the research reflects the various needs of all diverse users.”
(Agency for Healthcare Research and Quality, 2009)

Speaker notes: The initial legislation creating the Effective Health Care Program (Section 1013, Medicare Prescription Drug, Improvement, and Modernization Act of 2003) outlined the need for broad stakeholder involvement. While stakeholders have been involved in multiple points of EHC Program research, funding from the American Reinvestment and Recovery Act (ARRA) of 2009 allowed for an expansion of EHC Program activities, increasing opportunities for stakeholder involvement.

AHRQ has held the involvement of stakeholders as a central element of the EHC Program since its inception, and has clearly outlined the continued expectation for broad stakeholder engagement in its spend plan for ARRA funding. The EHC Program includes as stakeholders those individuals and organizations external to AHRQ and its network of research institutions; those individuals and organizations with an interest in applying the findings of the research completed under the Program. There is a particular interest by AHRQ for the research to inform health care decisions by stakeholders, be they consumers, clinicians, or policymakers.

Slide 9

AHRQ’s Vision for Stakeholder Engagement

  • Stakeholders are involved throughout the research process
  • Create strategic involvement and partnerships with a broad and inclusive range of stakeholders
  • Collaborative relationships should inform all phases of research
  • Stakeholder input maximize usefulness of EHC Program products


Speaker notes: AHRQ’s vision for stakeholder engagement has continued to evolve and expand, and aims to involve stakeholders throughout the research process. AHRQ seeks to create strategic points of involvement that are mutually beneficial to stakeholders and research institutions, and ultimately serve the goals of the Program. AHRQ’s vision of stakeholder involvement includes research institutions developing ongoing relationships with stakeholders—particularly patients and their families or caregivers, and frontline clinicians. It is intended that these collaborative relationships will inform topic nominations, topic development, refinement and review, and ultimately product dissemination. AHRQ intends for these partnerships to be inclusive and broad, and to support the development of quality EHC Program research products and dissemination activities.

Slide 10

Theory of Stakeholder Engagement

Evidence suggests that engaging stakeholders in research increases:

  • Relevance & context
  • Knowledge exchange
  • Application of evidence
  • Dissemination avenues


Speaker notes: A primary goal of the Effective Health Care Program is to make evidence accessible and actionable for all types of health care decision-makers. One approach to increasing the uptake and utilization of evidence by decision-makers is to engage these stakeholders early and often in the research process.

Engaging stakeholders early and throughout the research process increases the relevance of research products (Innvaer, Vist, Trommald & Oxman, 2002), knowledge exchange between investigators and stakeholders (Graham, 2006), application of evidence (Graham, 2006), real-world context (Keown et al., 2008; Whitlock et al., 2009), and dissemination avenues (Keown et al., 2008). Ultimately, AHRQ hopes that engaging stakeholders in the research process can lead to improved research products and greater utilization of evidence in subsequent decision making.

Slide 11

Theory of Stakeholder Engagement: Relevance & Context

Studies indicate that involving stakeholders early on leads to research products that feature:

  • Relevant information
  • Applicable evidence
  • Real-world context
  • Information to address existing stakeholder concerns

The question is written in a separate dialogue box

What are the best oral medicines to treat for type 2 diabetes?

    • Topic submitted by the American College of Physicians, a professional association representing 132,000 medical practitioners across the nation


Speaker notes: Early engagement of stakeholders and the opening of communication between research investigators and their audiences increases the likelihood that end products will be relevant and useful to decision-makers at the point of care (Keown, et al., 2008; Graham, et al., 2006) and provides real-world context for the application of evidence (Keown, et al., 2008; Whitlock, et al., 2009)

Investigators want their work and resulting products to be utilized, but they sometimes lack information regarding the context and subtleties facing decision-makers. Similarly, decision-makers need to know which research products are applicable to the dilemmas they face, but they sometimes lack knowledge about which questions the research addresses, which research is reliable, and how to incorporate research findings into decision making.

Slide 12

Theory of Stakeholder Engagement:
Knowledge Exchange

Involving stakeholders in the research process facilitates:

  • Mutual knowledge exchange
  • Exchange of perspectives
  • Collaborative learning

Each of these bullets is written in a circle. The three circles interlock with one another to show that they overlap.
Speaker notes: The Canadian Health Services Research Foundation describes knowledge exchanges as “collaborative problem-solving between researchers and decision makers,” which results in interactions that produce mutual learning “through the process of planning, disseminating, and applying existing or new research in decision making.”

The theory of knowledge exchange posits that investigators and decision-makers are normally separate groups with distinct cultures and perspectives on research and knowledge, with neither group fully understanding the other’s world. Exchange of information, rather than one-way communication between investigators and stakeholders is a collaborative process that can be new to both. Involving stakeholder decision-makers in research processes allows for the exchange necessary to provide “real world” context to investigators while providing valuable information about the research process and how to utilize end products to decision-makers (Keown, et al., 2008). This ongoing exchange of information and knowledge between investigators and stakeholders ensures that the products generated are relevant and applicable to stakeholder decision making as well as useful to investigators (Graham, et al., 2006).

The process of knowledge exchange and eliciting stakeholder input does not mean that stakeholders dictate what a final research product will say. Instead, it allows an opportunity to identify questions and outcomes that are important to end-users. The end product should provide answers to questions that matter to stakeholders. Even the answer “there was not sufficient evidence” can be useful.

Slide 13

Knowledge Transfer vs. Knowledge Exchange

Knowledge Transfer

One-way

Communicates intent and/or results

Single or limited opportunities

Before or after

Linear

Prescribed

Individual benefit/expectation

Knowledge Exchange

Two-way

Engages in dialogue, questions & exploration

Multiple opportunities

Before, during & after

Iterative

Organic

Mutual benefit/expectation

Speaker notes: The process of knowledge exchange can be very different from knowledge transfer. Some of these differences include:

  • One way versus two way communication
  • Communicating information versus discussing and exploring topics
  • Single point of communication, after research versus multiple points of communication throughout the research process
  • Linear versus iterative processes
  • Prescribed or preset process versus organic processes
  • Individual versus mutual benefit

Slide 14

Engagement Exercise 1: Stakeholder Engagement Techniques

  • Think about which of the following stakeholder involvement techniques have been used by you or your research team:
  • Discussing/identifying potential research topics with stakeholders
  • Attending meetings of stakeholder groups/organizations
  • Presenting information or findings at meetings of stakeholder organizations
  • Asking clinical experts to help develop or clarify a research suggestion
  • Involving stakeholders in developing research questions
  • Involving patients & consumers or frontline clinicians in topic refinement activities
  • Posting draft documents for review and comment
  • Publishing findings in journals, newsletter, etc.


Speaker notes: Think about which of the following stakeholder involvement techniques have been used by you or your research team:

  • Discussing/identifying potential research topics with stakeholders
  • Attending meetings of stakeholder groups/organizations
  • Presenting information or findings at meetings of stakeholder groups/organizations
  • Asking clinical experts to help develop or clarify a research suggestion
  • Involving stakeholders in developing research questions
  • Involving consumers or frontline clinicians in topic refinement activities
  • Hosting discussion or focus groups of consumers or frontline clinicians
  • Posting draft documents for review and comment
  • Publishing findings in journals, newsletter, etc.
  • Being “commissioned” by a specialty society for a specific report
  • Other

Slide 15

Engagement Exercise 2: Knowledge Transfer vs. Knowledge Exchange
The following slide provides examples of stakeholder engagement techniques. For each example, please consider if the activity is a form of knowledge transfer or knowledge exchange.

Slide 16

Engagement Exercise 2: Knowledge Transfer vs. Knowledge Exchange

  • Discussing/soliciting potential research questions with stakeholders
  • Attending meetings of stakeholder groups/organizations
  • Presenting information or findings at meetings of stakeholder groups/organizations
  • Involving a nominator in topic refinement activities
  • Posting a draft document for review & comment
  • Publishing findings in journals, newsletters, etc.

Slide 17

Engagement Exercise 2: Answers Knowledge Transfer vs. Knowledge Exchange
Knowledge Exchange

  • Discussing/soliciting potential research questions with stakeholders
  • Involving a nominator in topic refinement activities
  • Posting a draft document for review & comment


Knowledge Transfer

  • Attending meetings of stakeholder groups/organizations
  • Presenting information or findings at meetings of stakeholder groups/organizations
  • Publishing findings in journals, newsletters, etc.

Slide 18

Theory of Stakeholder Engagement:
Application of Evidence

Involving stakeholders in the research process leads to applicable research products that:

  • Address specific questions with appropriate context
  • Are useable and actionable
  • Incorporate wider contextual issues
  • Are accepted and valued by end-users

Speaker notes: “Bringing decision makers who can use the results of a particular piece of research into its formulation and conduct is the best predictor for seeing the findings applied” (Lomas, 2000). The process of applying knowledge to action is complex and challenging. It is not linear, but is instead iterative and organic. It involves a regular exchange of knowledge between researchers and relevant decision-makers, or stakeholders. And, it increases the application of evidence to practice, policy and decision making (Graham et al., 2006).

As Saunders et al. (2007) concluded, “Actively seeking the views of consumers regarding health research is now recognized as an opportunity to access valuable untapped knowledge, and a means of improving the relevance and translation of research into practice… Consumer perspectives trigger consideration for the wider environmental, psychosocial, and behavioral contexts of people’s lives. Further reported benefits include a greater acceptance and uptake of research findings in the community, and more efficient use of research resources.”

Further, as Whitlock et al. (2009) assert, “Engaging stakeholders as key informants provides credibility and avoids prioritizing topics that have no relevance to real-world issues.” Research that is valued and accepted by stakeholders is much more likely to be utilized and disseminated.

Slide 19

Theory of Stakeholder Engagement:
Dissemination Avenues

Involving stakeholders in the research process can lead to dissemination opportunities through:

  • Advocacy organizations
  • Professional societies
  • Family and friends
  • Clinicians
  • Health care systems
  • Payers
  • Public programs


Image: photograph of a sign with the words “The Beginning” on it)

Speaker notes: Engaging stakeholders creates dissemination avenues for completed products (Keown, et al., 2008). Every stakeholder that is involved in the research process provides a potential opportunity for dissemination. Stakeholders may open dissemination paths through advocacy organizations or professional societies of which they are members, family and friends, their own health care providers or co-workers, health care systems, payer networks, or public programs. Each stakeholder can represent multiple avenues for dissemination, and the more invested they are in the research process and resulting products, the more likely they will be to help disseminate final products and findings.

Slide 20

Engagement Exercise 3:

The following slide provide examples of engagement activities. Please consider the possible resulting benefit(s) for each example.

Slide 21

Engagement Exercise 3:
Please consider the possible resulting benefit(s) of stakeholder engagement:

Discussing/soliciting potential research ideas from stakeholders.

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 22

Engagement Exercise 3: Benefits of Stakeholder Engagement
Answer: A, B, C & D

Discussing/soliciting potential research ideas from stakeholders.

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 23

Engagement Exercise 3:
Please consider the possible resulting benefit(s) of stakeholder engagement:

Attending meetings of stakeholder groups/ organizations

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 24

Engagement Exercise 3:
Answer: A & D

Attending meetings of stakeholder groups/ organizations :

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 25

Engagement Exercise 3:
Please consider the possible resulting benefit(s) of stakeholder engagement:

Presenting information of research findings at meetings of stakeholder groups/organizations

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 26

Engagement Exercise 3:
Answer: C & D

Presenting information of research findings at meetings of stakeholder groups/organizations

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 27

Engagement Exercise 3:
Please consider the possible resulting benefit(s) of stakeholder engagement:

Involving stakeholders in topic refinement activities

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 28

Engagement Exercise 3:
Answer: A, B, C & D

Involving stakeholders in topic refinement activities

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 29

Engagement Exercise 3:
Please consider the possible resulting benefit(s):

Posting draft documents for review & comments

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 30

Engagement Exercise 3:
Answer: A, B, C & D

Posting draft documents for review & comments

    • Increased relevance & context
    • Knowledge exchange
    • Increased application of evidence
    • Increased dissemination avenues

Slide 31

EHC Program Principles
Key principles identified by the EHC Program to guide research activities include:

  • Research must be relevant and timely to meet the needs of decision-makers
  • Public participation and transparency increase public confidence in the scientific integrity, credibility of research processes, and provide further accountability to researchers (Slutsky, et al., 2008)


Speaker notes: The Effective Heath Care Program has established a series of methodological principles for conducting comparative effectiveness research and reviews. Stakeholder engagement is a guiding principle, along with transparency and scientific integrity. These principles can be found in the EHC Program Methods Guide, and have also been written about or referenced in other publications, such as the Journal of Clinical Epidemiology and several Institute of Medicine (IOM) reports.

Slide 32

Patient-Centered Approach to CER

  • Central to evidence-based medicine
  • Research should address patient & consumer concerns
  • Requires different techniques


Image:  3 photographs of different types of patients, overlapping one another

Speaker notes: A patient-centered approach to CER is central to evidence-based medicine and may require different techniques to address consumer concerns. “Clinicians must access the values and preferences of the patient and then place these in the context of the evidence about the benefits and risks of alternative courses of action” (Guyatt, 2002).

Patients and their clinicians are concerned about outcomes. However, the outcomes most important to each may be different. For example, in a CER focused on lipid lowering agents, a physician may be most focused on the outcome of clinical tests. A patient, on the other hand, might be most worried about side-effect profiles and frequency of dosage. “These principles mean that, regardless of who nominates a topic and who might use a CER, the research & reviews should address the circumstances and outcomes that are important to patients and consumers” (Helfand & Balshem, 2009).

Slide 33

AHRQ & Patient-Centered Approach to CER
One of the overarching principles of the EHC Program is to conduct CERs by approaching the evidence by a clinical, patient-centered perspective.
(Helfand & Balshem ,2009; Slutsky, et. al., 2008)

Speaker notes: One of the overarching principles of the Effective Health Care Program is to conduct CERs by approaching the evidence by a clinical, patient-centered perspective. According to Helfand and Balshem (2009) “CERs…seek to provide decision-makers with accurate, independent, scientifically rigorous information for comparing the effectiveness and safety of alternative clinical options, and have become the foundation for decision-making in clinical practice and health policy. To play this important role in decision making, CERs must address significant questions that are relevant to patients and clinicians….”

There are many ways investigators can demonstrate that their work is focused on patients and consumers, including having consumers initiate research through topic nomination, provide input into the formation of research questions, and assist in translation of final reports.

Slide 34

Engagement Exercise 4: Patient-Centered Research

There are many ways researchers can demonstrate that their work is patient-centered. Please think of three examples of how your work has been patient-centered.

Speaker notes: There are many ways researchers can demonstrate that their work is consumer-centered. Please list three examples of how your work has been patient-centered.

Slide 35

Examples of Patient-Centered Approaches

  • Focusing reports on outcomes and measures important to patients and their clinicians
  • Documenting and making publicly available the processes used to develop a report
  • Providing public access and opportunity to comment on draft documents


Speaker notes: There are many examples of patient-centered approaches in the EHC Program. Some of these examples are listed here.

Slide 36

Examples of Patient-Centered Approaches (Cont)

  • Involving patients and their clinicians in:
    • Generating and prioritizing research topics and questions
    • Identifying and refining key questions
    • Planning, developing and updating reviews and protocols/reports
    • Consulting during report development
    • Contributing to other dissemination & translational activities
  • Collecting patient “testimonials” that illustrate the context and circumstances of patients


Speaker notes: There are many examples of patient-centered approaches in the EHC Program. Some of these examples are listed here.

Slide 37

Summary of Module I

  • AHRQ’s vision for stakeholder engagement is inclusive, broad, and evolving
  • AHRQ’s expectation for stakeholder engagement is outlined in the initial legislation and current ARRA spend plan
  • Evidence suggests there are many benefits to engaging stakeholders in research
  • Principles of comparative effectiveness research (CER) and stakeholder engagement align closely


Speaker notes: AHRQ is continually evolving and expanding opportunities for stakeholder involvement to fulfill its ultimate vision for inclusive and broad stakeholder engagement and knowledge exchange. Further, AHRQ's expectation for stakeholder engagement is outlined in the founding legislation and current ARRA spend plan. AHRQ’s vision and expectation for stakeholder engagement are well grounded in the literature, and aim to increase topic nominations, improve product development to meet end-user needs, and expand dissemination of research products. AHRQ expects that strategic involvement of stakeholders throughout the EHC Program research process will be mutually beneficial to investigators, stakeholders, and the Program as the principles of engagement and CER are closely aligned. AHRQ will continue to seek opportunities to increase involvement and improve Program outputs.

Slide 38

Module I Quiz

To help you review the information presented in Module I, please complete the following quiz. Answers are provided at the end of the module.

Slide 39

Module I Quiz:  
Question 1

Please select the statement that most closely represents AHRQ's vision for stakeholder engagement in the EHC Program:

  1. Stakeholders should only be involved in the final stages of research
  2. Stakeholders should be strategically involved throughout the research process
  3. Stakeholder should be silent partners
  4. Stakeholders impede the research process, but have to be included according to Congress

Slide 40

Module I Quiz:
Question 1: ANSWER

Please select the statement that most closely represents AHRQ's vision for stakeholder engagement in the EHC Program:

  1. Stakeholders should only be involved in the final stages of research
  2. Stakeholders should be strategically involved throughout the research process
  3. Stakeholder should be silent partners
  4. Stakeholders impede the research process, but have to be included according to Congress

Slide 41

Module I Quiz:
Question 2

Which of the following does the evidence suggest are potential benefits of stakeholder involvement in research?

  1. Increased application of evidence
  2. Lower reading level of research reports
  3. Increased relevancy of research products
  4. Increased dissemination avenues
  5. Increased number of editors to review reports

Slide 42

Module I Quiz:
Question 2: ANSWER

Which of the following does the evidence suggest are potential benefits of stakeholder involvement in research?

  1. Increased application of evidence
  2. Lower reading level of research reports
  3. Increased relevancy of research products
  4. Increased dissemination avenues
  5. Increased number of editors to review reports

Slide 43

Module I Quiz:
Question 3

One of the core principles of comparative effectiveness research and evidence-based medicine that closely aligns with principles of stakeholder engagement is:

  1. Standardize methodology
  2. Peer review process
  3. Patient-centered approach

Slide 44

Module I Quiz:
Question 3: ANSWER

One of the core principles of comparative effectiveness research and evidence-based medicine that closely aligns with principles of stakeholder engagement

  1. Standardize methodology
  2. Peer review process
  3. Patient-centered approach

Slide 45

Works Cited

Agency for Healthcare Research and Quality (2009). Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD. Available at: http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productid=318.

Agency for Healthcare Research and Quality (2009). Operating Plan for American Recovery and Reinvestment Act Funds for Comparative Effectiveness Research as the Agency for Healthcare Research and Quality.

Burger, J., Gochfeld, M. & Pletnikoff, K. (2009). Collaboration versus communication: The Department of Energy’s Amchitka Island and the Aleut Community. Environmental Research, 109: 503-510.

Canadian Health Services Research Foundation (no date). Knowledge Exchange. Retrieved from http://www.chsrf.ca/knowledge_transfer/index_e.php.

Gooberman-Hill, R. & Hoorwood, J. (2008). Citizens’ juries in planning research priorities: process engagement and outcome. Health Expectations, 11:272-281.

Slide 46

Works Cited

Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W. & Robinson, N. (2006). Lost in Knowledge Translation: Time for a Map? The Journal of Continuing Education in the Health Professions, 26: 13-24.

Guyatt, G. & Rennie, D.  (Eds.). (2002). Users’ guides to the medical literature: A manual for evidence-based clinical practice. Chicago, IL: AMA Press.

Helfand, M. & Balshem, H. (2009). Principles for developing guidance: AHRQ and the effective health care program. Journal of Clinical Epidemiology. Article in press.

Innvaer, S., Vist, G., Trommald, M. & Oxman, A. (2002). Health policy-makers’ perceptions of their use of evidence: a systematic review. Journal of Health Services Research & Policy, 7(4): 239-44.

Keown, K., Van Eerd, D. & Irvin, E. (2008). Stakeholder Engagement Opportunities in Systematic Reviews: Knowledge Transfer for Policy and Practice. Journal of Continuing Education in the Health Professions, 28(2): 67-72.

Lomas, J. (2000). Using ‘Linkage and Exchange’ to Move Research into Policy at a Canadian Foundation. Health Affairs, 19(3): 236-240.

Slide 47

Works Cited

Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W. & Robinson, N. (2006). Lost in Knowledge Translation: Time for a Map? The Journal of Continuing Education in the Health Professions, 26: 13-24.

Guyatt, G. & Rennie, D.  (Eds.). (2002). Users’ guides to the medical literature: A manual for evidence-based clinical practice. Chicago, IL: AMA Press.

Helfand, M. & Balshem, H. (2009). Principles for developing guidance: AHRQ and the effective health care program. Journal of Clinical Epidemiology. Article in press.

Innvaer, S., Vist, G., Trommald, M. & Oxman, A. (2002). Health policy-makers’ perceptions of their use of evidence: a systematic review. Journal of Health Services Research & Policy, 7(4): 239-44.

Keown, K., Van Eerd, D. & Irvin, E. (2008). Stakeholder Engagement Opportunities in Systematic Reviews: Knowledge Transfer for Policy and Practice. Journal of Continuing Education in the Health Professions, 28(2): 67-72.

Lomas, J. (2000). Using ‘Linkage and Exchange’ to Move Research into Policy at a Canadian Foundation. Health Affairs, 19(3): 236-240.