Powered by the Evidence-based Practice Centers
Evidence Reports All of EHC
Evidence Reports All of EHC



Critical Analysis of the Evidence for Patient Safety Practices

Research Protocol ARCHIVED Dec 6, 2011
Download PDF files for this report here.

Page Contents

Archived: The report for this protocol is greater than 3 years old. Findings may be used for research purposes, but should not be considered current.

Background and Objectives for the Systematic Review

The critical systematic review and analysis of Patient Safety Practices is an expansion of earlier evidence reports and current listing of Safe Practices for Better Healthcare 2010 Update by the National Quality Forum’s (NQF)1. The analysis will assess the evidence of the effectiveness of new safe practices and the adoption of scientific literature, other appropriate analyses, and extensive peer review of the draft report that have been developed but not included in the 2010 updates.

The final report of this project will be used by AHRQ for strategic planning in its patient safety portfolio for future project development, implementation of safe practices. The report will also be used by external organizations such and the NQF, Joint Commission and others in their patient safety efforts.

The key questions originally listed in the Request for Proposals were:

Design, Development and Testing of New PSPs

  • What new patient safety practices have been developed since 2001 and/or not included in the NQF safe practice list 2010?
  • What is the nature of the safe practice i.e. clinical, organizational, or behavioral?
  • What is the intended risk that the practice is designed to prevent or mitigate?
  • Describe how the practice is a bundle of individual components or practices, if applicable
  • What is the intended setting for the practice i.e. in patient, ambulatory, combination, specialty or clinical domain, and organizational setting?
  • What is the nature, quality and weight of evidence of the practice's effectiveness?

Implementation of the PSP

  • Was the safe practice implemented outside the developing institution?
  • What were the contextual settings in which it was implemented?
  • What were the issues, barriers, problems, successes and failures in the implementation of the practice?
  • What modifications and/or customizations were made (if any) in the implementation process?
  • What are the different implementation settings outside the developing institution have been reported for this practice?
  • Describe how the practice has been sustained in its use after initial implementation.
  • Was there any external support for the implementation process i.e. AHRQ technical support, used by a collaborative, or QIO


  • What is the extent to which the practice has been adopted by multiple institutions/organizations outside the developing institution?
  • Was there any organized activity or program to support the diffusion of the innovation/practice?
  • What if any evidence exists on the sustain use of the practice?
  • Has the practice become a requirement for use by any accreditation or credentialing agency organization?

The Key Questions

Considering the above goals for the project, the first step we needed to perform was to assess what PSPs to include in this review. We did topic refinement with our 21 member TEP to identify and prioritize potential PSPs for this review. The project team separated the PSPs into three categories – “includes”, “excludes” and “unsure” and used the TEP to affirm the “includes” and the “excludes” and then spent most of their time and effort on refining the “unsures” into includes and excludes. The initial list of PSPs started with the Making Health Care Safer list of 73. We added in a table of “new” PSPs from the proposal, also from NQF list, Joint Commissions list, and finally used the project team and AHRQ’s input to finalize the list.

Then we went through a 2 step process that included the teleconference which ultimately resulted in the rank order that we have.

  • Handoffs (Transitions in care)
  • Medication reconciliation (Transitions in care)
  • Rapid response team
  • In addition to fall prevention strategies, interventions to reduce the use of phys. restraints
  • Diagnostic errors – meta cognition, computerized decision support
  • Protocols for notification of test results to patients
  • Geriatric/delirium prevention
  • Monitoring for patient safety problems
  • Preventing ventilator-associated pneumonia – coordinate with AHRQ HAI report
  • Pressure ulcer prevention
  • Promoting a culture of safety
  • Universal protocol/preoperative checklist (surgical safety)
  • Report cards/outcomes measurements like NSQUIP/SCIP
  • Staffing patterns and ratios
  • Other interventions targeting improved transitions in care – (Transitions in care)
  • Glycemic control by impl evidence-based interv pract that prevent hypoglycemia
  • Use of preoperative anesthesia checklists (Complications due to anesthesia equipment failures)
  • Protocols for high risk drugs, e.g., nomograms for heparin
  • Interventions to prevent contrast-induced renal failure
  • The patient's role in preventing errors
  • CPOE and clinical decision support systems (CDSS)
  • Bundles and checklists as a general strategy (not just for specific indications)
  • Simulator-based training
  • prevention of surgical items left inside patient (surgical safety)
  • Medication administration
  • Human factors – as a general topic, focus still to be more precisely defined
  • Display systems
  • Hand washing + interventions to improve hand washing compliance
  • Perioperative beta- blockers
  • VTE prophylaxis and methods for implementation
  • Team training/team practices
  • Limiting individual provider’s hours of service (possibly fold into #14 Staffing)
  • Smart pumps and other protocols for infusion pumps
  • Device-related strategies for preventing tubing misconnections
  • Clinical pharmacist consultation services
  • Prevention of nosocomial UTIs
  • Use of real-time ultrasound guidance during central line insertion
  • Patient understanding/informed consent (possibly includes health literacy)
  • Interventions for central venous catheter-related blood infections
  • Patient death or serious injury associated with prolonged fluoroscopy with cumulative dose
  • Death among surgical patients with serious treatable complications (failure to rescue)
  • Barrier precautions, patient isolation, routine surveillance for patients at admission
  • Identifying patients at risk for suicide
  • “Sign your site” protocols - potentially part of checklists
  • Processes related to reprocessing single-use medical devices (HC assoc. infections)
  • Do not use abbreviations, acronyms, symbols, and dose designation campaign
  • Ensure documentation of patients' preferences for life-sustaining treatment
  • Strategies to prevent stress-related gastrointestinal bleeding

The specifications about the population, interventions and comparators will be specified for each included PSP as they are selected and further refined with the TEP.

As for specific outcomes assessed we will use Making Health Care Safer’s set of categories i.e., patient, clinical, and surrogate outcomes, process measures, other things related to safety and no safety measures.

Analytic Framework

The general analytic framework for this review is as follows:

the diagram is an analytical framework that depicts many factors can influence effectiveness of a patient safety practice. includes which points to practice and then onto beneficial outcomes. implementation theory or logic model arrow towards unexpected effects. general described further in section iii as its most simple applied problem order improve numerous have been some cases including contexts external management process itself. potential for may be needs considered.

In its most simple form, a patient safety practice is applied to a patient safety problem in order to improve beneficial patient safety outcomes. Numerous factors have been postulated, and in some cases shown, to influence the effectiveness of a patient safety practice, including contexts (such as safety culture, external factors and implementation/management tools), and the implementation process itself. Also, the potential for unexpected effects, which may be adverse, needs to be considered.

The analytic framework to illustrate the population, interventions, outcomes and adverse effects will be worked out for each PSP.


We will follow the methods guide2 with the following clarifications and additions:

  • In addition to standard computerized databases such as PubMed, we will search in places like PSNet and internet searches to try and identify relevant evidence
  • Non-English language studies will be excluded, and for PSP that have an expected cultural component, such as “promoting a culture of safety”, studies outside of the USA, or outside of USA, Canada, England and Australia/New Zealand may be excluded
  • Assessment of methodological quality will follow the criteria established in the prior PSP project Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices – Developing Criteria3
    1. Explicitly describe the theory behind the chosen intervention components or an explicit logic model for why this patient safety practice should work
    2. Describe the patient safety practice in sufficient detail that it can be replicated, including the expected effect on staff roles
    3. Measure high-priority contexts in the 4 domains
      1. External factors, such as regulatory requirements, public reporting or pay-for-performance, and local sentinel events
      2. Organization structural characteristics, such as size, complexity, and financial status or strength
      3. Teamwork, leadership, and patient safety culture
      4. Management tools, such as training resources, internal organization incentives, audit and feedback, and quality improvement consultants
    4. Detail the implementation process, the actual effects on staff roles, and how the implementation or intervention changed over time
    5. Assess the effect of the patient safety practice on outcomes and possible unexpected effects, including data on costs, when available
    6. For studies with multiple intervention sites, assess the influence of context on the effectiveness of intervention and implementation
  • Data synthesis will be narrative or meta-analytic as allowed for by the data in the original articles
  • We plan to divide the list of topics presented above into two groups: those for which providers and policymakers need an in-depth review, and those for which a “light review” will suffice, that will focus on just one or two aspects of the PSP.

Grading the Evidence for Each Key Question

We will develop this with the TEP as the project develops, but it will take into consideration a number of domains relevant to both internal and external validity, including study design, consistency of results, the size of the effect, the theory or logic model of the PSP, the adequacy of the description of context and the degree to which implementation and results are context-sensitive, details of the implementation process, and an adequate assessment of the potential for unexpected harms.


  1. National Quality Forum (NQF). Safe Practices for Better Healthcare - 2010 Update. Washington, DC: National Quality Forum; 2010
  2. Agency for Healthcare Research and Quality. Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews, Version 1.0 [Draft posted Oct. 2007]. Rockville, MD. Available at: /sites/default/files/pdf/cer-methods-guide_research-2008.pdf
  3. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy S, Foy R, Hempel S, McDonald, K, Ovretveit J, Rubenstein L, and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria. Contract Final Report. AHRQ Publication No. 11-0006-EF, December 2010. Prepared under Contract No. HHSA-290-2009-10001C. Agency for Healthcare Research and Quality, Rockville, MD.

Definition of Terms

Not applicable.

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

Technical Experts

Technical Experts comprise a multi-disciplinary group of clinical, content, and methodologic experts who provide input in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published three months after the publication of the Evidence report.

Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

Project Timeline

Critical Analysis of the Evidence for Patient Safety Practices

Nov 8, 2011
Topic Initiated
Dec 6, 2011
Research Protocol Archived
Page last reviewed December 2019
Page originally created November 2017

Internet Citation: Research Protocol: Critical Analysis of the Evidence for Patient Safety Practices. Content last reviewed December 2019. Effective Health Care Program, Agency for Healthcare Research and Quality, Rockville, MD.

Select to copy citation