Skip to main content
Effective Health Care Program
Home » Products » Diagnostic Errors in the Emergency Department » Diagnostic Errors in the Emergency Department: A Systematic Review

Diagnostic Errors in the Emergency Department: A Systematic Review

Research Protocol

I. Background and Objectives for the Systematic Review

The National Academy of Medicine (NAM) has called diagnostic error a "blind spot" for modern medicine and improving diagnosis a "moral, professional, and public health imperative."1 The emergency department (ED) is a known high-risk site for diagnostic error.2-7  The key decisional dilemma for this evidence review is "What are the most common and significant medical diagnostic errors in the ED, and why do they happen?" The goal is to determine the following: (1) What are the major clinical conditions associated with diagnostic errors and misdiagnosis-related harms in the ED, particularly serious misdiagnosis-related harms (death or permanent disability)?; (2) How common are these diagnostic errors and any associated harms?; (3) What are the key causes for errors and harms, and are there commonalities across clinical conditions?

II. The Key Questions

Key Question 1: What clinical conditions are associated with the greatest number and highest risk of ED diagnostic errors and associated harms?

  1. What diseases or syndromes are associated with the greatest total number and the highest risk/probability of diagnostic errors or misdiagnosis-related harms?
  2. Do results vary based on the severity of any resulting misdiagnosis-related harms (e.g., death or permanent disability, as opposed to less serious harms)? 
  3. What are the most common clinical presenting symptoms or signs associated with diagnostic errors or misdiagnosis-related harms in the ED?
  4. Do the most common clinical presenting symptoms or signs associated with diagnostic error or misdiagnosis-related harms vary by disease or syndrome? 

Key Question 2: Overall and for the clinical conditions identified from KQ1, how frequent are ED diagnostic errors and associated harms?

  1. On a per-visit or symptom-specific basis, what is the rate of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms? 
  2. On a per-disease/syndrome basis, what is the rate of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms? 
  3. Approximately how many patients does this equate to nationally in the US? 
  4. Are there clear commonalities or differences across clinical conditions in the frequency or risk of ED diagnostic errors or misdiagnosis-related harms? 

Key Question 3: Overall and for the clinical conditions identified from KQ1, what are the major causal factors associated with ED diagnostic errors and associated harms?

  1. What are the most frequent causes identified? 
  2. Do causes identified differ based on severity of harms?
  3. Do different causes have differential impact on patient outcomes (i.e., harms)? 
  4. Overall and for each clinical condition: 
    1. Are patient characteristics associated with errors/harms? (e.g., age, gender, race/ethnicity, language, socioeconomic status/income, health literacy) 
    2. Are illness characteristics associated with errors/harms? (e.g., symptom type, clinical presentation, mode of arrival)
    3. Are clinician characteristics associated with errors/harms? (e.g., provider type, training background, experience level, prior disciplinary action)
    4. Are facility or health system characteristics associated with errors/harms? (e.g., region, ED patient volumes or discharge fraction, teaching status, access to imaging, access to or type of electronic health record system)?
    5. Are context-specific systems factors associated with errors/harms? (e.g., at the time of the error—high ED patient volume or severity of illness, night or weekend shift, provider fatigue, change of shift/handoff) 
  5. Are there significant commonalities or differences among causes of ED diagnostic errors or associated harms across clinical conditions? 

Prospectively analyzed subgroups and covariates as appropriate to the individual Key Questions (and data permitting), will include the following:

  1. Studies conducted in United States vs. non-United States
  2. Children (<18yo) vs. adults (≥18yo); adults <65yo vs. adults ≥65yo (KQ1 only)
  3. General vs. specialty EDs (e.g., psychiatric, eye and ear) (KQ1 only)
  4. ED discharges vs. admissions vs. transfers (KQ1 only)
  5. Epoch in which studies were reported (2000-2010 vs. 2011-2021) (KQ2 only)
  6. ED provider training: physicians vs. advanced practice providers; physicians who are trained vs. not trained in emergency medicine; trainees (residents) vs. fully-trained physicians, and years of experience (KQ2 only)

Methodological study design will be captured, as appropriate, including the following:

  1. How were diagnostic errors and harms defined, identified, and categorized? 
  2. How were clinical conditions (diseases or syndromes) defined and grouped? 
  3. What methods were used to investigate and categorize causes? 

Based on prior research and discussions with Key Informants, we anticipate the following clinical conditions to top the list of most frequent causes of harm: vascular events (stroke, myocardial infarction, venous thromboembolism, aortic aneurysm and dissection, arterial thromboembolism), infections (sepsis, meningitis and encephalitis, spinal abscess, pneumonia, endocarditis, and appendicitis), and selected fractures.2,3,8-10 Additional conditions that are likely relevant to pediatric populations include testicular torsion, necrotizing enterocolitis, and sudden cardiac death/arrythmias/congenital heart disease.6,11 Additional conditions that are likely relevant to pregnant populations are ectopic pregnancy and preeclampsia/eclampsia.2,12,13 

PerSPEcTiF Framework

Perspective

  • From the perspective of the health system

Setting

  • In the ED

Phenomenon

  • Diagnostic error / misdiagnosis-related harms

Environment

  • Within the environment of hospital and health systems based in the United States, Canada, United Kingdom, Western Europe, Australia, New Zealand.
    • We selected these countries because they have advanced medical care systems that are similar to the United States.

Timing

  • We will include studies conducted from 2000 to present. Where possible, we will compare results of studies conducted between 2000-2010 with those conducted 2011-2021.
    • We selected a date limit of 2000 because new imaging technology has significantly improved diagnostic capabilities in the ED.

Findings

  • List of key clinical conditions accounting for the majority of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms
  • Frequency of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms overall and for key clinical conditions
  • Patient, clinician, facility and health system characteristics, or context-specific systems factors that are associated with diagnostic errors/harms

III. Analytic Framework

Figure 1: Draft Analytic Framework

Figure 1 displays our analytic framework. Patients present to the emergency department and they receive a diagnosis. There are several factors that influence the diagnosis, including clinical conditions, patient characteristics, illness characteristics, clinician characteristics, facility or health system characterisitcs, and context-specific characteristics. The diagnosis is either correct or not correct. Misdiagnoses can lead to either serious harms (e.g., death or permanent disability) or to lesser harms.

IV. Methods

Criteria for Inclusion/Exclusion of Studies in the Review—The inclusion and exclusion criteria are listed in Table 1.

Table 1. Inclusion and exclusion criteria

PerSPEcTiF Inclusion criteria Exclusion criteria
Perspective N/A N/A
Population
  • Populations with a condition with the greatest number and highest risk of misdiagnosis in the ED (KQ2, KQ3)
N/A
Setting
  • Studies conducted in the ED or studies that have a reasonable prospect of including data about ED physician or APP diagnostic accuracy
  • No reasonable prospect that the study includes data about ED physician or APP diagnostic accuracy (e.g., about pre-hospital accuracy, ED resident training or education, reliability study of a diagnostic screening tool for a specific disease)
Phenomenon
  • Diagnostic error*
  • Misdiagnosis-related harms
  • Serious misdiagnosis-related harms (death or permanent disability [NAIC scale 6-9])
N/A
Environment
  • US, Canada, UK, Western Europe, Australia, New Zealand
  • Studies conducted outside these countries
Timing
  • At least 50% of the patients were seen in the year 2000 or later
  • More than 50% of the patients were seen prior to the year 2000
Findings
  • List of key clinical conditions accounting for the majority of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms (KQ1)
  • Frequency of diagnostic errors, misdiagnosis-related harms, and serious misdiagnosis-related harms overall and for key clinical conditions (KQ2)
  • Patient, illness, clinician, facility and health system characteristics, or context-specific systems factors that are associated with diagnostic errors/harms (KQ3)
N/A
Study
  • We will include studies regardless of language
  • We will include published, peer-reviewed studies with original data and ≥50 ED patients studied
  • We will include relevant reports from major medical liability insurance carriers or similar risk management entities, even if these are not peer-reviewed publications
  • We will exclude case reports or small case series with fewer than 50 ED patients
  • We will exclude studies with no original data (e.g., reviews, simulation studies)
  • We will exclude studies using qualitative research methods that do not rely on specific patient data (e.g., physician focus groups about the general causes of diagnostic error)
  • We will exclude meeting abstracts

Abbreviations: APP = advanced practice provider (e.g., advanced practice nurse or physician’s assistant); ED= Emergency department; KQ = key question; NAIC = National Association of Insurance Commissioners
Notes:
*The National Academy of Medicine defines diagnostic error as "the failure to (a) establish an accurate and timely explanation of the patient's health problem(s) or (b) communicate that explanation to the patient."1 We will use this definition for diagnostic error, while recognizing that many studies will only address either accuracy or timeliness, not effectiveness of communication with patients. We also note that we will use the term "misdiagnosis" in this review as synonymous with "diagnostic error" (though we recognize not all authors or studies do the same).
†Misdiagnosis-related harms are defined as harms resulting from the delay or failure to treat a condition actually present, when the working diagnosis was wrong or unknown (delayed or missed diagnosis [false negative]), or from treatment provided for a condition not actually present (wrong diagnosis [false positive]).3,8,14

 

Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions—We will search the following databases for primary studies: MEDLINE®, Embase™, Cumulative Index to Nursing and Allied Health Literature (CINAHL®). We will develop a search strategy for MEDLINE, accessed via PubMed®, based on an analysis of medical subject headings (MeSH®) and text words from eligible studies identified a priori. Our search strategy is presented in Appendix A. Our search will be peer-reviewed by a medical librarian with experience in developing literature searches in the field of diagnostic errors. To supplement the electronic searching, we will use a variety of forward and backward searching techniques. This will include hand searching the reference lists of included articles and relevant reviews. The backward searching will be limited to studies published in 2000 or later. We will use tools, such as the "Similar Articles" or "Cited By" features in PubMed, Web of Science, or Google Scholar for the forward searching. We will update the search during public comment.

We will conduct grey literature searches to identify data and studies not reported in the published literature, to assess for publication and reporting bias, and to inform future research needs. Studies identified through grey literature searches will be considered for inclusion into the review under two conditions: (1) if they are a source of a unique study that meets inclusion criteria and provides enough methodologic detail to assess risk of bias or (2) if they can be matched to an original publication that has been included into the review when the presentation reports data on an outcome that was not reported in the original publication. We will review malpractice claims reports from major medical liability insurance carriers or similar risk management entities. We will review any material that is submitted through the Supplemental Evidence and Data for Systematic Reviews (SEADS) portal.

Two independent reviewers will screen each abstract. Both reviewers will need to agree that the article meets at least one of the exclusion criteria to be excluded (see Table 1 for the list of inclusion/exclusion criteria). We will track and resolve differences between reviewers regarding abstract inclusion or exclusion through consensus adjudication. We use DistillerSR database (Evidence Partners Inc., Ottawa, Canada) to conduct and manage screening.

Articles promoted on the basis of the abstract screen will undergo another independent screen by two reviewers using the full-text article. We will track and resolve differences between reviewers regarding article inclusion or exclusion through consensus adjudication.

We anticipate that the conditions with the greatest number of diagnostic errors and misdiagnosis-related harms in the ED will be: stroke, myocardial infarction, venous thromboembolism, aortic aneurysm and dissection, arterial thromboembolism, sepsis, meningitis and encephalitis, spinal abscess, pneumonia, endocarditis, appendicitis, and selected fractures. Additional conditions that are likely relevant to pediatric populations include testicular torsion, necrotizing enterocolitis, and sudden cardiac death/arrythmias/congenital heart disease. Additional conditions that are likely relevant to pregnant populations are ectopic pregnancy and preeclampsia/eclampsia. While screening the full-text articles, we will exclude studies that do not include populations with at least one of these conditions. We will not exclude studies based on condition when screening abstracts.

If studies are conducted in the US, Canada, UK, Western Europe, Australia, or New Zealand and also in another country(ies) and the results are not separated by country, we will contact the authors. We will exclude the study if we receive no response from the authors. We will consider Western European countries to be Austria, Belgium, Cyprus, Denmark, Finland, France, Germany, Greece, Ireland, Italy, Luxembourg, Malta, Netherlands, Norway, Portugal, Spain, Sweden, and Switzerland.15

Data Abstraction and Data Management—We will use a systematic approach to extract all data to minimize the risk of bias in this process. We will create standardized forms for data extraction and pilot test them. 

Each article will undergo double review by the study investigators for data abstraction. The second reviewer will confirm the first reviewer’s abstracted data for completeness and accuracy. Reviewer pairs will be formed to include personnel with both clinical and methodological expertise. Reviewers will not be masked to the authors of the articles, their respective institutions, nor the journals in which their articles were published.

For all articles, the reviewers will extract information on general study characteristics (e.g., study design, data source, study period, and country), study participants (e.g., population, age, sex, race/ethnicity, whether they were admitted to the hospital or discharged), the type of provider (e.g., physician vs. advanced practice provider), if the provider was trained in emergency medicine or not (and the level of training or experience), the definition of diagnostic error, the method of ascertainment of outcomes, and the outcome results, including measures of variability. To the extent possible, we will also look for other variables that have been previously associated with diagnostic error as suspected causal factors (e.g., rural vs. urban ED location, night/weekend, ED discharge fraction, socioeconomic status, whether an interpreter was used).

All information from the article review process will be entered into a DistillerSR database (Evidence Partners Inc., Ottawa, Canada) by the reviewer. Reviewers will enter comments into the system whenever applicable. The DistillerSR database will be used to maintain the data and to create detailed evidence tables and summary tables. We may contact the authors of the included studies for additional data, if necessary. Data will later be uploaded into the Systematic Review Data Repository.

Assessment of Methodological Risk of Bias of Individual Studies—We will assess the risk of bias using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.16 The QUADAS-2 tool assesses the risk of bias in four domains: patient selection, index test, reference standard, and flow and timing. Two reviewers will independently evaluate the risk of bias of each study. Differences between reviewers will be resolved by consensus adjudication.

Data Synthesis—We will organize the report by Key Question and then by condition. For each Key Question, we will create a set of detailed evidence tables containing all information extracted from eligible studies. We will conduct meta-analyses when there are sufficient data (at least three studies) and studies are sufficiently homogenous with respect to key variables (population characteristics, condition, provider type, and data source/study design).

Heterogeneity among the studies for each outcome we consider appropriate for quantitative pooling will be tested using a standard chi-squared test using a significance level of alpha less than or equal to 0.10. We also will examine heterogeneity among studies with an I-squared statistic, which describes the variability in effect estimates that is due to heterogeneity rather than random chance. A value greater than 50 percent will be considered to indicate substantial heterogeneity.17

We will calculate a mean error rate or serious misdiagnosis-harm rate by using a random-effects model with the DerSimonian and Laird formula in settings of low heterogeneity18 or with appropriate analyses when there is higher heterogeneity.19

Publication bias will be examined by using Begg's test and Egger's test, including evaluation of the asymmetry of funnel plots for each comparison of interest for the outcomes for which meta-analyses are conducted and there are at least 10 studies.20,21 Publication bias will also be qualitatively considered as part of the strength of evidence determination.

We will consider study heterogeneity before all quantitative, meta-analytic syntheses. For example, we will exclude from the primary analysis of KQ1's most frequent clinical conditions any results from exclusive, specialty EDs (e.g., eye and ear), since specialty EDs are non-representative with respect to the spectrum of clinical conditions. Likewise, we will not combine incident report frequency (no denominator) with true error frequency (e.g., per 10,000 visits) in calculating overall error rates in the ED.

For KQ2, focused on error and harm rates, these may be expressed differently in different studies. In particular, there is likely to be a segregation in many studies between disease "present" and disease "absent" patients. Furthermore, some studies will present results conditioned on the presence or absence of true disease (sensitivity/specificity), while others will present results conditioned on the presence or absence of the diagnosis rendered (positive/negative predictive value). To make the results as clearly informative as possible, we will try to synthesize the following parameters across disease-specific studies, as permitted by the types of data that are available:

  • false negative rate (1-sensitivity) (disease present)
  • false positive rate (1-specificity) (disease absent)
  • false discovery rate (1-positive predictive value) (diagnosis present)
  • false omission rate (1-negative predictive value) (diagnosis absent)
  • total diagnostic error rate (1-accuracy for all patients [disease and non-disease])
  • overall cohort-based rates of errors and harms (e.g., 2 per 10,000 visits)

For KQ3, we do not anticipate being able to synthesize specific key causal factors of diagnostic errors and serious misdiagnosis-related harms. Therefore, we will use a high-level approach to synthesizing key causal factors into "cognitive" and "systems" factors, which are often used to classify causes in the diagnostic error literature. We will then describe common topics identified within each category.
STATA statistical software (Intercooled, version 14.2, StataCorp, College Station, TX) will be used for all meta-analyses.

Studies that are not amenable to pooling will be summarized qualitatively.

Grading the Strength of Evidence (SOE) for Major Comparisons and Outcomes—At the completion of our review, we will grade the strength of evidence addressing Key Question 2 by adapting an evidence grading scheme recommended by the Guide for Conducting Comparative Effectiveness Reviews.22 We will apply evidence grades to the bodies of evidence about diagnostic errors and serious misdiagnosis-related harms for each condition.

We will assess the limitations to individual study quality (using individual risk of bias assessments), consistency, directness, precision, and reporting bias. We will classify evidence pertaining to the Key Questions into four categories: (1) "high" grade (indicating high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of the effect); (2) "moderate" grade (indicating moderate confidence that the evidence reflects the true effect but further research could change our confidence in the estimate of the effect and may change the estimate); (3) "low" grade (indicating low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of the effect and is likely to change the estimate); and (4) "insufficient" grade (indicating evidence is unavailable or the body of evidence has unacceptable deficiencies that preclude reaching a conclusion).

Assessing Applicability—We will discuss the applicability of studies in terms of the degree to which the study population (e.g., age, sex, location), the types of providers, and settings are typical of emergency departments in the United States.

V. References

  1. National Academies of Science E, and Medicine,,. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press; 2015.
  2. Hussain F, Cooper A, Carson-Stevens A, et al. Diagnostic error in the emergency department: learning from national patient safety incident report analysis. BMC emergency medicine. 2019 Dec 4;19(1):77. doi: 10.1186/s12873-019-0289-3. PMID: 31801474.
  3. Newman-Toker DE, Schaffer AC, Yu-Moe CW, et al. Serious misdiagnosis-related harms in malpractice claims: The "Big Three" - vascular events, infections, and cancers. Diagnosis (Berlin, Germany). 2019 Aug 27;6(3):227-40. doi: 10.1515/dx-2019-0019. PMID: 31535832.
  4. Brown TW, McCarthy ML, Kelen GD, et al. An epidemiologic study of closed emergency department malpractice claims in a national database of physician malpractice insurers. Acad Emerg Med. 2010 May;17(5):553-60. doi: 10.1111/j.1553-2712.2010.00729.x. PMID: 20536812.
  5. Kachalia A, Gandhi TK, Puopolo AL, et al. Missed and delayed diagnoses in the emergency department: a study of closed malpractice claims from 4 liability insurers. Ann Emerg Med. 2007 Feb;49(2):196-205. doi: 10.1016/j.annemergmed.2006.06.035. PMID: 16997424.
  6. Selbst SM, Friedman MJ, Singh SB. Epidemiology and etiology of malpractice lawsuits involving children in US emergency departments and urgent care centers. Pediatric emergency care. 2005 Mar;21(3):165-9. PMID: 15744194.
  7. Trautlein JJ, Lambert RL, Miller J. Malpractice in the emergency department--review of 200 cases. Ann Emerg Med. 1984 Sep;13(9 Pt 1):709-11. doi: 10.1016/s0196-0644(84)80733-7. PMID: 6465652.
  8. Newman-Toker DE, Wang Z, Zhu Y, et al. Rate of diagnostic errors and serious misdiagnosis-related harms for major vascular events, infections, and cancers: toward a national incidence estimate using the "Big Three". Diagnosis (Berlin, Germany). 2020 May 14. doi: 10.1515/dx-2019-0104. PMID: 32412440.
  9. Guly HR. Diagnostic errors in an accident and emergency department. Emergency medicine journal : EMJ. 2001 Jul;18(4):263-9. doi: 10.1136/emj.18.4.263. PMID: 11435359.
  10. Okafor N, Payne VL, Chathampally Y, et al. Using voluntary reports from physicians to learn from diagnostic errors in emergency medicine. Emergency medicine journal: EMJ. 2016 Apr;33(4):245-52. doi: 10.1136/emermed-2014-204604. PMID: 26531860.
  11. Glerum KM, Selbst SM, Parikh PD, et al. Pediatric Malpractice Claims in the Emergency Department and Urgent Care Settings From 2001 to 2015. Pediatric emergency care. 2018 Sep 11. doi: 10.1097/pec.0000000000001602. PMID: 30211835.
  12. Matthys LA, Coppage KH, Lambers DS, et al. Delayed postpartum preeclampsia: an experience of 151 cases. American journal of obstetrics and gynecology. 2004 May;190(5):1464-6. doi: 10.1016/j.ajog.2004.02.037. PMID: 15167870.
  13. Al-Safi Z, Imudia AN, Filetti LC, et al. Delayed postpartum preeclampsia and eclampsia: demographics, clinical course, and complications. Obstetrics and gynecology. 2011 Nov;118(5):1102-7. doi: 10.1097/AOG.0b013e318231934c. PMID: 21979459.
  14. Newman-Toker DE, Pronovost PJ. Diagnostic errors—the next frontier for patient safety. Jama. 2009 Mar 11;301(10):1060-2. doi: 10.1001/jama.2009.249. PMID: 19278949.
  15. Ferreira PL, Tavares AI, Quintal C, et al. EU health systems classification: a new proposal from EURO-HEALTHY. BMC health services research. 2018 Jul 3;18(1):511. doi: 10.1186/s12913-018-3323-3. PMID: 29970085.
  16. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Annals of internal medicine. 2011 Oct 18;155(8):529-36. doi: 10.7326/0003-4819-155-8-201110180-00009. PMID: 22007046.
  17. Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ (Clinical research ed). 2003 Sep 6;327(7414):557-60. doi: 10.1136/bmj.327.7414.557. PMID: 12958120.
  18. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled clinical trials. 1986 Sep;7(3):177-88. PMID: 3802833.
  19. Cornell JE, Mulrow CD, Localio R, et al. Random-effects meta-analysis of inconsistent effects: a time for change. Annals of internal medicine. 2014 Feb 18;160(4):267-70. doi: 10.7326/m13-2886. PMID: 24727843.
  20. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics. 1994 Dec;50(4):1088-101. PMID: 7786990.
  21. Egger M, Davey Smith G, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ (Clinical research ed). 1997 Sep 13;315(7109):629-34. PMID: 9310563.
  22. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—agency for healthcare research and quality and the effective health-care program. Journal of clinical epidemiology. 2010 May;63(5):513-23. doi: 10.1016/j.jclinepi.2009.03.009. PMID: 19595577.

VI. Definition of Terms

APP = advanced practice providers

CINAHL = Cumulative Index to Nursing and Allied Health Literature

ED = emergency department

KQ = Key Question

NAM = National Academy of Medicine

NAIC = National Association of Insurance Commissioners

SEADS = Supplemental Evidence and Data for Systematic Reviews

SOE = strength of evidence

VII. Summary of Protocol Amendments

If we need to amend this protocol, we will give the date of each amendment, describe the change, and give the rationale in this section. Changes will not be incorporated into the protocol. Example table below:

Date Section Original Protocol Revised Protocol Rationale
This should be the effective date of the change in protocol. Specify where the change would be found in the protocol. Describe the language of the original protocol. Describe the change in protocol. Justify why the change will improve the report. If necessary, describe why the change does not introduce bias. Do not use justification as "because the AE/TOO/TEP/Peer reviewer told us to" but explain what the change hopes to accomplish.

 

VIII. Review of Key Questions

The Agency for Healthcare Research and Quality (AHRQ) posted the key questions on the AHRQ Effective Health Care Website for public comment. The Evidence-based Practice Center (EPC) refined and finalized the key questions after review of the public comments, and input from Key Informants and the Technical Expert Panel (TEP). This input is intended to ensure that the key questions are specific and relevant. 

IX. Key Informants

Key Informants are the end users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions.  Within the EPC program, the Key Informant role is to provide input into identifying the Key Questions for research that will inform healthcare decisions.  The EPC solicits input from Key Informants when developing questions for systematic review or when identifying high priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Key Informants must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest.  Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained.  The AHRQ Task Order Officer (TOO) and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

X. Technical Experts

Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and suggest approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor do they contribute to the writing of the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Technical Experts must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest.  Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The AHRQ TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

XI. Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report.  Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published three months after the publication of the evidence report. 

Potential Peer Reviewers must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $5,000.  Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

XII. EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.  

XIII. Role of the Funder

This project was funded under Contract No.75Q80120D00003 from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The AHRQ Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.  

XIV. Registration

This protocol will be registered in the international prospective register of systematic reviews (PROSPERO).

Appendix A. Search Strategy

# String Hits
1 "diagnosis error"[tiab] OR "diagnosis errors"[tiab] OR "diagnostic error"[tiab] OR "diagnostic errors"[tiab] OR "misdiagnosis"[tiab] OR "misdiagnoses"[tiab] OR "missed diagnosis"[tiab] OR "missed diagnoses"[tiab] OR "wrong diagnosis"[tiab] OR "wrong diagnoses"[tiab] OR "inaccurate diagnosis"[tiab] OR "inaccurate diagnoses"[tiab] OR "delayed diagnosis"[tiab] OR "delayed diagnoses"[tiab] OR "diagnosis delay"[tiab] OR "diagnosis delays"[tiab] OR "diagnostic delay"[tiab] OR "diagnostic delays"[tiab] OR "failure to diagnose"[tiab] OR "diagnostic interval"[tiab] OR "diagnostic intervals"[tiab] OR (Delayed diagnosis[mh]) OR (Diagnosis, differential[mh])] OR (diagnos*[tiab] AND delay*[tiab]) 547,587
2 (emergency services, hospital[mh] OR emergency treatment[mh] OR emergency department*[tiab] OR emergency service*[tiab] OR emergency physician*[tiab] OR casualty[tiab] OR ambulance*[tiab] OR initial diagnosis[tiab] OR initial contact[tiab] OR warning[tiab] OR urgent care[tiab]) OR emergency room[tiab] OR “accident and emergency”[tiab] OR “accident & emergency”[tiab] OR “Emergency department returns”[tiab] OR “ED returns”[tiab] 316,844
3 #1 AND #2 14,202
4 Cerebrovascular disorders[mh:noexp] OR Basal ganglia cerebrovascular disease[mh] OR Brain ischemia[mh] OR Carotid artery diseases[mh] OR Intracranial arterial diseases[mh] OR "Intracranial embolism and thrombosis"[mh] OR Intracranial hemorrhages[mh] OR Stroke[mh:noexp] OR Brain infarction[mh] OR Vertebral artery dissection[mh] OR stroke[tiab] OR cerebrovasc*[tiab] OR brain vasc*[tiab] OR cerebral vasc*[tiab] OR CVA[tiab] OR apoplex*[tiab] OR ((brain*[tiab] OR cerebr*[tiab] OR cerebell*[tiab] OR vertebrovasilar[tiab] OR hemispher*[tiab] OR intracran*[tiab] OR intracerebral[tiab] OR infratentorial[tiab] OR supratentorial[tiab] OR MCA[tiab] OR anterior circulation[tiab] OR posterior circulation[tiab] OR basal gangla[tiab]) AND (ischaemi*[tiab] OR ischemi*[tiab] OR infarct*[tiab] OR thrombo*[tiab] OR emboli*[tiab])) 
OR ((brain*[tiab] OR cerebr*[tiab] OR cerebell*[tiab] OR intracerebral[tiab] OR intracran*[tiab] OR parenchymal[tiab] OR intraventricular[tiab] OR infratentorial[tiab] OR supratentorial[tiab] OR basal gangli*[tiab]) AND (haemorrhage*[tiab] OR hemorrhage*[tiab] OR haematoma*[tiab] OR hematoma*[tiab] OR bleed*[tiab])) OR Myocardial infarction[mh] OR myocardial infarct*[tiab] OR heart infarct*[tiab] OR (coronary[tiab] AND syndrome[tiab]) OR heart attack[tiab] OR Thrombosis[mh:noexp] OR Thromboembolism[mh:noexp] OR Venous thromboembolism[mh:noexp] OR Venous thrombosis[mh] OR thromboprophyla*[tiab] OR thrombus*[tiab] OR thrombotic*[tiab] OR thrombolic*[tiab] OR thromboemboli*[tiab] OR thrombos*[tiab] OR embol*[tiab] OR Pulmonary embolism[mh] OR PE[tiab] OR DVT[tiab] OR VTE[tiab] OR ((vein*[tiab] OR veno*[tiab] OR vent*[tiab]) AND thromb*[tiab]) 
OR Aortic aneurysm[mh] OR Aneurysm, dissecting[mh:noexp] OR Aneurysm, ruptured[mh] OR ((aort*[tiab] AND (aneurys*[tiab] OR dissect*[tiab] OR ruptur*[tiab] OR tear*[tiab] OR trauma*[tiab] OR split*[tiab])) OR Mesenteric ischemia[mh] OR (ischemi*[tiab] AND mesenteric[tiab]) OR (arterial[tiab] AND thromb*[tiab]) OR Sepsis[mh] OR Septicemia[mh] OR Shock, Septic[mh] OR septicem*[tiab] OR septicaem*[tiab] OR seps*[tiab] OR (sept*[tiab] AND shock*[tiab]) OR Meningitis[mh] OR meningit*[tiab] OR Encephalitis[mh] OR encephalitis[tiab] OR meningoencephalitis[tiab] OR ((brain[tiab] OR cerebral[tiab]) AND (infection*[tiab] OR infectious[tiab] OR inflamm*[tiab] OR swell*[tiab])) OR Epidural Abscess[mh] OR ((spin* OR epidural[tiab]) AND abscess*[tiab]) OR Pneumonia[mh] OR Respiratory tract infections[mh] OR pneumonia*[tiab] OR lung inflammation*[tiab] OR respiratory tract infection*[tiab] OR respiratory infection*[tiab]) OR Endocarditis[mh] OR endocarditis[tiab] OR (endocardium AND (inflamm*[tiab] OR infect*[tiab])) OR Appendicitis[mh] OR appendic*[tiab] OR appendicitis acuta[tiab]
[2,115,643
5 #1 AND #4 87,452
6 #3 OR #5