As trained professionals who assist in out-of-hospital emergencies, Emergency Medical Services (EMS) clinicians provide around-the-clock, life-saving prehospital care to individuals with medical or traumatic emergencies. EMS clinicians have been shown to be at high risk for anxiety, depression, posttraumatic stress disorder (PTSD), and suicide.¹ A 2018 systematic review documented the following prevalence among first responders: anxiety (15%), depression (15%), PTSD (11%), and general psychological distress (27%).² The proportion of deaths attributed to suicide among EMS clinicians (5.2%) is more than twice that in the general population (2.2%).³

Even larger proportions of the workforce are impacted by burnout and moral injury. A 2019 survey of 1,547 EMS clinicians from the world’s largest cities found that 60% agreed with the statement “I feel burned out in my EMS work” and 36% agreed with the statement “I don’t want to do EMS work anymore.”⁴ A 2023 survey of 850 professionals working in 911 call centers in the U.S., Canada, and Mexico found that 84% of respondents experienced high call volumes multiple times a week (50% experienced this daily).⁵ Three in four respondents (75%) noted that their call center faced staff burnout.⁵

Burnout has now been classified as an occupational phenomenon by the International Classifications of Diseases-11 (ICD-11), which defines it as “a syndrome resulting from chronic workplace stress that has not been successfully managed.”⁶ Feelings of lethargy and emotional exhaustion when on the job, negativism toward one’s occupation, and reduced professional output characterize burnout.⁶ The EMS profession exposes EMS clinicians to various traumatic or stressful circumstances in which they may “perpetrate, fail to prevent, or witness events that contradict deeply held moral beliefs and expectations.”⁷ Moral injury is defined as the distressing psychological, behavioral, social, and sometimes spiritual aftermath of exposure to such events.⁸ Moral injury has been shown to contribute to burnout and reduced ability to provide care.⁹ Moral injury can also be a predecessor to mental health concerns among EMS/911 workers.¹⁰

Various factors contribute to burnout, moral injury, and mental health issues. Various underlying factors may be associated with burnout, stress, moral injury, and mental health issues (see Figure 1). We have conceptualized these underlying factors as psychosocial factors (e.g., health behaviors, social support), organizational conditions (e.g., long hours, shift work), and environmental exposures (e.g., exposure to violence on the job). This is based on the Psychosocial factors, Organizational conditions, and Environmental exposures (POE) framework.^11-13 Frequent shifts and frequent calls during shifts can lead to inadequate sleep, poor diet, overworking, injuries on the job, and greater numbers of interactions with abusive or difficult patients and family.^1,14 Due to such factors, EMS clinicians are routinely exposed to high levels of stress. Approximately 69% of first responders do not have enough time to recover completely from occupational stressful events because of their frequent occurrence.¹⁵ Constant exposure to such situations leads to chronic stress, which is often untreated. Constant occupational stressors, such as excessive work hours,¹⁶ job dissatisfaction, inadequate salaries and financial stress, workplace violence, and repeated layoffs of professional staff (which increase the burden on remaining staff).¹⁷ During the coronavirus disease 2019 (COVID-19) pandemic, major stressors included exposure to the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), related shortages of personal protective equipment (PPE), and the inability to provide adequate care for all patients.¹⁰ These stressors fueled additional burnout, moral injury, anxiety, and depression among EMS clinicians.¹⁸ In one study, a third of paramedics suffered high levels of emotional exhaustion and a third had high levels of depersonalization while treating COVID-19 patients, reflecting significant burnout.¹⁹

There is an urgent need to address burnout, moral injury, and mental health issues among EMS clinicians. Burnout, moral injury, and mental health concerns have threatened clinician retention in the EMS workforce.²⁰ Even for those EMS clinicians who remain in the workforce, these challenges impact their ability to provide care that adequately addresses the needs of their patients.²¹ The patient population that needs emergency care is perhaps the most vulnerable to the impacts of clinician burnout and a diminished workforce. Better resources and interventions are needed urgently to improve the mental and behavioral health of the EMS and 911 workforces.

Research has identified some factors, such as strengthened social networks, positive coping responses, and religious beliefs, that may mitigate the impact of mental health and burnout among EMS clinicians.²² Although widely accepted approaches exist to cope with stress in the general population, such as promoting sleep, exercise, engagement with peers, and meditation, these approaches may not be feasible for many EMS workers in the context of increased burden on a diminishing workforce. Healthcare organizations try to mitigate stressors on the EMS workforce through resilience training, wellness courses, and similar strategies. Frontline healthcare workers have reported that peer-to-peer support and dedicated wellness spaces have helped them cope with the stress and burnout related to the pandemic,²³ but whether these are effective on a wider scale is unclear.

Kaminsky and colleagues described the Johns Hopkins Resistance–Resilience–Recovery Model of Human Resistance, Resilience, and Recovery. According to this model, resilience is conceptualized along the spectrum that includes resistance, resilience, and recovery.²⁴ Resistance specifically refers to “the ability of an individual, a group, an organization, or even an entire population to literally resist manifestations of clinical distress, impairment, or dysfunction associated with critical incidents, terrorism, and even mass disasters.”²⁴ In other words, resistance is “a form of psychological/behavioral immunity to distress and dysfunction.”²⁴ Resilience specifically refers to the “ability of an individual, a group, an organization, or even an entire population to rapidly and effectively rebound from psychological and/or behavioral perturbations associated with critical incidents, terrorism, and even mass disasters.” Resilience is considered one of the antidotes to the challenges faced by EMS professionals. Improved health of EMS clinicians and their resilience to mental health challenges, burnout, and moral injury, both in the short term and the long term, are essential to foster the well-being of the EMS/911 workforce and to sustain its effectiveness in handling the emergency needs of the general population. Recovery specifically refers to the “ability of an individual, a group, an organization, or even an entire population to literally recover the ability to adaptively function, both psychologically and behaviorally, in the wake of a significant clinical distress, impairment, or dysfunction subsequent to critical incidents, terrorism, and even mass disasters.”²⁴ Similarly, the Substance Abuse and Mental Health Services Administration (SAMHSA) defines recovery as “a process of change through which individuals improve their health and wellness, live self-directed lives, and strive to reach their full potential.”²⁵ Thus, recovery is considered as a process rather than an end state.

Interventions targeting behavioral health issues can therefore be conceptualized as targeting resistance, resilience, and/or recovery. Interventions that aim to improve resistance include pre-incident preventive interventions (primary prevention of the behavioral health issues²⁶), such as behavioral preparation and psychological preparation (dedication, tenacity, embracing challenges, confidence, sense of control/ self-efficacy). The goal of such interventions is to enable the person faced with adversity to maintain a relatively or consequentially imperturbable level of well-being and functioning. Interventions that aim to improve resilience include acute, short-term psychological crisis interventions, such as psychological first aid typically administered during and shortly after a potentially distressing incident, that are designed to stabilize and mitigate acute distress (secondary prevention of the behavioral health issues²⁶). Interventions that aim to improve recovery address the subacute manifestations of distress and dysfunction through counseling, psychotherapy, and psychiatric medications. In the EMS/911 workforce context, interventions that aim to promote resistance and resilience are particularly tailored to this population, whereas interventions that aim to promote recovery are very similar to interventions that aim to do so in the general population. Therefore, the current project focuses on interventions that aim to promote resistance and/or resilience.

The critical decisional dilemma underpinning the proposed systematic review relates to the identification of comprehensive strategies, both at the individual level and the system level, to improve mental health and reduce burnout and moral injury in the EMS and 911 workforce.

Key Questions

Key Question 1: What are the incidence, prevalence, and severity of mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) among the EMS and the 911 workforce?

1. Are the incidence, prevalence, and severity modified by:
   1. Agency composition including workflow, regulations, financing?
   2. Characteristics of EMS and 911 personnel (e.g., education/training, proficiency, experience, trauma exposure)?
   3. Physical and mental health resources?

Key Question 2: What are the effectiveness and comparative effectiveness, including benefits and harms, of interventions addressing mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) among the EMS and 911 workforce?

1. Are the effectiveness of the interventions modified by:
   1. Intervention type?
   2. Characteristics of EMS and 911 personnel (e.g., education/training, proficiency, experience)?
   3. EMS/911 agency characteristics including workflow, regulations, financing?
   4. Physical and mental health resources?

Key Question 3: What are the context and implementation factors of studies with effective EMS/911 workforce practices to prevent, recognize and treat mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury)? This description might include distinguishing factors such as workforce training, surveillance, resilience training, occupational health services, peer-to-peer support, preparedness for trauma exposure, and program funding.

Key Question 4: What future research is needed to close existing evidence gaps regarding preventing, recognizing, and treating mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) in the EMS/911 workforce?

Study Eligibility Criteria

The specific eligibility criteria provided in the following table have been refined based on discussions with a Technical Expert Panel (TEP). The table depicts criteria for KQs 1, 2, and 3 only because KQ4 is about gaps in the evidence pertaining to the other three KQs.

*Important outcomes that will be used when developing Strength of Evidence tables.

Abbreviations: CISM = critical incident stress management, EAP = employee assistance program, EMS = Emergency Medical Services, KQ = Key Question, PPE = personal protective equipment, PTSD = posttraumatic stress disorder.

KQ1= What are the incidence, prevalence, and severity of mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) among the EMS and the 911 workforce?

KQ2= What are the effectiveness and comparative effectiveness, including benefits and harms, of interventions addressing mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) among the EMS and 911 workforce?

KQ3= What are the context and implementation factors of studies with effective EMS/911 workforce practices to prevent, recognize and treat mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury)?

KQ4 (not depicted in Figure) = What future research is needed to close existing evidence gaps regarding preventing, recognizing, and treating mental health issues (depression, anxiety, PTSD, suicidality, and substance use disorders) and occupational stress issues (burnout, stress, and moral injury) in the EMS/911 workforce?

Criteria for Inclusion/Exclusion of Studies in the Review: See Study Eligibility Criteria in Section II. In terms of study design, we expect to include a variety of non-randomized study designs because very few RCTs may be identified. We will not restrict by sample size or study quality. In terms of timing, we will restrict to the last 22 years because older studies likely have little relevance to modern EMS practices. A 22-year cut-off corresponds to the September 11, 2001, terrorist attacks in the United States.

Literature Search Strategies to Identify Relevant Studies to Answer the Key Questions: We will conduct a literature search in Medline (via PubMed), Embase, The Cochrane Register of Clinical Trials, PsycINFO, and the Cumulative Index to the Nursing and Allied Health Literature (CINAHL). We will restrict the search to English-language studies published in the year 2001 onwards. We will include filters to remove nonhuman studies and articles that are not primary studies or systematic reviews. We will include specific controlled vocabulary terms (medical subject headings [MeSH]or Emtree), along with specific free-text words, related to EMS-, prehospital-, and 911-related terms combined with mental and behavioral health-related terms. The searches will be independently peer reviewed by a librarian using the Peer Review of Electronic Search Strategies (PRESS) checklist. Appendix A includes the search strategy for Medline We will also search the following journals that are not indexed in Medline: International Journal of Paramedicine, Journal of Paramedic Practice, International Paramedic Practice, Irish Journal of Paramedicine, and Annals of Emergency Dispatch and Response.

To identify studies that are not published in journals, we will also search the ClinicalTrials.gov registry for ongoing studies, unpublished study protocols, and unpublished study results. We will also search the websites of the National Association of State EMS Officials, the National Association of EMTs (https://www.naemt.org), the National Association of EMS Educators, the EMS Eagles Global Alliance, the Administration for Strategic Preparedness and Response Technical Resources, Assistance Center, and Information Exchange (ASPR TRACIE), and the International Academies of Emergency Dispatch.

The reference lists of all included studies and relevant existing systematic reviews identified will be screened for additional eligible studies. Additional articles suggested to us from any source, including peer and public review, will be screened applying identical eligibility criteria.

A Supplemental Evidence and Data for Systematic review (SEADS) portal will be available for this review. A Federal Register Notice will also be posted for this review. Additional articles suggested to us from any source, including peer and public review, will be screened applying identical eligibility criteria.

We will update the search when the Draft Report is posted for peer and public review.

Screening Process: Citations from all searches will be deduplicated and entered into DistillerSR® to enable title and abstract screening. The team will conduct two or more rounds of pilot screening. During each pilot round, two or more members of the team will screen the same 100 abstracts and discuss conflicts, with the goal of training the team in the nuances of the eligibility criteria and refining them as needed. After the pilot rounds, we will continue abstract screening in duplicate. DistillerSR® has machine learning capabilities that predict the likelihood of relevance of each citation. DistillerSR® then presents the most potentially relevant articles first. This process will make screening more efficient and will enable us to capture almost all relevant articles relatively early in the abstract-screening process.      

Potentially relevant citations will be retrieved in full text. All these articles will be rescreened in duplicate.

Data Extraction and Data Management: Data from eligible studies will be extracted into the Systematic Review Data Repository Plus (SRDR+) software. Each study will be extracted by one researcher, and entered data will be confirmed by a second, independent researcher. Individual studies with multiple publications will be extracted as a single study (with a single record in SRDR+). Each study will be entered into SRDR+ separately, even if two or more studies are reported within a single publication.

For each study, we will extract publication identifying data, study design features, population characteristics, intervention and comparator names and descriptions, relevant outcomes and their definitions, results, and funding source. We will extract, as available, data on the effect modifiers that are relevant to the KQ(s) being addressed by each study.

Assessment of Risk of Bias in Individual Studies: We will evaluate each study for risk of bias and methodological quality. Because we anticipate including a variety of study designs, we will use various existing commonly used tools.

For KQ 1 (incidence/prevalence/severity), for longitudinal studies, we will use items from the Joanna Briggs Institute Checklist for Cohort Studies²⁷ and the Newcastle Ottawa Scale for Cohort Studies.²⁸ For cross-sectional studies, we will use items from the tool proposed by Hoy et al.²⁹ For RCTs addressing KQs 2 and 3 (intervention effectiveness and harms), we will use items from the Cochrane Risk of Bias Tool.³⁰ For non-randomized comparative studies of interventions, we will use items from the ROBINS-I Tool.³¹ For KQ4 (summary of the gaps identified in the evidence), we will not conduct a risk of bias assessment.

Data Synthesis: We will summarize the evidence qualitatively and, when feasible and appropriate, quantitatively (i.e., by meta-analysis). Each study included in the systematic review will be described in summary and evidence tables presenting study design features, study participant characteristics, descriptions of interventions, outcome results, and risk of bias/methodological quality. Summary tables will briefly describe the studies and their findings.

We anticipate heterogeneity among interventions in terms of their content, intensity, and complexity. We will detail these features in evidence tables and summarize them in the text of the report to allow readers to compare the components of various interventions as well as how and why their effectiveness and harms may differ.

As reported data allow, we will primarily evaluate relative risks (RRs) for dichotomous outcomes (e.g., presence of suicidality), net mean differences (NMDs) (i.e., difference in differences or between-intervention comparisons of within-intervention changes) for continuous outcomes with both pre- and postintervention data (e.g., depression scales), and differences (between interventions) in continuous outcome data postintervention (e.g., anxiety scales). For non-randomized studies, we will consider excluding unadjusted analyses or at the least prioritize adjusted over unadjusted analyses. Where there are at least three studies that compare sufficiently similar interventions (or strategies) and report sufficiently similar outcomes at sufficiently similar time points, we plan to conduct pairwise meta-analyses using random-effects models. We will explore opportunities to evaluate outcomes by effect modifiers both from within-study data and across studies.

Grading the Strength of Evidence (SoE) for Major Comparisons and Outcomes: For KQs 1, 2, and 3, we will grade the strength of the body of evidence as per the Agency for Healthcare Research and Quality (AHRQ) Methods Guide on assessing SoE.^32,33 We will use our discussions with the technical expert panel (TEP) to finalize the list of outcomes that will be prioritized for strength of evidence assessment. The current list of prioritized outcomes includes:

• Hospitalizations
• Burnout
• Resilience
• Anxiety
• Depression
• PTSD
• Substance use
• Suicidality
• Withdrawal from EMS/911 workforce
• Unintended harms of interventions

For each strength of evidence assessment, we will consider the number of studies, their study designs, the study limitations (i.e., risk of bias and overall methodological quality), the directness of the evidence to the KQs, the consistency of study results, the precision of any estimates of effect, the likelihood of reporting bias, other limitations, and the overall findings across studies. Based on these assessments, we will assign a strength of evidence rating as being either high, moderate, low, or insufficient evidence to estimate an effect.

Outcomes with imprecise estimates or inconsistent findings across studies that preclude a conclusion or with data from only one study will be deemed to have insufficient evidence to allow for a conclusion (with the exception that a particularly large, low risk of bias, well-generalizable single study could provide low strength of evidence). This approach is consistent with the concept that for imprecise evidence “any estimate of effect is very uncertain,” the definition of very low-quality evidence per GRADE.³⁴

We will summarize the data sources, basic study characteristics, and each strength of evidence dimensional rating in an evidence profile table. This table will detail our reasoning for arriving at the overall strength of evidence rating.

Assessing Applicability: For each KQ, we will assess the applicability of the included studies to the EMS and 911 workforce in the U.S. based primarily on the studies’ eligibility criteria and their included participants, specifically related to such factors as demographics, intervention type, agency characteristics (e.g., size, regulations), and country.

1. Mock EF, Wrenn KD, Wright SW, Eustis TC, Slovis CM. Anxiety levels in EMS providers: effects of violence and shifts schedules. Am J Emerg Med. Oct 1999;17(6):509-11. doi:10.1016/s0735-6757(99)90186-9
2. Petrie K, Milligan-Saville J, Gayed A, et al. Prevalence of PTSD and common mental disorders amongst ambulance personnel: a systematic review and meta-analysis. Soc Psychiatry Psychiatr Epidemiol. Sep 2018;53(9):897-909. doi:10.1007/s00127-018-1539-5
3. Vigil NH, Grant AR, Perez O, et al. Death by Suicide-The EMS Profession Compared to the General Public. Prehosp Emerg Care. May-Jun 2019;23(3):340-345. doi:10.1080/10903127.2018.1514090
4. Rosenerger RA, Fowler RL, Robbins M. Burnout among EMS professionals: Incidence, assessment and management.
5. National Emergency Number Association (NENA) and Carbyne. Pulse of 9-1-1: 2023 NENA & Carbyne State of the Industry Survey. Accessed September 13, 2023.
6. World Health Organization (WHO). Burn-out an "occupational phenomenon": International Classification of Diseases.
7. Litz BT, Stein N, Delaney E, et al. Moral injury and moral repair in war veterans: a preliminary model and intervention strategy. Clin Psychol Rev. Dec 2009;29(8):695-706. doi:10.1016/j.cpr.2009.07.003
8. Griffin BJ, Purcell N, Burkman K, et al. Moral Injury: An Integrative Review. J Trauma Stress. Jun 2019;32(3):350-362. doi:10.1002/jts.22362
9. Wu AW, Connors C, Everly GS, Jr. COVID-19: Peer Support and Crisis Communication Strategies to Promote Institutional Resilience. Ann Intern Med. Jun 16 2020;172(12):822-823. doi:10.7326/m20-1236
10. Williamson V, Murphy D, Greenberg N. COVID-19 and experiences of moral injury in front-line key workers. Occup Med (Lond). Jul 17 2020;70(5):317-319. doi:10.1093/occmed/kqaa052
11. The Johns Hopkins P.O.E. Total Worker Health® Center (POE Center) in Mental Health. About Us. Accessed May 20, 2023.
12. Woods EH, Zhang Y, Roemer EC, Kent KB, Davis MF, Goetzel RZ. Addressing Psychosocial, Organizational, and Environmental Stressors Emerging From the COVID-19 Pandemic and Their Effect on Essential Workers' Mental Health and Well-being: A Literature Review. J Occup Environ Med. May 1 2023;65(5):419-427. doi:10.1097/jom.0000000000002802
13. Zhang Y, Woods EH, Roemer EC, Kent KB, Goetzel RZ. Addressing Workplace Stressors Emerging from the Pandemic. Am J Health Promot. Sep 2022;36(7):1215-1223. doi:10.1177/08901171221112488b
14. Firth-Cozens J, Greenhalgh J. Doctors' perceptions of the links between stress and lowered clinical care. Soc Sci Med. Apr 1997;44(7):1017-22. doi:10.1016/s0277-9536(96)00227-4
15. Bentley MA, Crawford JM, Wilkins JR, Fernandez AR, Studnek JR. An assessment of depression, anxiety, and stress among nationally certified EMS professionals. Prehosp Emerg Care. Jul-Sep 2013;17(3):330-8. doi:10.3109/10903127.2012.761307
16. Reith TP. Burnout in United States Healthcare Professionals: A Narrative Review. Cureus. Dec 4 2018;10(12):e3681. doi:10.7759/cureus.3681
17. Gold Y. Burnout: Causes and Solutions. The Clearing House: A Journal of Educational Strategies, Issues and Ideas. 1985/01/01 1985;58(5):210-212. doi:10.1080/00098655.1985.9955539
18. Novilla MLB, Moxley VBA, Hanson CL, et al. COVID-19 and Psychosocial Well-Being: Did COVID-19 Worsen U.S. Frontline Healthcare Workers' Burnout, Anxiety, and Depression? Int J Environ Res Public Health. Mar 1 2023;20(5)doi:10.3390/ijerph20054414
19. Abd El Salam Amin Yacout D, Yousef Mohamed N, Hosni El Sherbini H. Post-Traumatic Stress Disorders and Burnout Syndrome among Community Paramedic Staff. Egyptian Journal of Health Care. 2021;12(1):340-363.
20. Moring SM. Burnout and retention in EMS systems: Mid-level managers and policy effectiveness. Doctoral. Capella University; 2023. Accessed May 20, 2023.
21. Patterson PD, Probst JC, Leith KH, Corwin SJ, Powell MP. Recruitment and retention of emergency medical technicians: a qualitative study. J Allied Health. Fall 2005;34(3):153-62.
22. Boland LL, Mink PJ, Kamrud JW, Jeruzal JN, Stevens AC. Social Support Outside the Workplace, Coping Styles, and Burnout in a Cohort of EMS Providers From Minnesota. Workplace Health Saf. Aug 2019;67(8):414-422. doi:10.1177/2165079919829154
23. Blake H, Gupta A, Javed M, et al. COVID-Well Study: Qualitative Evaluation of Supported Wellbeing Centres and Psychological First Aid for Healthcare Workers during the COVID-19 Pandemic. Int J Environ Res Public Health. Mar 31 2021;18(7)doi:10.3390/ijerph18073626
24. Kaminsky M, McCabe OL, Langlieb AM, Everly GS, Jr. An Evidence-Informed Model of Human Resistance, Resilience, and Recovery: The Johns Hopkins’ Outcome-Driven Paradigm for Disaster Mental Health Services. Brief Treatment and Crisis Intervention. 2007;7(1):1-11.
25. Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services. SAMHSA’s Working Definition of Recovery. . Vol. PEP12-RECDEF. 2012. Accessed September 13, 2023.
26. Caplan G. Principles of Preventive Psychiatry. Basic Books; 1964.
27. Munn Z, Moola S, Lisy K, Riitano D, Tufanaru C. Chapter 5: Systematic reviews of prevalence and incidence. In: Aromataris E, Munn Z, eds. JBI Manual for Evidence Synthesis. 2020.
28. Wells GA, Wells G, Shea B. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses. 2014. Accessed July 24, 2023.
29. Hoy D, Brooks P, Woolf A, et al. Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol. Sep 2012;65(9):934-9. doi:10.1016/j.jclinepi.2011.11.014
30. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Bmj. Oct 18 2011;343:d5928. doi:10.1136/bmj.d5928
31. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Bmj. Oct 12 2016;355:i4919. doi:10.1136/bmj.i4919
32. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Accessed March 5, 2023.
33. Berkman ND, Lohr KN, Ansari MT, et al. Grading the strength of a body of evidence when assessing health care interventions: an EPC update. J Clin Epidemiol. Nov 2015;68(11):1312-24. doi:10.1016/j.jclinepi.2014.11.023
34. Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. Bmj. May 10 2008;336(7652):1049-51. doi:10.1136/bmj.39493.646875.AE

Technical experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, and outcomes and identify particular studies or databases to search. The TEP is selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that fosters a thoughtful, relevant systematic review. Therefore, study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts.

Technical experts provide information to the EPC to identify literature search strategies and suggest approaches to specific issues as requested by the EPC. Technical experts do not do analysis of any kind; neither do they contribute to the writing of the report. They do not review the report, except as given the opportunity to do so through the peer or public review mechanism.

Members of the TEP must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as technical experts and those who present with potential conflicts may be retained. The AHRQ Task Order Officer and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparing the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers.

The EPC will complete a disposition of all peer review comments. The disposition of comments will be published 3 months after publication of the evidence report.

Potential peer reviewers must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Invited peer reviewers with any financial conflict of interest greater than $5,000 will be disqualified from peer review. Peer reviewers who disclose potential business or professional conflicts of interest can submit comments on draft reports through the public comment mechanism.

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Direct financial conflicts of interest that cumulatively total more than $1,000 will usually disqualify an EPC core team investigator.

This project is funded under Contract No. 75Q80120D00003/75Q80123F32012 from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The AHRQ Task Order Officer reviewed the EPC response to contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by either the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

This protocol will be registered in the international prospective register of systematic reviews (PROSPERO).