Draft Key Questions
Systematic Review Draft
On May 31, 2017 amendments were made to this protocol. To view the amendments, go to “Summary of Protocol Amendments.”
I. Objectives, Background and Rationale
The purpose of this systematic review is to identify and summarize the research evidence evaluating measures of circulatory and respiratory compromise. The review is focused on measures that can be used in the field to triage trauma patients. In field triage, emergency medical services (EMS) personnel assess and monitor circulation and respiration in a broad range of trauma patients to identify those who have severe injuries and who are likely to need high level trauma care. EMS personnel then use this information to inform triage and transport decisions.
This review is designed to help inform decisions about what measures should be recommended in field triage guidelines and promoted for use in EMS practice. The ultimate goal is to promote the efficient and effective use of trauma care resources in order to achieve good outcomes for patients.
Background and Rationale
In 2014, unintentional injury was the fourth leading cause of death in the United States, the leading cause for people 1 to 44 years of age, and in the top five for all age groups except people 65 and older (seventh for this group). In 2011, there were approximately 40 million emergency department (ED) visits for injuries; of these, 2.5 million were due to trauma complications and unspecified injuries.1 Pathways of care for trauma patients, which include people with intentional as well as unintentional injuries, are based on systems of care that address the various stages along the trauma chain of survival (Figure 1).2 Out-of-hospital care includes the early interventions and life support needed to prevent immediate deterioration and secure vital functions. Appropriate initial care is critical for successfully addressing injury and restoring quality of life. One important aspect of this initial care is accurately identifying which trauma patients are likely to have a serious injury so that this information can be used to inform triage and transport decisions. A key transport decision is selecting the appropriate destination hospital, specifically whether a patient should be transported to a hospital designated as a trauma center or not. These decisions are referred to as field triage.
In the United States, out-of-hospital trauma care is delivered predominately by EMS personnel. EMS personnel can include individuals with different levels of training and certification, including Emergency Medical Responder, Emergency Medical Technician (EMT), Advanced EMT, and Paramedic.3 EMS personnel must assess patients in environments that are often chaotic and sometimes dangerous. While it is often clear that a patient is injured, a more nuanced assessment is frequently required to inform the best course of action. EMS assessments must quickly identify if there are immediate life-threatening conditions or serious injuries that require emergent interventions. This assessment is often referred to as field triage. These assessments inform the crucial decisions about what actions to take, including deciding whether or not a patient requires transport to the highest level trauma center available in the system.
Figure 1. Trauma chain of survival
The trauma chain of survival. First displayed at the TraumaCare2002 conference in Stavanger, Norway, and reproduced with permission from Laerdal Medical, Stavanger, Norway. BLS = basic life support; ALS = advanced life support.
Deciding whether or not to transport a patient to a trauma center is an important decision as it affects outcomes.4,5 Under-triage, meaning transporting someone to a lower level of care, after trauma is associated with a substantial increase in mortality,6 and survival among patients with serious injuries is 25 percent higher when patients are treated at a major trauma center compared to a non-trauma center.7 However, trauma centers are a costly and limited resource. Patients without serious injuries can be treated effectively outside of major trauma centers and at hospitals that may be closer, and studies suggest that accurate triage could reduce over-use and produce substantial cost savings.8,9 Therefore, one of the goals of trauma systems is to efficiently concentrate patients with serious injuries in major trauma centers. Field trauma triage plays a critical role in this process.
Field triage criteria have been developed to help EMS personnel match patient needs with the appropriate level of care. The utility of these criteria is based on their ability to help identify trauma patients with serious injuries who have a high likelihood of mortality and other adverse clinical outcomes and who, therefore, need higher levels of care. Individual measures may also be combined into risk assessment instruments. For trauma care in the field, ideal measures and instruments need to be accurate and easy to administer and interpret under a variety of field conditions by personnel with varying levels of training.
In the current guidelines,10 field triage of injured patients consists of four steps designed to identify different levels of risk and match the risk to different transport decisions. The first step is to assess variables such as level of consciousness, circulation, and respiration. This first step is combined with the second step, an assessment of the anatomy of the injury and results are used to identify the most seriously injured patients who "should be transported preferentially to the highest level of care within the defined trauma system."1 If steps one and two do not identify the patient as most seriously injured, step three considers the details mechanism of injury (i.e., falls, high-risk auto crash, or motorcycle crash) and the recommendation is that patients with these injuries should be transported to the closest trauma center, which need not be the highest level. Step four adds consideration of several additional factors such as age, prior conditions (e.g., anticoagulation and bleeding disorders, end-stage renal disease, pregnancy >20 weeks), burns with or without trauma, and EMS judgement and recommends that medical control be contacted and transport to a trauma center or specific resource hospital be considered.
The initial triage criteria in the current guidelines are physiological (blood pressure, respiration status) and level of consciousness. Measures, monitors, and tools are needed to facilitate assessment of physiologic status, because unlike the anatomy of the injury, physiologic status cannot be directly observed. Thresholds indicating need for major level trauma care have been operationalized as GCS ≤13, systolic blood pressure <90 mmHg and respiratory rate less <10 or >29 breaths per minute (>20 in infants aged less than 1 year), or need for ventilatory support.10 Blood pressure and respiratory rate have been part of the guidelines since their initial publication in 1986; ventilatory support was added in 2011; and the GCS threshold has changed since initial publication.
Additionally, over the past decade, the research base has grown: the number of studies used to support the trauma triage guidelines has increased 24-fold from approximately two per year in the 2006 guidelines to about 48 per year in the 2011 guideline.10 Other developments, including point-of-care testing and sensors, may impact the entire triage process. Despite these changes, the criteria have remained relatively stable since the initial version.11 Recent prospective research suggests that the current Field Triage Decision Scheme fails to identify a substantial number of patients with serious injuries12 and that there is opportunity to optimize the field triage criteria, particularly the physiologic measures.
New continuous monitoring and communications technologies have been developed and present opportunities to collect, use, and transmit additional information in trauma care.13-16 Additionally, testing and evaluation of these physiologic indicators of trauma have revealed that they may perform differently in different populations. For example, some measures may underestimate risk and therefore lead to under-triaged elderly trauma victims,17,18 and some measures may require different cut-offs when assessing children.19 These factors have led to proposals to consider new potential indicators such as complexity/variability of heart rate, tissue O2 saturation, mean arterial pressure, lactate, end-tidal CO2, descriptors for respiratory effort, or derivatives such as the shock index (heart rate divided by systolic blood pressure), as well as new age-specific thresholds.20,21
Determining the need for trauma center care among trauma patients is important in evaluating and comparing different measures of physiologic compromise that can be used to inform triage decisions, but operationalizing and measuring the need for trauma care is challenging. Indicators that have been used or proposed include in-hospital mortality, measures of resource use (e.g., a published consensus-based criterion,12,22 lists of life-saving interventions,13,14 or a need for a single intervention such as major nonorthopedic surgery), and ratings of injury severity (e.g., the Injury Severity Score [ISS]). However, none of these indicators is perfect and their advantages and disadvantages need to be considered when evaluating the evidence base for the predictive utility of the various physiologic measures.
II. Scope and Key Questions
The scope of this review is limited to measures that assess the physiologic status (i.e., circulatory or respiratory compromise) of a trauma patient and that can be used in the field by out-of-hospital providers. The purpose of the measures is to identify patients likely to have serious injuries and use this information to inform decisions about whether an injured patient needs immediate transport to a trauma center. Measurements included in the review can be obtained by standard medical equipment or new devices specially designed for field assessment or monitoring. This review will provide a synthesis of currently available evidence about the performance of measures and identify gaps in evidence in order to inform guideline development, clinical practice, and future research.
Key Questions 1, 2, and 3 differ only in that they address the utility of different categories of physiologic measures (i.e., circulatory, respiratory, or combinations) for predicting the likelihood that a patient has a serious injury, requiring transport the highest level trauma center available. There are multiple ways to define serious injury and several indicators will be included in the review. These are listed in the PICOTs section.
The scope and Key Questions for this topic were initially developed by AHRQ in conjunction with the sponsoring partner agency, the National Highway Transportation Safety Administration (NHTSA). The questions were reorganized by the project team and revised after input from the Technical Expert Panel (TEP). There was no formal topic refinement for this review.
Key Question 1
For patients with known or suspected trauma who are treated out-of-hospital by EMS personnel, what is the predictive utility of measures of circulatory compromise (e.g., systolic blood pressure, mean arterial pressure, heart rate, heart rate complexity/variability) or derivative measures (e.g., the shock index) for predicting serious injury requiring transport to the highest level trauma center available?
- How does the predictive utility of the studied measures of circulatory compromise vary across age groups (e.g., children or the elderly)? Specifically, what age ranges and values for the different age ranges are supported by the evidence?
Key Question 2
For patients with known or suspected trauma who are treated out-of-hospital by EMS personnel, what is the predictive utility of measures of respiratory compromise, (e.g., ventilatory support, respiration rate, tissue O2 saturation, respiratory effort, measures of acidemia such as end-tidal CO2, lactate, or base deficit) for predicting serious injury requiring transport to the highest level trauma center available?
- How does the predictive utility of the studied measures of respiratory compromise vary across age groups (e.g., children or the elderly)? Specifically, what age ranges and values for the different age ranges are supported by the evidence?
Key Question 3
For patients with known or suspected trauma who are treated out-of-hospital by EMS personnel, what is the predictive utility for combinations of measures of respiratory and circulatory compromise together with or without measures of altered levels of consciousness (as defined by Glasgow coma scale or its components), for predicting serious injury requiring transport to the highest level trauma center available?
- How does the predictive utility of combinations of measures vary across age groups (e.g., children or the elderly)? Specifically, what age ranges and values for the different age ranges are supported by the evidence?
Using the PICOTS framework and a graphical analytic framework required adapting these tools as they were designed for and usually used for intervention studies. Our approach is informed by guidance related to frameworks in the Methods Guide for Systematic Reviews of Diagnostic Tests in addition to the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.23,24 We have included the standard PICOTS terms, but added detail to explain how we are using them for this review and we have added a legend and text to the graphical framework.
The PICOTS outlined below are the same across all three Key Questions except for the Intervention/Physiologic Measures.
Population refers to the patients who are the subjects in the studies to be included.
Studies of patients of any age with known or suspected trauma who require assessment of physiologic compromise by EMS out-of-hospital
Studies of patients with nontrauma conditions or illnesses, patients with burns or chemical exposures, healthy people, and animal studies, Studies of patients in which other assessments are used (e.g., type of injury) or in which the patient population is limited to a subgroup of patients defined as seriously injured.
- Studies in which the patient population is a priori restricted to patients with serious traumatic injuries.
- Studies in which all patients have injuries that can be assessed or would be defined as serious based on direct observation (e.g., an amputation)
Interventions (Physiologic Measures)
The intervention is usually the treatment or health service of interest that is being evaluated in terms of its impact on the population. In this review the physiologic measures are what are evaluated. This review will include any measure of circulatory or respiratory compromise or combination measures. Examples are provided for each Key Question; however, additional measures may be identified by the search.
- Key Question 1: Physiologic measures of circulatory compromise, including but not limited to systolic blood pressure, mean arterial pressure, heart rate, heart rate complexity/variability, or derivative measures such as the shock index
- Key Question 2: Physiologic measures of respiratory compromise or effort, including but not limited to respiration rate, tissue O2 saturation, respiratory effort, measure of acidemia (e.g., end-tidal CO2, lactate, base deficit), or advanced out-of-hospital airway intervention
- Key Question 3: Combinations of measures of respiratory and circulatory compromise with or without measures of altered levels of consciousness (as defined by Glasgow coma scale or its components)
- All Key Questions: Additional measures may be identified during the search and included based on input from clinical experts. Studies of newer devices that provide these or other measurements will be included if available and relevant.
In all cases measurement can be for a single point in time, change over time, or can be trends in the measure evaluated by a person or technology.
Clinical assessment or indicator of health status that is not a separate indicator or a combination indicator including a measure of circulatory or respiratory compromise (e.g., temperature, consciousness, eye tracking, musculoskeletal soundness, balance, blood glucose, orientation).
Comparisons and Outcomes
As this is not a review of intervention studies, the structure of the questions for the review as well as the questions posed by included studies are different. The Key Questions address how well measures of physiologic compromise identify trauma patients likely to have a serious injury requiring high-level trauma care.
We include two types of evaluations of measures: (1) studies of how well single measures predict severe injury; and (2) studies that compare the performance of two or more measures directly (head-to-head studies).
The end points or "outcomes" of interest are the predictive utility of the measures. We include three different approaches to assessing predictive utility: (1) adjusted risk estimates (e.g., odds ratio, relative risk, hazards ratio); (2) discrimination (e.g., area under the receiver operating characteristic curve [AUROC]); and (3) measures of diagnostic accuracy (e.g., sensitivity, specificity, positive predictive values, and negative predictive values).
The predictive utility is defined in terms of the physiologic measure's ability to identify patients who have severe injury. Defining and operationalizing what "severe injury" means is challenging for several reasons. Whether a patient had a serious injury at the time of field triage cannot be determined conclusively and we expect that clinical outcomes (e.g., death or disability) are affected by out-of-hospital and in-hospital treatment (i.e., a person can have a serious injury and recover). For this reason, we accept several indicators that a patient was seriously injured. These include outcomes, such as death, whether the patient required treatments and interventions used for serious injury, or whether the injury is rated as severe using accepted rating scales. It is possible the review will identify additional indicators that a patient had a severe injury; however the following list includes those that have been used in prior research.
Indicators of serious injury
- In-hospital mortality
- Resource use/intervention standards or lists
- Published Consensus-Based Criterion Standard — This list defines need for trauma center care as any one of the following 10 specific indicators: major surgery, advanced airway, blood products, admission for spinal cord injury, thoracotomy, pericardiocentesis, cesarean delivery, intracranial pressure monitoring, interventional radiology, and in-hospital death.12,22
- Need For Life-Saving Interventions — Lists used by the U.S. military that include angioembolization, blood transfusion, cardiopulmonary resuscitation, chest tube, intubation, needle decompression, surgical cricothyrotomy or thoracotomy, pericardiocentesis, angiography with embolization, angiography without and surgical intervention.13,14
- Major Surgery — Not including orthopedic surgery
- Ratings of Injury Severity — Injury Severity Score (ISS) >15, as this is a commonly used threshold for high risk patients, but other cut-offs will be considered if used in included studies. The ISS score is based on an assessment that divides the body into nine regions, classifies the level of injury in each of the three most severely injured regions on a scale of 1 to 6, squares these values, and adds them together.25,26
Physiological measures upon the arrival of EMS personnel to the scene of injury, during treatment in the field, and during transport (referred to as out-of-hospital or in the field). Studies with measures taken upon arrival at an emergency department will be considered. Details about timing of measurement will be recorded in data abstraction if they are reported.
- Studies measuring physiologic compromise in the field/out-of-hospital
- Studies of initial ED measurement as indirect evidence only if out-of-hospital evidence is not available and the measure is deemed clinically relevant
- Studies conducted in civilian or military settings
- Inpatient, clinic, or emergency department (ED)
- Studies conducted in developing countries with out-of-hospital care systems that differ from those in the United States.
- Any study that assesses the predictive utility of included measures either individually or that compares two or more measures. Designs may include trials and prospective and retrospective observational studies.
- Systematic reviews
- Nonsystematic reviews, commentaries, and letters
- Descriptions of the properties or performance of measures that do not include predictive utility
III. Analytic Framework
The analytic framework depicts the relationship among the major elements of the Key Questions as well as their expected relationships to intermediate and clinical outcomes, even though these relationships are not always included in the review. In Figure 2 the solid lines indicate the scope of the review and the dashed lines represent important relationships that are assumed to exist but are outside the scope.
This review is limited to considering how well the physiologic measures predict serious injury in trauma patients evaluated by EMS. These are the components connected with solid lines. The assumption is that being able to distinguish seriously injured patients will inform triage and transport decisions and these decisions will impact care for the injury which will affect outcomes. These relationships are represented with dashed lines as they are important assumptions, but not part of the review.
Figure 2: Analytic framework
*Defined by inpatient mortality, resource use (e.g., the published consensus-based criterion standard, need for life-saving interventions, major surgery), or ISS >15
EMS = emergency medical services; KQ = Key Question
Solid lines = relationships within the scope; dashed lines = assumed relationships outside the scope of the review
Criteria for Inclusion/Exclusion of Studies in the Review
The criteria for inclusion and exclusion of studies will be based on the Key Questions and are described in the PICOTS section above. Below are additional details on the scope of this project:
We will include any study that evaluates a measure that meets our inclusion criteria. The studies maybe randomized trials or prospective or retrospective observational studies. We will include studies of individual measures as well as studies that compare multiple measures directly. If we identify a large number of studies, we will prioritize the studies based on the rigor of the study design and the risk of bias.27 If systematic reviews are identified, we will consider whether their results can be included in the review and integrated with other primary studies based on how well they match our Key Questions and their methodological quality.28 At a minimum, all systematic reviews will be considered as sources of studies to be reviewed for possible inclusion.
We will restrict inclusion to English-language articles, but will review English-language abstracts of non-English-language articles to identify studies that would otherwise meet inclusion criteria and assess the likelihood of language bias.
Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions
Publication Date Range
We will include studies with publication date from 1996 to November 2016 in the initial search and the search will be updated during the public comment and peer review period. This search start date was selected because trauma care has changed over time; only rudimentary measures and monitors existed prior to 1996 and only five states had fully implemented trauma systems in the early 1990's.29
MEDLINE®, CINAHL®, Embase®, and the Cochrane databases will be searched to capture both published and grey literature. The search strategies are provided in Appendix A.
Notice may be posted in the Federal Register requesting published and unpublished evidence relevant to the review, and all submissions will be reviewed according to the criteria and processes for all evidence as described in this protocol. Recently released devices are commercially available, including heart rate and tissue perfusion monitors, and information about measurements obtained using these devices will be included if provided. Data obtained through this process need to meet the same inclusion/exclusion criteria as other evidence included in the review; these reports/articles must also contain information on methodology so that the risk of bias can be assessed.
Reference lists of included articles will be reviewed for additional relevant citations.
Sources for unpublished literature will include any responses to the requests that are received, white papers and information posted on websites of key organizations such as the American College of Surgeons, and searches of the ClinicalTrials.gov trial registry to identify trials that are in progress or have been recently completed and may not yet have published results.
In the event information regarding methods or results appears to be omitted from the published results of a study, or if we are aware of unpublished data, we will query the authors to obtain additional information.
Process for Selecting Studies
Pre-established criteria will be used to determine eligibility for inclusion and exclusion of abstracts in accordance with the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.24 To ensure accuracy, all excluded abstracts will be independently reviewed by at least two reviewers. To avoid bias or the appearance of bias, team members involved in research studies on this topic will not triage studies. All citations deemed appropriate for inclusion by at least one of the reviewers will be retrieved for full-text review. Each full-text article will be independently reviewed for eligibility by at least two investigators, including any articles suggested by peer reviewers or that arise from the public posting or Scientific Information Packet processes. Any disagreements will be resolved by discussion and consensus across the investigators.
Data Abstraction and Data Management
For studies that meet inclusion criteria, the following data will be abstracted: study design, year, setting, geographic location, sample size, data source, timing of data collection, mode of data collection, eligibility criteria, and population demographic and clinical characteristics (e.g., patient, injury and personnel types). Potential overlap across data sources will be identified as trauma registries are an important source of data in this field. We will also abstract how the included studies evaluate the measures, the measure characteristics, and the results. We will include information on the timing of the measurements of physiologic compromise and for the indicators of high risk of severe injury used to operationalize need for trauma care. Abstracted information relevant for assessing applicability will include the number of patients randomized/eligible for inclusion in an observational study relative to the number of patients enrolled, and characteristics of the population, intervention, setting, and administering personnel. Sources of funding for all studies will also be recorded.
All abstracted study data will be verified for accuracy and completeness by a second team member. A record of studies excluded at the full-text level with reasons for exclusion will be maintained.
Assessment of Methodological Risk of Bias of Individual Studies
Predefined criteria will be used to assess the risk of bias for individual controlled trials and for observational studies by using clearly defined templates and criteria. Studies will be evaluated using appropriate study-design specific criteria24 or an appropriate tool for studies that evaluate, measure, or aim to establish thresholds such as the Quality in Prognosis Studies (QUIPS) tool.30 The QUIPS tool includes domains on study participation, study attrition, prognostic factor measurement, outcomes measurement, study funding, and statistical analysis and reporting. These criteria and methods will be used in conjunction with the approach recommended in the chapter, Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.31 Studies will be rated as "low risk of bias," "medium risk of bias," or "high risk of bias."
Studies rated low risk of bias are considered to have the least risk of bias, and their results are generally considered valid. Low risk of bias studies include clear descriptions of the population, setting, and measures; sufficient description of how the measure was executed and instrumentation; how the measure was interpreted; if specific threshold values were used; and how the risk of serious injury was evaluated.
Studies rated medium risk of bias are susceptible to some bias, though not enough to invalidate the results. These studies may not meet all the criteria for a rating of low risk of bias, but no flaw is likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems. The medium risk of bias category is broad, and studies with this rating will vary in their strengths and weaknesses. The results of some medium risk of bias studies are likely to be valid, while others may be only possibly valid.
Studies rated high risk of bias have significant flaws that imply biases of various types that may invalidate the results. They have a serious or "fatal" flaw in design, analysis, or reporting; large amounts of missing information; discrepancies in reporting; or serious problems in the delivery of the intervention. In general, observational studies that do not perform adjustment for potential confounders will be assessed as high risk of bias. The results of these studies are at least as likely to reflect flaws in the study design as the true difference between the compared interventions. We will not exclude studies rated high risk of bias a priori, but high risk of bias studies will be considered to be less reliable than low or medium risk of bias studies when synthesizing the evidence, particularly if discrepancies between studies are present.
Each study evaluated will be independently reviewed for risk of bias by two team members. Any disagreements will be resolved by consensus. Team members who were involved in the conduct of a study will not be involved in triage, data abstraction, or risk of bias assessment for that study.
Data will be summarized in tables, using descriptive statistics and narrative text. We will assess whether quantitative synthesis (i.e., meta-analysis) is possible based on the quantity and quality of the data from included studies, and we will provide a qualitative synthesis if meta-analysis is not advisable. It is likely we will identify several different measures and several different indicators of serious injury. Lists of what is expected are provided in the PICOTS section. We will consult with the clinical experts on our team, local experts, and TEP members to determine what measures and indicators are sufficiently similar to allow grouping.
For each measure identified in the review we will consider predictive utility evaluated in terms of (1) adjusted risk estimates, (2) area under the receiver operating characteristic curve (AUROC), and (3) diagnostic accuracy. We will assess whether each type of measure can be pooled and apply the methods appropriate to the type of data and approach to predictive utility.
Adjusted risk estimates may include odds ratios, risk ratios, and hazard ratios. We will report point estimates and confidence intervals or significance and record reported values and counts. We will record what was adjusted for in each study and group studies by the measure they evaluate and the indicator of high risk of serious injury used to determine if studies are similar enough to be pooled. The feasibility of a quantitative synthesis will depend on the number and completeness of reported outcomes and amount of heterogeneity among the included studies. To determine whether meta-¬analysis could be meaningfully performed, we will consider the risk of bias for each of the studies and the heterogeneity among studies in design, patient population, the measures studied and how the measures are used (e.g., monitoring or recording technology and use of the same thresholds or ranges of values). We will perform meta-analysis using random effects models only if evidence is suitable for combining, based on similarities in the populations, interventions, comparisons, and settings evaluated.34
For studies that report AUROCs, which are measures of discrimination that consider the sensitivity and specificity across a range of values, we will report the AUROCs and ranges from studies of individual measures. If studies directly compare two measures and report differences in AUROCs, we will calculate pooled differences. We will use different approaches to this synthesis (e.g., DerSimonian-Laid random effects model and Profile Likelihood method) and compare results.
Measures of diagnostic accuracy include sensitivity, specificity, and predicted values. We will report the values cited in articles and confirm these if counts are provided. We will also identify what thresholds were used in individual studies or if data were reported that would allow medians and ranges for sensitively and specificity to be calculated across studies as well as whether it is possible for positive and negative likelihood ratios to be modeled for included measures.
We will stratify the primary analyses by age of patients to address the sub Key Questions that are about the performance of the tests and thresholds in different age groups. We will consider additional subgroup analysis by other variables such as the age of the studies, risk of bias ratings, geographic characteristics (e.g., country; urban vs. rural), type of injury, level of EMS personnel, timing of the measurements (e.g., at EMS arrival on scene vs. ED arrival), or source of the data (e.g., registries or individual sites).
Grading the Strength of Evidence for Major Comparisons and Outcomes
The strength of evidence (SOE) for each Key Question will be initially assessed by one researcher for the predictive utility of each identified measure paired with each indicator of the need for trauma care in accordance with the approaches described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews24 and the guidance for diagnostic tests.35 To ensure consistency and validity of the evaluation, the grades will be reviewed by the entire team of investigators for:
- Study limitations (low, medium, or high level of study limitations)
- Consistency (consistent, inconsistent, or unknown/not applicable)
- Directness (direct or indirect)
- Precision (precise or imprecise)
- Reporting bias (suspected or undetected).
The strength of evidence will be assigned an overall grade of high, moderate, low, or insufficient according to a four-level scale by evaluating and weighing the combined results of the above domains:
- High—We are very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. We believe that the findings are stable (i.e., another study would not change the conclusions).
- Moderate—We are moderately confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. We believe that the findings are likely to be stable, but some doubt remains.
- Low—We have limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). We believe that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
- Insufficient—We have no evidence, we are unable to estimate an effect, or we have no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.
Applicability is the extent to which the findings in published studies are likely to reflect the results when the measures are used to evaluate trauma patients in similar situations. will be considered according to the approach described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews24 and the guidance for systematic reviews of diagnostic tests.36 We will use the PICOTS framework to consider the applicability of the evidence base for each Key Question, for example, examining the characteristics of the patient populations (e.g., age and type of trauma) and triage situation (e.g., characteristics of the EMS personnel and the environment) as well as how the measures of physiologic compromise are obtained and used (e.g., use of different monitors or threshold values). Variability in the studies may limit the ability to apply the results to other populations and setting.
Managing Bias and the Appearance of Bias
The review team and TEP include experts who have conducted and published research in this field. In order to avoid bias or the appearance of bias we will take the following steps. First, authors will not be involved in any decisions about including or excluding their own work. Second, to the extent it is feasible, reviewers will be blinded to authors during title and abstract review so that the other team members/reviewers are not biased in favor of colleagues. Third, for full-text review no one will be assigned to review research they contributed to, but the two reviewers will not be blinded to the authors. However, if there are articles that require a third reviewer or that need to be discussed with the team to achieve consensus, the additional reviewers will be blinded to the authors of the article in question. Finally, team members and experts will not rate the risk of bias or abstract data from studies to which they contributed.
- Centers for Disease Control and Prevention. National Hospital Ambulatory Medical Care Survey: 2011 Emergency Department Summary Tables. http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/2011_ed_web_tables.pdf. Accessed May 5, 2016.
- Soreide K. Strengthening the trauma chain of survival. Br J Surg. 2012 Jan;99 Suppl 1:1-3. doi: 10.1002/bjs.7795. PMID: 22441846.
- National Emergency Medical Services Workforce Data Definitions. (Prepared by the American Institutes for Research through contract DTNH22-080F-00122.) National Highway Traffic Safety Administration. Washington, DC: February 2013. www.nhtsa.gov/staticfiles/nti/pdf/811720.pdf.
- Demetriades D, Martin M, Salim A, et al. Relationship between American College of Surgeons trauma center designation and mortality in patients with severe trauma (injury severity score > 15). J Am Coll Surg. 2006 Feb;202(2):212-5; quiz A45. doi: 10.1016/j.jamcollsurg.2005.09.027. PMID: 16427544.
- Sampalis JS, Denis R, Frechette P, et al. Direct transport to tertiary trauma centers versus transfer from lower level facilities: impact on mortality and morbidity among patients with major trauma. J Trauma. 1997 Aug;43(2):288-95; discussion 95-6. PMID: 9291375.
- Haas B, Gomez D, Zagorski B, et al. Survival of the fittest: the hidden cost of undertriage of major trauma. J Am Coll Surg. 2010 Dec;211(6):804-11. doi: 10.1016/j.jamcollsurg.2010.08.014. PMID: 21036070.
- MacKenzie EJ, Rivara FP, Jurkovich GJ, et al. A national evaluation of the effect of trauma-center care on mortality. N Engl J Med. 2006 Jan 26;354(4):366-78. doi: 10.1056/NEJMsa052049. PMID: 16436768.
- Faul M, Wald MM, Sullivent EE, et al. Large cost savings realized from the 2006 Field Triage Guideline: reduction in overtriage in U.S. trauma centers. Prehosp Emerg Care. 2012 Apr-Jun;16(2):222-9. doi: 10.3109/10903127.2011.615013. PMID: 22008012.
- Lerner EB, Shah MN, Swor RA, et al. Comparison of the 1999 and 2006 trauma triage guidelines: where do patients go? Prehosp Emerg Care. 2011 Jan-Mar;15(1):12-7. doi: 10.3109/10903127.2010.519819. PMID: 21054176.
- Sasser SM, Hunt RC, Faul M, et al. Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011. MMWR Recomm Rep. 2012 Jan 13;61(RR-1):1-20. PMID: 22237112.
- Mackersie RC. History of trauma field triage development and the American College of Surgeons criteria. Prehosp Emerg Care. 2006 Jul-Sep;10(3):287-94. doi: 10.1080/10903120600721636. PMID: 16801263.
- Newgard CD, Fu R, Zive D, et al. Prospective Validation of the National Field Triage Guidelines for Identifying Seriously Injured Persons. J Am Coll Surg. 2016 Feb;222(2):146-58 e2. doi: 10.1016/j.jamcollsurg.2015.10.016. PMID: 26712244.
- Convertino VA, Ryan KL, Rickards CA, et al. Physiological and medical monitoring for en route care of combat casualties. J Trauma. 2008 Apr;64(4 Suppl):S342-53. doi: 10.1097/TA.0b013e31816c82f4. PMID: 18385586.
- Liu NT, Holcomb JB, Wade CE, et al. Improving the prediction of mortality and the need for life-saving interventions in trauma patients using standard vital signs with heart-rate variability and complexity. Shock. 2015 Jun;43(6):549-55. doi: 10.1097/SHK.0000000000000356. PMID: 25692260.
- Woodford MR, Mackenzie CF, DuBose J, et al. Continuously recorded oxygen saturation and heart rate during prehospital transport outperform initial measurement in prediction of mortality after trauma. J Trauma Acute Care Surg. 2012 Apr;72(4):1006-11. doi: 10.1097/TA.0b013e318241c059. PMID: 22491618.
- Traverso G, Ciccarelli G, Schwartz S, et al. Physiologic status monitoring via the gastrointestinal tract. PLoS One. 2015;10(11):e0141666. doi: 10.1371/journal.pone.0141666. PMID: 26580216.
- Kehoe A, Rennie S, Smith JE. Glasgow Coma Scale is unreliable for the prediction of severe head injury in elderly trauma patients. Emerg Med J. 2015 Aug;32(8):613-5. doi: 10.1136/emermed-2013-203488. PMID: 25280479.
- Zarzaur BL, Croce MA, Magnotti LJ, et al. Identifying life-threatening shock in the older injured patient: an analysis of the National Trauma Data Bank. J Trauma. 2010 May;68(5):1134-8. doi: 10.1097/TA.0b013e3181d87488. PMID: 20453769.
- Price CL, Brace-McDonnell SJ, Stallard N, et al. Performance characteristics of five triage tools for major incidents involving traumatic injuries to children. Injury. 2016 May;47(5):988-92. doi: 10.1016/j.injury.2015.10.076. PMID: 26653268.
- Bruijns SR, Guly HR, Bouamra O, et al. The value of traditional vital signs, shock index, and age-based markers in predicting trauma mortality. J Trauma Acute Care Surg. 2013 Jun;74(6):1432-7. doi: 10.1097/TA.0b013e31829246c7. PMID: 23694869.
- Chuang JF, Rau CS, Wu SC, et al. Use of the reverse shock index for identifying high-risk patients in a five-level triage system. Scand J Trauma Resusc Emerg Med. 2016;24:12. doi: 10.1186/s13049-016-0208-5. PMID: 26861172.
- Lerner EB, Willenbring BD, Pirrallo RG, et al. A consensus-based criterion standard for trauma center need. J Trauma Acute Care Surg. 2014 Apr;76(4):1157-63. doi: 10.1097/TA.0000000000000189. PMID: 24662885.
- Samson D, Schoelles KM. Chapter 2: medical tests guidance (2) developing the topic and structuring systematic reviews of medical tests: utility of PICOTS, analytic frameworks, decision trees, and other frameworks. J Gen Intern Med. 2012 Jun;27 Suppl 1:S11-9. doi: 10.1007/s11606-012-2007-7. PMID: 22648670.
- Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication No. 10(14)-EHC063-EF. Rockville, MD: Agency for Healthcare Research and Quality; January 2014. Chapters available at: www.effectivehealthcare.ahrq.gov.
- Baker SP, O'Neill B, Haddon W, Jr., et al. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974 Mar;14(3):187-96. PMID: 4814394.
- Copes WS, Champion HR, Sacco WJ, et al. The Injury Severity Score revisited. J Trauma. 1988 Jan;28(1):69-77. PMID: 3123707.
- Treadwell JR, Singh S, Talati R, et al. A framework for best evidence approaches can improve the transparency of systematic reviews. J Clin Epidemiol. 2012 Nov;65(11):1159-62. doi: 10.1016/j.jclinepi.2012.06.001. PMID: 23017634.
- Robinson KA, Chou R, Berkman ND, et al. Integrating Bodies of Evidence: Existing Systematic Reviews and Primary Studies. Methods Guide for Comparative Effectiveness Reviews (Prepared by the Scientific Resource Center under Contract No. 290-2012-00004-C). AHRQ Publication No. 15-EHC007-EF. Rockville, MD: Agency for Healthcare Research and Quality; February 2015. www.effectivehealthcare.ahrq.gov/reports/final.cfm Accessed November 17, 2016.
- Bazzoli GJ, Madura KJ, Cooper GF, et al. Progress in the development of trauma systems in the United States. Results of a national survey. JAMA. 1995 Feb 1;273(5):395-401. PMID: 7823385.
- Hayden JA, van der Windt DA, Cartwright JL, et al. Assessing bias in studies of prognostic factors. Ann Intern Med. 2013 Feb 19;158(4):280-6. doi: 10.7326/0003-4819-158-4-201302190-00009. PMID: 23420236.
- Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality Methods Guide for Effectiveness and Comparative Effectiveness Reviews. March 2012. AHRQ Publication No. 12-EHC047-EF. Available at: www.effectivehealthcare.ahrq.gov/
- Trikalinos TA, Balion CM. Chapter 9: Options for summarizing medical test performance in the absence of a "gold standard". J Gen Intern Med. 2012 Jun;27 Suppl 1:S67-75. doi: 10.1007/s11606-012-2031-7. PMID: 22648677.
- Takwoingi Y, Leeflang MM, Deeks JJ. Empirical evidence of the importance of comparative studies of diagnostic test accuracy. Ann Intern Med. 2013 Apr 02;158(7):544-54. doi: 10.7326/0003-4819-158-7-201304020-00006. PMID: 23546566.
- Trikalinos TA, Balion CM, Coleman CI, et al. Chapter 8: Meta-analysis of test performance when there is a "gold standard". J Gen Intern Med. 2012 Jun;27 Suppl 1:S56-66. doi: 10.1007/s11606-012-2029-1. PMID: 22648676.
- Singh S, Chang SM, Matchar DB, et al. Chapter 7: Grading a body of evidence on diagnostic tests. J Gen Intern Med. 2012 Jun;27 Suppl 1:S47-55. doi: 10.1007/s11606-012-2021-9. PMID: 22648675.
- Hartmann KE, Matchar DB, Chang S. Chapter 6: Assessing applicability of medical test studies in systematic reviews. J Gen Intern Med. 2012 Jun;27 Suppl 1:S39-46. doi: 10.1007/s11606-011-1961-9. PMID: 22648674.
VI. Definition of Terms
VII. Summary of Protocol Amendments
|Date||Section||Original Protocol||Revised Protocol||Rationale|
|5/31/17||Key Question 3||Key Question 3: For patients with known or suspected trauma who are treated in the prehospital setting by EMS personnel, what is the evidence that combinations of measures of respiratory and circulatory compromise together with or without measures of altered levels of consciousness (as defined by Glasgow coma scale or its components) can predict the need for transport to a trauma center?||Key Question 3: For patients with known or suspected trauma who are treated in the prehospital setting by EMS personnel, what is the evidence that scales combining (a) measures of respiratory and circulatory compromise or (b) measures of respiratory and/or circulatory compromise together with measurement of altered levels of consciousness (as defined by Glasgow coma scale or its components) can predict the need for transport to a trauma center?||The original wording is limiting in that it would require that the combination measure include both a circulatory measure and a respiratory measure and then the GCS is optional. The issue is that this would exclude studies of measures that combine GCS with a circulatory or respiratory measure, but not both (e.g., a measure that combines GCS and blood pressure). We have identified studies like this and do not want to exclude them.|
|5/31/17||PICOTS, Comparisons and Outcomes||The end points or “outcomes” of interest are the predictive utility of the measures. We include three different approaches to assessing predictive utility: (1) adjusted risk estimates (e.g., odds ratio, relative risk, hazards ratio); (2) discrimination (e.g., area under the receiver operating characteristic curve [AUROC]); and (3) measures of diagnostic accuracy (e.g., sensitivity, specificity, positive predictive values, and negative predictive values).||The end points or “outcomes” of interest are the predictive utility of the measures. We include three different approaches to assessing predictive utility: (1) adjusted risk estimates (e.g., odds ratio, relative risk, hazards ratio); (2) discrimination (e.g., area under the receiver operating characteristic curve [AUROC]); and (3) measures of diagnostic accuracy (e.g., sensitivity, specificity, positive predictive values, and negative predictive values). Studies that provide only descriptive information, unadjusted risk estimates, or assessments of continuous variables (e.g., correlations or tests of differences in means) will be excluded unless data is provided that can be used to calculate included outcomes.||We added a sentence to clarify criteria for excluding studies due to insufficient data for included outcomes.|
VIII. Technical Experts
Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and suggest approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor do they contribute to the writing of the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The AHRQ TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
IX. Peer Reviewers
Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published three months after the publication of the evidence report.
Potential Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.
X. EPC Team Disclosures
EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.
XI. Role of the Funder
This project was funded under Contract No. HHSA290201500009I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The AHRQ Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.
This protocol will be registered in the International Prospective Register of Systematic Reviews (PROSPERO).
Database: Ovid MEDLINE(R) without Revisions 1996 to November Week 5 2016
- exp "Wounds and Injuries"/
- (prehospital or trauma or traumatic).mp.
- exp Emergency Medical Services/
- (EMS or ambulance or transport* or triage).mp.
- (1 or 2) and (3 or 4)
- exp Vital Signs/
- exp Shock/
- exp "circulatory and respiratory physiological phenomena"/
- ("systolic blood pressure" or SBP or "mean arterial pressure" or "heart rate" or "shock index").mp.
- (airway and (intervention or management)).mp.
- (respira* and (rate or effort)).mp.
- ("tissue oxygen saturation" or "end-tidal" or "lactate").mp.
- 5 and 13
- limit 14 to humans
Database: EBM Reviews - Cochrane Central Register of Controlled Trials November 2016
- exp "Wounds and Injuries"/
- (prehospital or trauma or traumatic).mp.
- exp Emergency Medical Services/
- (EMS or ambulance or transport* or triage).mp.
- (1 or 2) and (3 or 4)
- exp Vital Signs/
- exp Shock/
- exp "circulatory and respiratory physiological phenomena"/
- ("systolic blood pressure" or SBP or "mean arterial pressure" or "heart rate" or "shock index").mp.
- (airway and (intervention or management)).mp.
- (respira* and (rate or effort)).mp.
- ("tissue oxygen saturation" or "end-tidal" or "lactate").mp.
- 5 and 13
Database: EBM Reviews - Cochrane Database of Systematic Reviews 2005 to December 07, 2016
- ("systolic blood pressure" or SBP or "mean arterial pressure" or "heart rate" or "shock index").mp. [mp=title, abstract, full text, keywords, caption text] (1001)
- (airway and (intervention or management)).mp. [mp=title, abstract, full text, keywords, caption text] (787)
- (respira* and (rate or effort)).mp. [mp=title, abstract, full text, keywords, caption text] (2033)
- ("tissue oxygen saturation" or "end-tidal" or "lactate").mp. [mp=title, abstract, full text, keywords, caption text] (203)
- or/1-4 (2845)
- (prehospital or pre-hospital or trauma or traumatic or EMS or ambulance or transport* or triage).ti. (119)
- 5 and 6 (23)
Database: CINAHL Plus with Full Text
- S1 (MH "Wounds and Injuries+")
- S2 (MH "Trauma+")
- S3 (MH "Prehospital Care")
- S4 (MH "Emergency Medical Services+")
- S5 (MH "Transportation of Patients+")
- S6 S1 OR S2
- S7 S3 OR S4 OR S5
- S8 S6 AND S7
- S9 (MH "Cardiopulmonary Physiology+")
- S10 (MH "Respiratory Tract Physiology+")
- S11 S9 OR S10
- S12 S8 AND S11
- S13 S12 Limiters - Published Date: 19960101-20161231
Database: Elsevier Embase
((('injury'/exp or prehospital:ab,ti or trauma:ab,ti or traumatic:ab,ti) and ('emergency health service'/exp or ems:ab,ti or ambulance:ab,ti or transport*:ab,ti)) and ('vital sign'/exp or 'shock'/exp or 'cardiovascular function'/exp)) and [embase]/lim not [medline]/lim and ('article'/it or 'article in press'/it or 'review'/it) and (1996:py or 1997:py or 1998:py or 1999:py or 2000:py or 2001:py or 2002:py or 2003:py or 2004:py or 2005:py or 2006:py or 2007:py or 2008:py or 2009:py or 2010:py or 2011:py or 2012:py or 2013:py or 2014:py or 2015:py or 2016:py or 2017:py) and 'human'/de