Skip to main content
Effective Health Care Program
Home » Products » Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication » Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication

Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication

Research Protocol

I. Background and Objectives for the Systematic Review

Background

Almost 1/5 of the population of the United States lives in rural areas. Health disparities for people living in rural areas are well studied1 and include higher mortality2-5 and morbidity from a wide range of conditions such as substance/opioid abuse,6,7 chronic illnesses,8-11 and HIV/human papillomavirus (HPV) and other infectious diseases.12,13 Rural-urban health inequities have been extensively documented and the subject of research for several decades,2,14,15 persist today despite this recognition, and may be amplified as the COVID-19 pandemic reaches rural areas,16 potentially increasing risk of morbidity and mortality17,18 and impacting access to health services.19-21 Underlying causes are complex and varied as they are related to macro and micro sociologic-demographic forces22 and economic trends.23,24 Research on health disparities continues to evolve from first describing the extent of disparities, to attempts to understand the underlying reasons disparities exist, and finally to the current focus on identifying and implementing interventions to reduce disparities.25-27 The need for interventions to reduce disparities has garnered additional attention due to both the pandemic and increased attention to structural racism and its impact on both access to care and health outcomes.

Telehealth is the use of information and telecommunications technology to provide health care across time and/or distance.28 It is a tool with the potential to increase access, improve the quality of care, increase patient satisfaction, positively impact patient outcomes, and reduce the cost of care. Telehealth includes using technology to directly deliver care but it can encompass broader applications of technologies to health care functions such as consultation, distance education, or mentoring, monitoring and data collection as well as consumer outreach.29 Provider-to-provider telehealth includes using technology to support clinicians through mechanisms including consultations, mentoring, and education with the goal of improving the care they provide to individual patients or patient populations. 

Application of telehealth had been steadily increasing in many areas of healthcare before the COVID-19 pandemic30 began in 2020. The potential benefits of telehealth are frequently cited,31,32 and there is a sizable body of research on telehealth, including systematic reviews and reviews of reviews.33-38 Yet implementation and spread has been slow.39-41 Nevertheless, telehealth adoption appeared to be accelerating with improvement in technologies42 and expansion of coverage by both public and private payers prior to the pandemic. The increase in use has been accompanied by an increase in the research and published literature on telehealth. Use of provider-to- provider telehealth for consultations has been studied across a range of clinical indications43 including specialty care,44 acute/emergency care,45 and intensive care.46 Additionally, other forms of provider-to-provider telehealth such as distance learning47 and Project ECHO48,49 that combines education with review of specific cases are increasingly the subject of study. 

The COVID-19 pandemic created an urgent need to provide care while minimizing exposure and maximizing resources. Growth in the use of telehealth has been exponential, spurred by this need and supported by temporary changes in payment and regulation.50 Documentation of implementation and research on the effectiveness of this recent expansion of telehealth is now becoming available, though with the time lags necessary for data collection and for the measurement of patient-centered outcomes in addition to intermediate, process measures (e.g., numbers of telehealth encounters). 

Identifying and synthesizing the available evidence about the use of provider-to-provider telehealth as a means of addressing rural health disparities could support ongoing spread, conversion of telehealth friendly pandemic policies to permanent support, and the identification of potential new areas and approaches for the expansion of telehealth in rural America.51 

Purpose of the Review

This systematic review of provider-to-provider telehealth for rural patients and populations is intended to address a number of decisional dilemmas, including areas of uncertainty and lack of accessible evidence to support clinical and policy decision-making. This review will identify and synthesize the literature on telehealth-guided clinical decision-making by healthcare providers for rural patients, telehealth-associated clinical outcomes, the benefits and unintended consequences of use of telehealth, and the effectiveness of implementation and sustainability strategies for provider-to-provider telehealth in rural areas. 

The review will first identify and synthesize the literature regarding the use of telehealth technologies to support provider-to-provider consultation in rural areas. In addition, the report will identify known facilitators, barriers and strategies that are effective in promoting the adoption, implementation and sustainability of provider-to-provider telehealth for rural patients and populations. Finally, the project will systematically categorize and summarize gaps in the evidence and the strengths and weaknesses of study designs to inform the design and conduct of future research. 

While this review is intended for a broad audience of stakeholders interested in telehealth as an approach to address rural health disparities and improve the health and wellbeing of rural populations, it will also inform the Pathways to Prevention workshop: Improving Rural Health through Telehealth-guided Provider-to-Provider Communication that is being developed by the National Institutes of Health (NIH) Office of Disease Prevention (ODP) in collaboration with the National Center for Advancing Translational Sciences (NCATS); National Heart, Lung, and Blood Institute (NHLBI); Federal Office of Rural Health Policy (FORHP) at the Health Resources and Services Administration; and the Office of the Associate Director for Policy and Strategy, Centers for Disease Control and Prevention (CDC) [hereafter referred to as the NIH/ODP Working Group].

II. Key Questions

The Key Questions (KQ) for this systematic review are based on questions provided in the scope of work that accompanied the Request for Task Order. The questions were reviewed, reorganized, and refined by the systematic review project team and further revised after input from the AHRQ Task Order Officer (TOO) and the NIH/ODP Working Group. The review is defined by two overarching key questions, the first focusing on evidence about clinical effectiveness and the second focusing on evidence of the effectiveness of implementation strategies.

Key Questions for the Systematic Review

Key Question 1. What is the effectiveness of provider-to-provider telehealth for rural patients?

  1. What is the impact of provider-to-provider telehealth on rural patient and population outcomes?
  2. What is the impact of provider-to-provider telehealth on healthcare providers?
  3. What is the impact of provider-to-provider telehealth on private and public payers?
  4. What adverse events or unintended consequences are associated with provider-to-provider telehealth for rural patients?
  5. What are the methodological weaknesses of the identified effectiveness studies of provider-to-provider telehealth for rural patients and what improvements in study design (e.g., focus on relevant comparisons and outcomes) might increase the impact of future research?

Key Question 2. What is the effectiveness of implementation strategies for provider-to-provider telehealth in rural areas?

  1. What is the uptake of different types of provider-to-provider telehealth in rural areas?
    • Who are the current patients, providers, and payers engaged in provider-to-provider telehealth in rural areas?
    • What factors affect whether provider-to-provider telehealth in rural areas can be sustained? 
  2. Which barriers and facilitators impact adoption and implementation of provider-to-provider telehealth in rural areas? 
  3. Which strategies are effective in sustaining provider-to-provider telehealth in rural areas? 
  4. What are the methodological weaknesses of the identified studies of implementation and sustainability of provider-to-provider telehealth in rural areas and what improvements in study design (e.g., focus on relevant comparisons and outcomes) might increase the impact of future research?

PICOS

The populations, interventions, comparators, outcomes, and settings (PICOS) of interest are described below. Additional details are outlined in the inclusion and exclusion criteria in Table 1.

  • Population(s):
    • Rural individual patients, patient families/care partners, and patient populations
    • Healthcare providers (individuals and organizations) who provide health care services to rural patients or populations
      • Providers include any profession or occupation providing formal, paid services
      • Family or informal care partners are not considered providers
    • Payers who pay for healthcare services for rural patients or populations
  • Interventions:
    • Provider-to-provider telehealth defined as: any telecommunications facilitated interaction among, or support for, healthcare professionals designed to improve access, quality of care, or health outcomes for rural patients and populations
  • Comparators:
    • KQ1: other telehealth facilitated care (not provider-to-provider), usual (in-person) provider-to-provider supports, no interaction or no care 
    • KQ2: different strategies for dissemination, implementation, or spread; no strategies; time periods prior to implementation 
  • Outcomes:
    • KQ1: Clinical outcomes for the identified conditions (patient-reported outcomes, mortality, morbidity, such as function, illness recovery, infection); Economic outcomes such as return on investment, cost, volume of visits, and resource use, including length of stay; Intermediate Outcomes; Patient satisfaction, behavior, and decisions such as completion of treatment, or satisfaction with less travel to access healthcare; Provider satisfaction, behavior, and decisions such as choice of treatment or antibiotic stewardship; Access measure and indicators including but not limited to time to diagnosis or time to treatment
    • KQ2: Indicators and measures of uptake (e.g., rates of use, timing to implementation) and characteristics of users; categories and descriptors of barriers and facilitators; categories and descriptors of strategies
  • Settings:
    • Outpatient (primary care and specialty care), inpatient, prehospital and emergency care, post-acute and long-term care
    • Civilian, Veterans Administration, or military
    • Health care and non-healthcare settings where health services are delivered
    • U.S. relevant settings

III. Analytic Framework

Figure 1. Analytic framework for Improving Rural Health Through Telehealth-Guided Provider-to-Provider Communication

The analytic framework represents the relationships among the elements in the Key Questions. Rural patients and populations, providers, and payers are in boxes at the left side of the figure, as these are the populations impacted by the intervention. Arrows connect these three boxes to a single box to the right that represents the intervention provider-to-provider telehealth (Key Question 2a). Above the intervention box are two boxes that represent elements of provider-to-provider telehealth. The first box represents barriers and facilitators (Key Questions 2b and 2d) and the second box represents implementation strategies (Key Questions 2c and 2d). There are arrows from these boxes to the intervention box. An oval below the intervention box represents adverse events and unintended consequences. There is an arrow from the intervention box to this oval (Key Question 1d). An arrow connects the intervention provider-to-provider telehealth box to a box with rounded corners that represents intermediate outcomes for rural patients, providers, and payers (Key Questions 1a, 1b, 1c and 1e). From the intermediate outcomes box, there is a dashed line connecting to a box representing outcomes for rural patents, providers and payers. There is also an overarching arrow connecting directly from the intervention provider-to-provider telehealth box to the outcomes box, bypassing the intermediate outcomes box (Key Questions 1a, 1b, 1c, and 1e).

IV. Methods

Criteria for Inclusion/Exclusion of Studies in the Review

The criteria for inclusion and exclusion of individual studies are based on the Key Questions and PICOS described above. Additional details on the scope of this project are provided below and the inclusion and exclusion criteria are outlined in Table 1.

Study Designs: We will include comparative studies of any design including comparative trials and observational studies. We will include observational cohort studies, as well as before-after designs (i.e., the comparison can be across time points). We will exclude descriptive studies with no outcomes data or studies that include only data from one point in time (post only). We will also exclude modeling studies or studies that use synthetic data. We will access existing systematic reviews, and include their results if appropriate. At a minimum, we will use systematic reviews to identify studies. We will also exclude commentaries, letters, and articles that describe telehealth systems or implementations but do not assess impact.

Non-English-Language Studies: We will restrict to English language articles, but will review English language abstracts of non-English language articles to identify studies that would otherwise meet inclusion criteria, in order to assess for the likelihood of language bias.

Table 1. PICOS and Corresponding Inclusion and Exclusion Criteria

  Include Exclude
Population

Individual patients and partners of any age and Populations with healthcare needs in geographically rural areas regardless of where the providers are located.

  • Rural is broadly defined; any commonly used or endorsed definition is acceptable

Providers (clinicians broadly defined or healthcare organizations) of health care to patients and populations in rural areas.

Payers (public or private; insurers or self-pay) for health care provided to patients and populations in rural areas

Urban patients or populations

Mixed patients and populations that are not separated and predominately urban

Interactions between a formal provider and informal/family care partners/givers 

Interventions Provider-to-provider telehealth broadly defined as any form of interactive support using telecommunications technology provided to health care professionals while they are caring for rural patients and populations. This includes:
  • Remote consultations across space (e.g., video) and time (e.g., store and forward) that support diagnosis, treatment, or management of patients
  • Video, audio, or digital only consultations 
  • Remote mentoring
  • Remote rounds or group education and case review (e.g., Project ECHOs) 
  • Remote continuing education

Use of telehealth for patient encounters involving one clinician (virtual visits)

Remote patient monitoring (transmitting data from patient to a single provider)

Referrals for services that involve no interaction among providers

Comparators

No service or support

Care provided without telehealth

In-person activities

Time period prior to provider-to-provider telehealth implementation

Excluded types of telehealth

KQ1: No comparators (e.g., descriptive and cross-sectional studies)

KQ2: None

Outcomes  KQ1: 
  • Clinical outcomes such as patient-reported outcomes, mortality, morbidity, function, illness recovery, infection rates, or viral load for the identified conditions* 
  • Economic outcomes such as return on investment, cost, volume of visits, and resource use
  • Intermediate Outcomes such as:
    • Patient satisfaction, behavior, and decisions such as completion of treatment, or satisfaction with less travel to access healthcare
    • Provider satisfaction, behavior, and decisions such as choice of treatment or antibiotic stewardship 
    • Time to diagnosis, time to treatment, length of stay (if applicable), 30 days hospitalization
    • Appropriate utilization of health care services and avoided, preventable hospitalizations/readmissions/ED visits/test, treatments, procedures, etc. 

KQ2:

  1. Indicators and measures of uptake of telehealth (e.g., rates of use, timing to implementation) and characteristics of users
  2. categories and descriptors of barriers and facilitators of telehealth
  3. categories and descriptors of strategies of use of telehealth
KQ1:
  • Results of models, simulations, or projections without actual outcomes data
  • Results of cross-sectional studies and surveys that include no comparison (e.g., descriptive statistics)
  • Results from surveys of attitudes or opinions about hypothetical scenarios (i.e., not actual experience)
  • Diagnostic concordance or accuracy or other measures of agreement between in-person and telehealth consultations 
Settings

Outpatient (primary care and specialty care)
Inpatient (e.g., remote ICU, consultations for hospitalized patients)
Prehospital and Emergency Care (e.g., Telestroke, EMS, ED, urgent care)
Post-Acute and Long-term Care (e.g., home care and nursing homes)
Studies of health care services delivered outside of health care settings (e.g., social services, churches, schools, prisons)

Civilian, Veterans Administration, or Military (except battlefield)

United States and other countries with similar or more advanced health care systems and resources 

Mass casualty and war/battlefield events

Countries with significantly different healthcare systems and fewer resources (e.g., low-income countries)

Study Types and Designs

KQ1: Comparative studies including trial and observational studies, including prospective and retrospective cohort studies and before-after studies (i.e., natural experiments)

KQ2: Comparative or descriptive studies

All: Nonsystematic reviews, commentaries, or letters

KQ1: Descriptive studies, feasibility assessments

Study Years

Published in 2010 or later. Included systematic reviews may include studies prior to 2010 if such studies are relevant to current technology

For KQ2a: published in 2015 or later

Published prior to 2010

For KQ2a: Published prior to 2015

Abbreviations: ED = Emergency Department; EMS = Emergency Medical Services; ICU = Intensive Care Unit; KQ = key question; Project ECHO = Extension for Community Healthcare Outcomes
*Include only studies which examine outcomes for the following conditions: substance abuse/alcohol, HIV/HPV/other infectious diseases, suicide, heart disease, cancer, unintentional injury, chronic lower respiratory disease, and stroke

Literature Search Strategies To Identify Relevant Studies To Answer the Key Questions:

Literature Databases: Ovid MEDLINE®, CINAHL®, EMBASE, and Cochrane CENTRAL will be searched to capture both published and gray literature. The search strategies will be developed by a librarian with expertise in conducting searches for systematic reviews. Searches will be peer reviewed by a second librarian.

Search Strategy: The Ovid MEDLINE® search strategy is included in Appendix A. This will be refined or augmented as needed based on the results. In order to focus the search on rural applications of telehealth, we will use three specific indexing terms (Rural Health Services, Rural Population, and Medically Underserved Area) as well as key word searches of titles and abstracts for "rural", "remote" and "resource limited." Search, and triage results will be reviewed with the Technical Expert Panel (TEP) and NIH/ODP Working Group. Additional search strategies will be considered in consultation with these two groups if needed.

Publication Date Range: Searches will begin in January 2010 for Key Question 1 and January 2015 for Key Question 2. This will capture studies of systems that rely on more current technology. We will include information on the dates the studies were conducted and the technologies used as well the dates of publication.

Initial searches will be conducted through October 2020. These searches will be updated during the public comment period of the draft report to capture any new publications. Literature identified during the update search will be assessed by following the same process of dual review as all other studies considered for inclusion in the report. If any pertinent new literature is identified for inclusion in the report, it will be incorporated before the final submission of the report.

Supplemental Evidence and Data for Systematic review (SEADS): The AHRQ Evidence-based Practice Center (EPC) Scientific Resource Center will notify stakeholders about the opportunity to submit information via the SEADS portal. There will also be an announcement published in the Federal Register. 

Gray Literature: Sources for gray (unpublished) literature will include reports produced by government agencies, health care provider organizations, or others. With the help of AHRQ we will contact the federal government community of practice on telehealth, and other appropriate organizations including, but not limited to the American Telemedicine Association, The Society for Education and the Advancement of Research in Connected Health (SEARCH) and AcademyHealth to make initial inquires, and we will also follow up on any suggestions made by TEP members. Preprints will be monitored as a way to identify studies that may be published during the period between the draft and final report or inform our discussion of future research needs. However, data from preprints will not be included as evidence in accordance with NIH policy. 

Hand Searching: Reference lists of included articles, selected excluded articles (e.g., narrative reviews), and systematic reviews will be reviewed for includable literature.

Contacting Authors: In the event that information regarding methods or results appears to be omitted from the published results of a study, or if we are aware of unpublished data, we will attempt to contact the authors to obtain additional information.

Process for Selecting Studies: Pre-established criteria as presented in Table 1 will be used to determine eligibility for inclusion and exclusion of abstracts in accordance with the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.52 To ensure accuracy, all excluded abstracts will be dual reviewed. Full text for all citations deemed appropriate for inclusion by at least one of the reviewers will be retrieved. Each full-text article will be independently reviewed for eligibility by two team members, including any articles suggested by peer reviewers or that arise from the public posting process. Any disagreements will be resolved by consensus among investigators. A record of studies excluded at the full-text level with reasons for exclusion will be maintained and made available as an appendix to the final report.

Data Abstraction and Data Management: After studies are deemed to meet inclusion criteria, data will be abstracted, such as: study design, year, setting, country, sample size, patient and providers types and characteristics (e.g., age, sex, race, reason for presentation, diagnosis, provider training/background/scope of practice and primary care or specialty type), intervention characteristics (e.g., mode of delivery, duration or frequency, function) and results relevant to each Key Question as outlined in the previous PICOS section. Data abstraction forms will be developed after full text review and the data to be included in evidence tables will be discussed with the AHRQ TOO and the stakeholder group. Information relevant for assessing applicability will include the number of patients randomized/eligible for inclusion in an observational study relative to the number of patients enrolled or the number and diversity of settings or locations as well characteristics of the population, telehealth intervention or implementation strategy, and administrating personnel. Sources of funding for all studies will also be recorded. All study data will be verified for accuracy and completeness by a second team member. 

Assessment of Methodological Risk of Bias of Individual Studies: Predefined criteria will be used to assess the risk of bias (also referred to as quality or internal validity) for each individual included study, using criteria appropriate for the study designs. Controlled trials and observational studies will be assessed using a priori established criteria consistent with the approach recommended in the chapter, Assessing the Risk of Bias of Individual Studies When Comparing Medical Interventions in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.52 Studies will be rated as "low risk of bias," "medium risk of bias," or "high risk of bias."

Studies rated "low risk of bias" are considered to have the least risk of bias, and their results are generally considered valid. "Low risk of bias" studies include clear descriptions of the population, setting, interventions, and comparison groups; a valid method for allocation of patients to treatment; low dropout rates and clear reporting of dropouts; appropriate means for preventing bias; and appropriate measurement of outcomes. 

Studies rated "medium risk of bias" are susceptible to some bias, though not enough to invalidate the results. These studies may not meet all the criteria for a rating of low risk of bias, but no flaw is likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems. The "medium risk of bias" category is broad, and studies with this rating will vary in their strengths and weaknesses. The results of some medium risk of bias studies are likely to be valid, while others may be only possibly valid.

Studies rated "high risk of bias" have significant flaws that imply biases of various types that may invalidate the results. They have a serious or "fatal" flaw in design, analysis, or reporting; large amounts of missing information; discrepancies in reporting; or serious problems in the delivery of the intervention. In general, observational studies that do not perform adjustment for potential confounders will be assessed as "high risk of bias." This is because it is likely the results of these studies are at least as likely to reflect flaws in the study design as the true difference between the compared interventions. We will not exclude studies rated high risk of bias a priori, but high risk of bias studies will be considered to be less reliable than low or medium risk of bias studies when synthesizing the evidence, particularly if discrepancies between studies are present.

Each study evaluated will be independently reviewed for risk of bias by two team members. Any disagreements will be resolved by consensus. Team members who were involved in the conduct of a study will not be involved in data abstraction or risk of bias assessment for that study.

Data Synthesis: We will construct evidence tables identifying the study characteristics (as discussed above), results of interest, and risk of bias ratings for all included studies, and summary tables to highlight the main findings. We will review and highlight studies by using a hierarchy-of-evidence approach, where the best evidence is the focus of our synthesis for each key question.

Qualitative data will be summarized in summary tables and ranges and descriptive analysis and interpretation of the results will be provided. If sufficient data are available, meta-analyses will be conducted to summarize data and obtain more precise estimates of outcomes for which studies are homogeneous enough to provide a meaningful combined estimate. The feasibility of a quantitative synthesis will depend on the number and completeness of reported outcomes and a lack of major heterogeneity. To determine whether meta-analysis could be meaningfully performed, we will consider the risk of bias for each of the studies and the heterogeneity among studies in design, patient population, interventions, and outcomes, and may conduct sensitivity analyses. If meta-analysis is performed, randomized controlled trials will be analyzed separately from observational studies. Meta-regression may be conducted to explore statistical heterogeneity using additional variables for methodological or other characteristics (e.g., risk of bias, randomization or blinding, outcome definition, and ascertainment) given enough number of studies.

Grading the Strength of Evidence (SOE) for Major Comparisons and Outcomes: The strength of evidence (SOE) for each Key Question will be initially assessed by one researcher for each clinical outcome (see PICOS). For KQ1a-d (effectiveness) we will use the approach described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.52 To ensure reliability and validity of the evaluation, the body of evidence will be assessed for the following criteria as they are defined in the Methods Guide:

  • Study limitations (low, medium, or high level of study limitations)
  • Consistency (consistent, inconsistent, or unknown/not applicable)
  • Directness (direct or indirect)
  • Precision (precise or imprecise)

Key question 2a (the uptake of telehealth in rural areas) is descriptive and a formal SOE assessment will not be conducted. We will prioritize reports of U.S. national or regional studies over local reports or data from other countries. We will summarize the strengths and limitations of the data collection and analyses of the included reports for KQ2a, with a focus on elements such as the extent the sample represents the population of interest and the completeness and reliability of the data. 

The evidence for Key Questions 2b and 2c is likely to consist of studies that use qualitative methods (e.g., interviews, case studies, focus groups) as well as quantitative methods and the studies may not be comparative. For these reasons the SOE approach planned for KQ1 is unlikely to be applicable. To address this we will assess the fit of the GRADE-CERQual approach to our included studies for these questions.53 If applicable to the body of literature, we will assess SOE based on the following domains from this framework:

  • Methodological limitations
  • Coherence
  • Adequacy
  • Relevance

For both approaches, the strength of evidence will be assigned an overall grade of high, moderate, low, or insufficient according to a four-level scale by evaluating and weighing the combined results of the included domains. The four levels are:

  • High—Very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. The findings are stable (i.e., another study would not change the conclusions).
  • Moderate—Confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. The findings are likely to be stable, but some doubt remains.
  • Low—Limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). Additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
  • Insufficient—No evidence. Investigators are unable to estimate an effect, or have no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.

Assessing Applicability: Applicability will be considered according to the approach described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.52 We will use the PICOS framework to consider the applicability of the evidence base for each key question, for example, examining the characteristics of the patient populations (e.g., clinical condition) and study setting (e.g., inpatient or outpatient). Variability in the studies may limit the ability to generalize the results to other populations and settings.

V. References

  1. Rural health. Health Aff (Millwood). 2019;38(12):1964-5. doi: 10.1377/hlthaff.2019.01365. PMID: 31794298.
  2. Singh GK, Siahpush M. Widening rural-urban disparities in all-cause mortality and mortality from major causes of death in the USA, 1969-2009. J Urban Health. 2014;91(2):272-92. doi: 10.1007/s11524-013-9847-2. PMID: 24366854.
  3. Garcia MC, Rossen LM, Bastian B, et al. Potentially excess deaths from the five leading causes of death in metropolitan and nonmetropolitan counties—United States, 2010-2017. MMWR Surveill Summ. 2019;68(10):1-11. doi: 10.15585/mmwr.ss6810a1. PMID: 31697657.
  4. Hall JE, Moonesinghe R, Bouye K, et al. Racial/ethnic disparities in mortality: contributions and variations by rurality in the United States, 2012-2015. Int J Environ Res Public Health. 2019;16(3). doi: 10.3390/ijerph16030436. PMID: 30717345.
  5. Villapiano N, Iwashyna TJ, Davis MM. Worsening Rural-Urban Gap in Hospital Mortality. J Am Board Fam Med. 2017;30(6):816-23. doi: 10.3122/jabfm.2017.06.170137. PMID: 29180557.
  6. Keyes KM, Cerdá M, Brady JE, et al. Understanding the rural-urban differences in nonmedical prescription opioid use and abuse in the United States. Am J Public Health. 2014;104(2):e52-9. doi: 10.2105/ajph.2013.301709. PMID: 24328642.
  7. Monnat SM, Rigg KK. Examining rural/urban differences in prescription opioid misuse among US adolescents. J Rural Health. 2016;32(2):204-18. doi: 10.1111/jrh.12141. PMID: 26344571.
  8. Boring MA, Hootman JM, Liu Y, et al. Prevalence of arthritis and arthritis-attributable activity limitation by urban-rural county classification—United States, 2015. MMWR Morb Mortal Wkly Rep. 2017;66(20):527-32. doi: 10.15585/mmwr.mm6620a2. PMID: 28542117.
  9. Croft JB, Wheaton AG, Liu Y, et al. Urban-rural county and state differences in chronic obstructive pulmonary disease - United States, 2015. MMWR Morb Mortal Wkly Rep. 2018;67(7):205-11. doi: 10.15585/mmwr.mm6707a1. PMID: 29470455.
  10. Krishna S, Gillespie KN, McBride TM. Diabetes burden and access to preventive care in the rural United States. J Rural Health. 2010;26(1):3-11. doi: 10.1111/j.1748-0361.2009.00259.x. PMID: 20105262.
  11. Primm K, Ferdinand AO, Callaghan T, et al. Congestive heart failure-related hospital deaths across the urban-rural continuum in the United States. Prev Med Rep. 2019;16:101007. doi: 10.1016/j.pmedr.2019.101007. PMID: 31799105.
  12. Nelson JA, Kinder A, Johnson AS, et al. Differences in selected HIV care continuum outcomes among people residing in rural, urban, and metropolitan areas-28 US jurisdictions. J Rural Health. 2018;34(1):63-70. doi: 10.1111/jrh.12208. PMID: 27620836.
  13. Zahnd WE, Rodriguez C, Jenkins WD. Rural-urban differences in human papillomavirus-associated cancer trends and rates. J Rural Health. 2019;35(2):208-15. doi: 10.1111/jrh.12305. PMID: 29808500.
  14. Galambos CM. Health care disparities among rural populations: a neglected frontier. Health Soc Work. 2005;30(3):179-81. doi: 10.1093/hsw/30.3.179. PMID: 16190293.
  15. Garcia MC, Faul M, Massetti G, et al. Reducing potentially excess deaths from the five leading causes of death in the rural United States. MMWR Surveill Summ. 2017;66(2):1-7. doi: 10.15585/mmwr.ss6602a1. PMID: 28081057.
  16. Souch JM, Cossman JS. A Commentary on Rural-Urban Disparities in COVID-19 Testing Rates per 100,000 and Risk Factors. J Rural Health. 2020 doi: 10.1111/jrh.12450. PMID: 32282964.
  17. Centers for Disease Control and Prevention. Why Rural Communities May Be at Higher Risk During the COVID-19 Pandemic. 2020. Accessed September 5, 2020.
  18. Murphy T, Marema T. New Covid-19 Cases are on the Rise Again in Rural Counties. 2020. Accessed September 5, 2020.
  19. Wright B, Fraher E, Holder MG, et al. Will community health centers survive COVID-19? J Rural Health. 2020;16:16. doi: 10.1111/jrh.12473. PMID: 32415879.
  20. The Larry A Green Center. Quick COVID-19 Survey. 2020. Accessed September 4, 2020.
  21. The Chartis Center for Rural Health. The Rural Health Safety Net Under Pressure: Rural Hospital Vulnerability. 2020.
  22. Probst JC, Moore CG, Glover SH, et al. Person and place: the compounding effects of race/ethnicity and rurality on health. Am J Public Health. 2004;94(10):1695-703. doi: 10.2105/ajph.94.10.1695. PMID: 15451735.
  23. Austin B, Glaeser E, Summers L. Saving the heartland: place-based policies in 21st century America. BPEA Conference Draft. Spring 2018.
  24. Porter E. The Hard Truths of Trying to "Save" the Rural Economy. 2018. Accessed September 9, 2020 2020.
  25. Carnethon MR, Kershaw KN, Kandula NR. Disparities research, disparities researchers, and health equity. Jama. 2019 doi: 10.1001/jama.2019.19329. PMID: 31841578.
  26. Jones NL, Breen N, Das R, et al. Cross-cutting themes to advance the science of minority health and health disparities. Am J Public Health. 2019;109(S1):S21-S4. doi: 10.2105/ajph.2019.304950. PMID: 30699031.
  27. Srinivasan S, Williams SD. Transitioning from health disparities to a health equity research agenda: the time is now. Public Health Rep. 2014;129 Suppl 2(Suppl 2):71-6. doi: 10.1177/00333549141291s213. PMID: 24385668.
  28. Tuckson RV, Edmunds M, Hodgkins ML. Telehealth. N Engl J Med. 2017;377(16):1585-92. doi: 10.1056/NEJMsr1503323. PMID: 29045204.
  29. Telemedicine, telehealth, and health information technology. An ATA Isssue Paper. American Telemedicine Association. May 2006.
  30. Rae M, Cox C, Claxton G. Coverage and utilization of telemedicine services by enrollees in large employer plans. 2020. Accessed September 5, 2020.
  31. Castro D, Miller B, Nager A. Unlocking the Potential of Physician-to-Patient Telehealth Services. 2014. Accessed September 5, 2020.
  32. Lustig T. The role of telehealth in an evolving health care environment—workshop summary. Washington DC; 2012.
  33. Bashshur R, Shannon G, Smith B. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health. 2014;20(9):769-800. doi: 10.1089/tmj.2014.9981. PMID: 24968105 
  34. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform. 2010;79(11):736-71. doi: 10.1016/j.ijmedinf.2010.08.006. PMID: 20884286.
  35. Hersh W, Helfand M, Wallace J, et al. Telemedicine for the Medicare Population. Evidence Report/Technology Assessment: Number 24. AHRQ Publication Number 01-E011. Rockville, MD: Agency for Healthcare Research and Quality; July 2001.
  36. Hersh W, Hickam D, Severance S, et al. Telemedicine for the Medicare Population: Update. Evidence Report/Technology Assessment Number 131. Rockville, MD: Agency for Healthcare Research and Quality; February, 2006.
  37. Hersh W, Wallace J, Patterson P, et al. Telemedicine for the Medicare Population: Pediatric, Obstetric, and Clinician-Indirect Home Interventions. Evidence Report/Technology Assessment: Number 24, Supplement. AHRQ Publication Number 01-E059. Rockville, MD: Agency for Healthcare Research and Quality; August 2001. PMID: 11569328.
  38. Totten A, Womack D, Eden K, et al. Telehealth: Mapping the Evidence for Patient Outcomes From Systematic Reviews. Technical Brief No. 26. (Prepared by the Pacific Northwest Evidence-based Practice Center under Contract No. 290-2015-00009-I.) AHRQ Publication No.16-EHC034-EF. Rockville, MD: Agency for Healthcare Research and Quality; June 2016.
  39. Scott Kruse C, Karem P, Shifflett K, et al. Evaluating barriers to adopting telemedicine worldwide: a systematic review. J Telemed Telecare. 2018;24(1):4-12. doi: 10.1177/1357633x16674087. PMID: 29320966.
  40. Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood). 2014;33(2):207-15. doi: 10.1377/hlthaff.2013.1054. PMID: 24493762.
  41. Broderick A, Lindeman D. Scaling Telehealth Programs: Lessons from Early Adopters. New York, NY: Commonwealth Fund. 2013. Accessed July 8, 2016.
  42. Beck M. How telemedicine is transforming health care. The Wall Street Journal. June 27, 2016;Sect. R1-R2.
  43. Totten AM, Hansen RN, Wagner J, et al. Telehealth for acute and chronic care consultations. Agency for Healthcare Research and Quality. 2019:04. PMID: 31577401.
  44. Liddy C, Moroz I, Mihan A, et al. A systematic review of asynchronous, provider-to-provider, electronic consultation services to improve access to specialty care available worldwide. Telemedicine Journal & E-Health. 2019;25(3):184-98. doi: 10.1089/tmj.2018.0005. PMID: 29927711.
  45. du Toit M, Malau-Aduli B, Vangaveti V, et al. Use of telehealth in the management of non-critical emergencies in rural or remote emergency departments: a systematic review. J Telemed Telecare. 2019;25(1):3-16. doi: 10.1177/1357633X17734239. PMID: 28980853.
  46. Ramnath VR, Khazeni N. Centralized monitoring and virtual consultant models of tele-ICU care: a side-by-side review. Telemed J E Health. 2014;20(10):962-71. doi: 10.1089/tmj.2014.0024. PMID: 25225795.
  47. Berndt A, Murray CM, Kennedy K, et al. Effectiveness of distance learning strategies for continuing professional development (CPD) for rural allied health practitioners: a systematic review. BMC Medical Education. 2017;17(1):117. doi: 10.1186/s12909-017-0949-5. PMID: 28701199.
  48. McBain RK, Sousa JL, Rose AJ, et al. Impact of project ECHO models of medical tele-education: a systematic review. J Gen Intern Med. 2019;34(12):2842-57. doi: 10.1007/s11606-019-05291-1. PMID: 31485970.
  49. Steeves-Reece AL, Elder NC, Graham TA, et al. Rapid deployment of a statewide COVID-19 ECHO program for frontline clinicians: early results and lessons learned. J Rural Health. 2020;12:12. doi: 10.1111/jrh.12462. PMID: 32396224.
  50. Centers for Medicare & Medicaid Services. Coronavirus waivers & flexibilities. 2020. Accessed September 8, 2020.
  51. Hirko KA, Kerver JM, Ford S, et al. Telehealth in response to the Covid-19 pandemic: implications for rural health disparities. J Am Med Inform Assoc. 2020;26:26. doi: 10.1093/jamia/ocaa156. PMID: 32589735.
  52. AHRQ Methods for Effective Health Care. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008.
  53. Lewin S, Booth A, Glenton C, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018;13(Suppl 1):2. doi: 10.1186/s13012-017-0688-3. PMID: 29384079.

VI. Definition of Terms

Abbreviation Definition
CDC Centers for Disease Control and Prevention
COVID-19 Novel Coronavirus Disease 2019
ECHO Extension for Community Healthcare Outcomes
ED Emergency department
EMS Emergency medical services
EPC Evidence-based Practice Center
HPV Human papillomavirus
HRSA Health Resources and Services Administration
ICU Intensive care unit
KQ Key question
NIH National Institutes of Health
ODP Office of Disease Prevention
PICOS Populations, interventions, comparators, outcomes, and setting
TEP Technical Expert Panel
TOO Task Order Officer

 

VII. Summary of Protocol Amendments

If needed, amendments to the protocol will include the date of each amendment, description of the change, and rationale. 

VIII. Review of Key Questions

Key questions were refined by the Evidence-based Practice Center (EPC), then reviewed by Agency for Healthcare Research and Quality (AHRQ) staff and the National Institutes of Health Office of Disease Prevention (NIH/ODP) Working Group to ensure they addressed the clinical questions driving the nomination of this topic. These reviews also aimed to make the key questions more explicit about the populations, interventions, comparisons, outcomes, treatment duration, settings, and study designs being considered.

IX. NIH/ODP Working Group

In place of Key Informants, the NIH/ODP Working Group (including subject matter experts from the Centers for Disease Control and Prevention [CDC] Office of the Associate Director for Policy and Strategy, the NIH National Heart, Lung, and Blood Institute, the NIH National Center for Advancing Translational Sciences, the Health Resources and Services Administration (HRSA) Federal Office of Rural Health Policy, and staff from the ODP) provided input into identifying the development and refinement of the protocol. The NIH/ODP Working Group has participated in monthly calls with AHRQ staff and the EPC; provided written and verbal feedback on drafts of the Topic Refinement documents; and will participate with AHRQ staff, the EPC, and a Content Area Expert Group in a meeting to refine the project scope.

X. Technical Experts

Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. The Technical Expert Panel (TEP) is selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that fosters a thoughtful, relevant systematic review. Therefore, study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts.

Technical Experts provide information to the EPC, AHRQ, and the NIH/ODP Working Group to identify literature search strategies and suggest approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind; neither do they contribute to the writing of the report. They do not review the report, except as given the opportunity to do so through the peer or public review mechanism.

Members of the TEP must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The AHRQ TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

XI. Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparing the final report. Peer reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers.

The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published 3 months after publication of the evidence report.

Potential peer reviewers must disclose any financial conflicts of interest greater than $5,000 and any other relevant business or professional conflicts of interest. Invited peer reviewers with any financial conflict of interest greater than $5,000 will be disqualified from peer review. Peer reviewers who disclose potential business or professional conflicts of interest can submit comments on draft reports through the public comment mechanism.

XII. EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Direct financial conflicts of interest that cumulatively total more than $1,000 will usually disqualify an EPC core team investigator.

XIII. Role of the Funder

This project was funded under Contract No. 75Q80120D00006 from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services, through an Interagency Agreement with the National Institutes of Health, which provided the funds for the project. The AHRQ Task Order Officer reviewed the EPC response to contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by either the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

XIV. Registration

This protocol will be registered in the international prospective register of systematic reviews (PROSPERO).

Appendix A. Search Strategy

Database: Ovid MEDLINE(R) ALL

  1. Rural Health Services/ 
  2. Rural Population/ 
  3. (rural or "resource limited" or (remote adj5 (population* or community or communities or area*))).ti,ab,kf. 
  4. Medically Underserved Area/ 
  5. exp Community Health Services/ or Community Health Centers/ or United States Indian Health Service/ 
  6. "Referral and Consultation"/ 
  7. exp Health Services Accessibility/ 
  8. "Delivery of Health Care"/ 
  9. 2 or 3 
  10. or/4-8 
  11. 9 and 10 
  12. 1 or 11 
  13. exp Telemedicine/ 
  14. Mobile Applications/ 
  15. exp Telecommunications/ 
  16. (telemedicine or telemedical or telehealth or teleconsult* or "e-consult*" or "econsult*" or ((remote or virtual) adj3 (consult* or education or round* or mentor*))).ti,ab,kf. 
  17. or/13-16 
  18. 9 and 17 
  19. 12 and 17 
  20. "provider to provider".ti,ab,kf. 
  21. ("Project ECHO" or "Extension for Community Healthcare Outcomes").ti,ab,kf. 
  22. or/18-21 
  23. "journal of telemedicine & telecare".jn. 
  24. "telemedicine journal & e-health".jn. 
  25. 9 and (23 or 24) 
  26. "journal of rural health".jn. 
  27. 17 and 26 
  28. 22 or 25 or 27 
  29. limit 28 to english language
  30. limit 29 to yr="2010 - 2021"