Objectives and Background

The motivation for this technical brief emanates from U.S. Senators Bill Nelson and John Thune, who asked for a literature review on the value of telehealth and remote patient monitoring, particularly for the chronically ill, with a focus on expanding access to care and reducing costs. A multi-stakeholder letter from several medical, patient advocacy, and industry groups supported the call for such a report.¹ Initial searches in response to this request confirmed that there is a large volume of literature consisting of both primary studies and systematic reviews about telehealth. This literature covers a broad range of topics and is of varying quality. Given both the volume and variability of the literature, it was not possible to quickly assess where there is sufficient evidence to support policy and practice decisions and where additional systematic reviews or primary research are needed. For this reason the Agency for Healthcare Research and Quality (AHRQ) Evidence-Based Practice program commissioned an evidence map as the initial step in responding to the Senators' request.

An evidence map is a combination of a systematic approach to identifying the existing literature on a topic with a description of key characteristics of the existing evidence. This description may include graphic presentation of these key characteristics. It has been labeled a "map" both because of the use of graphics and the idea that this type of summary and description help clarify where we are and where research needs to go next. Evidence mapping is "emerging as a less exhaustive yet systematic and replicable methodology that allows an understanding of the extent and distribution of evidence in a broad clinical area, highlighting both what is known and where gaps in evidence exist."² Methodology and guidance for the creation of literature maps exist.^3,4 However, there are currently no accepted standards for this type of review, and, as one methods guide points out, the exact content and approach may vary based on the goals of the project: "Systematic maps aim to describe the existing literature, and gaps in the literature, in a broad topic area, and the literature quality and content can be analyzed in depth or more superficially as appropriate to individual projects."³

Objectives

The purpose of this technical brief is to provide a framework and an evidence map of the currently available research about the impact of telehealth on health outcomes and care utilization that can be used to inform policy and practice decisions. Creating this framework and evidence map requires identifying and then categorizing telehealth interventions and their applications as well as the types of potential benefits and harms that have been reported in research conducted to date.

This literature map will include 1) a description of the currently available systematic reviews, 2) identification of areas where primary literature is robust enough for further systematic reviews, and 3) enumeration of areas where there are gaps in the evidence that will require additional primary research.

Background

Telehealth encompasses several technologies that have been applied to a wide range of health conditions, populations, and settings. Additionally, telehealth mirrors the rapidly changing technology environment that makes it challenging to quickly and easily monitor the body of evidence as the technology and the evidence base is rapidly expanding in both volume and scope. Many different definitions of telehealth are used in the scientific literature, among policy leaders, and by industry and others. A well-accepted definition comes from the Health Resources and Services Administration (HRSA) that states telehealth is "the use of electronic information and telecommunications technologies to support long-distance clinical health care, patient and professional health-related education, public health and health administration."⁵ There are also many related terms such as telemedicine, eHealth, and mHealth, which have been defined by the Office of the National Coordinator for Health Information Technology (ONC).⁶

Van Dyk et al. illustrated the many varied definitions of telehealth (Figure 1)⁷ Telehealth technologies evaluated in the literature can range from videoconferencing, image exchange, and streaming media to wireless communications and monitoring.⁶ These telecommunications technologies can provide long-distance health care, educate patients and providers, and address public health needs. The wide-ranging capabilities also create one of the major challenges of systematically reviewing the literature for effectiveness of telehealth? the heterogeneity among existing studies. Studies of telehealth may vary by setting: rural or urban, home or community-based care, clinic, radiology, pharmacy, nursing home, or hospital-based care.⁸ A related challenge is that the National Library of Medicine's Medical Subject Headings (MeSH) vocabulary equates telehealth, telemedicine, eHealth, and mHealth all as "entry terms" (synonyms). This makes identifying the scientific literature on telehealth less precise and sorting it more labor intensive. A preliminary PubMed search identified over 350 biomedical journal articles with Telemedicine as a major MeSH heading that were also tagged with the "Systematic Review" publication type. These include a recent high-profile systematic review by Bashshur et al.⁹ and two "systematic reviews of systematic reviews" published in 2010.^10,11

Figure 1: Scope of Telehealth Terminology

Guiding Questions

The questions below will guide our work in providing a framework and an evidence map for research on the effectiveness of telehealth interventions.

1. Describe the current research on the effectiveness of telehealth interventions.
   1. What telehealth interventions have been studied for effectiveness or harms?
      1. For which interventions are there systematic reviews available?
   2. What patient populations and conditions have been studied with telehealth interventions?
   3. What settings and situations have been studied with telehealth interventions?
   4. What primary outcomes have been studied with telehealth interventions?
   5. What study designs have been used in studies of the effectiveness of telehealth interventions?
2. Describe the gaps that exist in the current research.
   1. Which telehealth interventions identified by experts as currently relevant have no research evidence, or inadequate evidence?
   2. For which telehealth interventions are additional primary research studies needed to answer questions important to policy and practice, e.g., additional patient populations or outcome measures?
   3. For which telehealth interventions are there sufficient primary research studies that a new systematic review would add to current knowledge?

Methods

1. Data Collection

A. Discussions with Key Informants

In order to ensure that we are including current technologies and research, we convened a group of Key Informants (KIs) with diverse experiences and perspectives in implementing and evaluating telehealth to contribute to our understanding of the current policy and practice issues in telehealth and how evidence might help address these issues. We will continue to engage with KIs to assist in refining the framework for categorizing and describing telehealth interventions, to help us identify areas where telehealth is being used but has not been studied, and to identify telehealth issues, that will be important in developing our evidence map. An essential task of the KIs is to verify that we are aware of the important questions about telehealth that different stakeholders want research evidence to answer. This will allow us to compare and contrast the state of the science with the needs of stakeholders and precisely identify the next steps in research that would be most likely to move the field forward. The KIs include the perspectives of patients and consumers, clinicians, policymakers and payers, and their roles span implementation and evaluation. We will consult the KIs in the early work of finalizing the protocol and later in our efforts to organize and present our results.

B. Published Literature Search

The guiding questions described above will be used to create search strategies. See Appendix A for the primary Ovid MEDLINE search strategy. We will base our final search on a combination of searches conducted for prior reviews and the suggestions of our KIs as well as expert librarians. We will search Ovid MEDLINE and the Cochrane Library databases, and will supplement those searches with reviewing reference lists, tables of contents of key journals, and consulting experts (our KIs). Searches will be limited to articles published after 2006 through the end of May 2015. For systematic reviews, we will limit to reviews with search date ranges ending in 2006 or later. The start date is early enough to capture all relevant published systematic reviews and primary studies of current telehealth approaches and technologies and coincides with the publication date of a previous systematic review of telemedicine that our EPC performed.¹²

C. Grey Literatures Search

As there is a large volume of published literature on telehealth, our grey literature search will be designed to identify if there are reports or publications that would fill in gaps not covered by the published literature. We will:

• Search selected grey literature databases and clearinghouses, such as the one maintained by the New York Academy of Medicine
• Search a targeted number of selected web sites of organizations such as government agencies, foundations, or professional associations that have been active in producing, implementing, or evaluating telehealth
• Review the reference lists of systematic reviews and primary articles for additional citations
• Followup on suggestion made by KIs

Scientific Information Packets: Scientific information packets (SIPs) will not be requested for this technical brief. With AHRQ’s assistance we will attempt to identify and describe efforts to study or synthesize telehealth that are underway in other Federal agencies.

D. Process for Selecting Studies

The guiding questions described above will be used to determine eligibility for inclusion and exclusion of abstracts in accordance with the AHRQ Methods Guide.¹³ To ensure accuracy, all excluded abstracts will be dual reviewed. All citations deemed appropriate for inclusion by at least one of the reviewers will be retrieved. Each full-text article will be reviewed for eligibility by two team members, including any articles suggested by peer reviewers or that arise from the public posting process. Any disagreements will be resolved by review by the project team.

We will select studies based on a hierarchy of evidence and our a priori definitions. Appendix B describes our inclusion and exclusion criteria for systematic reviews. These same criteria, minus the requirement that a study must be a systematic review, will be applied to individual studies and the grey literature. Interaction between a patient and a health care professional or interaction between two or more providers when the interaction is directly related to a patient’s care is a key inclusion criterion that helps operationalize the definition of telehealth. Following a precedent set in a previous study, telephone-only voice conversations are not considered telehealth. E-mail and Short Message Service (SMS) text are considered to be telehealth if they replace an in-person interaction.

We will first select systematic reviews of telehealth interventions. To qualify as a systematic review, a study must include a search of one or more citation databases, study selection based on prespecified inclusion and exclusion criteria, and an assessment of the quality (or risk of bias) of individual studies included in the review. These reviews will be the primary analysis tool for the evidence map. For topics where no systematic reviews are available, we will include primary research studies in the evidence map. We will also note any interventions identified by our KIs where no evidence is found.

2. Data Organization and Presentation

A. Information Management and Data Abstraction

Identified studies will be entered into citation management software that allows for coding of study design, population, etc. Descriptive data needed to categorize the studies (both primary and systematic reviews) will be abstracted. All study data will be verified for accuracy and completeness by a second team member. Data abstracted will include: study characteristics (author, search or study dates, number and type of included studies for systematic reviews), clinical indication and setting, intervention characteristics (modalities of telehealth intervention, telehealth function), outcomes, quality of included studies (systematic reviews) or study design (individual studies), and rigor of the systematic review (systematic reviews only).

B. Data Presentation

We will develop a framework for cataloging research on telehealth around the following dimensions: condition, population, modality, setting, outcomes, and study design or other selected dimensions of the quality of the research. We will organize the evidence according to this framework to aid in understanding the current evidence base. Prominent evaluation frameworks for telehealth (the Khoja-Durrani-Scott model¹⁴ and the unified approach for the evaluation of telehealth implementations in Australia¹⁵) will inform the development of our framework and evidence map. These frameworks were recently reviewed by van Dyk.¹⁶ Our approach to organizing and presenting the evidence map results will be informed by other evidence mapping efforts and examples of recently produced evidence maps. The latter include bubble plot summaries^17,18 and flow chart (or logic model) approaches.^2,4,19,20

We will organize our results to display which population, conditions, technologies, and outcomes are covered by systematic reviews, which are the subject of primary research, but not systematic reviews, and which areas are gaps in the current evidence base. We will review our results and our approach to presenting this information with the KIs, and the KI input will be incorporated into our report. The report will document our methods and results, but in order to make it more accessible and usable, technical details will be included in an appendix and the text will focus on the results and their potential implications.

References

1. Annonymous. Multistakeholder Letter to Senate Aging re AHRQ Literature Review. 2015.
2. Hetrick SE, Parker AG, Callahan P, et al. Evidence mapping: illustrating an emerging methodology to improve evidence-based practice in youth mental health. J Eval Clin Pract. 2010;16(6):1025-30. PMID: 20337833.
3. Clapton J, Rutter D, Sharif N. SCIE Systematic mapping guidance. London: SCIE. 2009.
4. Bragge P, Clavisi O, Turner T, et al. The Global Evidence Mapping Initiative: scoping research in broad topic areas. BMC Med Res Methodol. 2011;11:92. PMID: 21682870.
5. HRSA. Telehealth. 2015; https://www.hrsa.gov/ruralhealth/about/telehealth/. Accessed March 9th, 2015.
6. HealthIT.gov. What is telehealth? How is telehealth different from telemedicine? https://www.healthit.gov/providers-professionals/faqs/what-telehealth-how-telehealth-different-telemedicine. Accessed March 9th, 2015.
7. vanDyk L. A review of telehealth service implementation frameworks. International Journal of Environmental Research and Public Health. 2014;11:1279-98.
8. Lustig T. The Role of Telehealth in an Evolving Health Care Environment - Workshop Summary. Washington, DC: Institute of Medicine 2012.
9. Bashshur RL, Shannon GW, Smith BR, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health. 2014;20(9):769-800. PMID: 24968105.
10. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform. 2010;79(11):736-71. PMID: 20884286.
11. McLean S, Sheikh A, Cresswell K, et al. The impact of telehealthcare on the quality and safety of care: a systematic overview. PLoS One. 2013;8(8):e71238. PMID: 23977001.
12. Hersh WR, Hickam DH, Severance SM, et al. Diagnosis, access and outcomes: Update of a systematic review of telemedicine services. J Telemed Telecare. 2006;12 Suppl 2:S3-31. PMID: 16989671.
13. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication No. 10(13)-EHC063-EF. 2014. Accessed at www.effectivehealthcare.ahrq.gov on April 4th, 2014.
14. Khoja S, Durrani H, Scott RE, et al. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. 2013;19(1):48-53. PMID: 22957502.
15. Dattakumar A. A unified approach for the evaluation of telehealth implementations in Australia University of Melbourne. September 2, 2013 https://apo.org.au/node/35845.
16. van Dyk L. A review of telehealth service implementation frameworks. Int J Environ Res Public Health. 2014;11(2):1279-98. PMID: 24464237.
17. Hempel S, Taylor SL, Marshall NJ, et al. Evidence Map of Mindfulness. 2014. PMID: 25577939.
18. Hempel S, Taylor SL, Solloway MR, et al. Evidence Map of Acupuncture. 2014. PMID: 24575449.
19. Allmark P, Baxter S, Goyder E, et al. Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health Soc Care Community. 2013;21(1):59-68. PMID: 23039788.
20. Callahan P, Liu P, Purcell R, et al. Evidence map of prevention and treatment interventions for depression in young people. Depress Res Treat. 2012:820735. PMID: 22496974.

Definition of Terms

Not applicable.

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

Key Informants

Within the technical brief process, Key Informants serve as a resource to offer insight into the context of the technology/intervention, how it works, how it is currently used or might be used, and which features may be important from a patient or policy standpoint. They may include clinical experts, patients, manufacturers, researchers, payers, or other perspectives, depending on the technology/intervention in question. Differing viewpoints are expected, and all statements are crosschecked against available literature and statements from other Key Informants. Information gained from Key Informant interviews is identified as such in the report. Key Informants do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.

Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will be published three months after the publication of the Evidence report.

Potential reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited peer reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.

Role of the Funder

This project was funded under Contract No. HHSA 290-2015-00014I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

Appendixes

Appendix A. Preliminary Ovid MEDLINE Search Strategy

1. exp Telemedicine/
2. exp Patient Care/
3. exp Therapeutics/
4. exp Health Services/
5. exp Diagnosis/
6. exp Professional-Patient Relations/
7. exp Health Services Accessibility/
8. exp Health Behavior/
9. 2 or 3 or 4 or 5 or 6 or 7 or 8
10. exp *Telecommunications/
11. exp *Computer Communication Networks/
12. 10 or 11
13. 9 and 12
14. 1 or 13
15. limit 14 to English language
16. limit 15 to systematic reviews
17. limit 16 to yr="2006 -Current"

Appendix B. Inclusion Criteria for Systematic Reviews