Skip Navigation
AHRQ--Agency for Healthcare Research and Quality: Advancing Excellence in Health Care
  • Home
  • Search for Research Summaries, Reviews, and Reports
 
 

EHC Component

  • EPC Project

Topic Title

  • Telehealth for Acute and Chronic Care Consultation


Research Protocol – Feb. 16, 2017

Telehealth for Acute and Chronic Care Consultations

Formats

Table of Contents

I. Background and Objectives for the Systematic Review

Telehealth is the use of information and telecommunications technology to provide health care across time and/or distance. It is a tool with the potential to increase access, improve the quality of care, increase patient satisfaction, positively impact patient outcomes, and reduce the cost of care. Telehealth for consultations uses technology to involve another provider, often a specialist, which can allow medical expertise to be available where and when it is needed, minimizing potential time or geographic barriers to care and maximizing the efficient use of scarce resources. Telehealth's potential benefits are frequently cited,1,2 and there is a sizable body of research on telehealth, including systematic reviews and reviews of reviews.3-8 Implementation and spread has been slow,9,10 although this appears to be accelerating with improvement in technologies.11 Telehealth for consultations has been studied across a range of clinical situation including injuries,12 burn care,13 and infectious disease.14-16 Identifying and summarizing the available evidence about the use of telehealth for consultations could support the best use of this technology across clinical topics in the future.

The overarching goal of this project is to maximize the utility of available information by presenting it in a form that supports decision makers as they consider policy and practice changes related to telehealth for consultation. To accomplish this goal the project will combine two evidence synthesis methods, systematic review and decision modeling, and use this combination to identify, organize, and analyze the available research about the use of telehealth for consultations.

II. The Key Questions

Below are the Key Questions for the systematic review (SR) and the Guiding Questions for the decision models (DM). The Key Questions for the SR are based on questions provided in the scope of work that accompanied the Request for Task Order. The questions were reviewed, reorganized, and refined by the project team and revised after input from the Technical Expert Panel (TEP). There was no formal topic refinement for this project.

The Guiding Questions for the DM were also included in the scope of work. The topics, specific questions, and scope for the DM will be based on the literature triage and initial findings of the SR.

Key Questions for the Systematic Review

KQ 1: Are telehealth consultations effective in improving clinical and economic outcomes?

Telehealth consultations can be for any acute or chronic clinical condition across any specialty ranging from infectious disease to psychiatry.

Clinical and economic outcomes may include, but are not limited to, mortality and morbidity, utilization of health services, cost of services, and access to services.

KQ 2: Are telehealth consultations effective in improving intermediate outcomes?

Intermediate outcomes include both outcomes that precede the ultimate outcomes of interest and secondary outcomes.

Intermediate outcomes may include, but are not limited to, patient and provider satisfaction, behavior, and decisions (e.g., patient completion of treatment, provider antibiotic stewardship); volume of services; and health care processes (e.g., time to diagnosis or treatment).

KQ 3: Have telehealth consultations resulted in harms, adverse events, or negative unintended consequences?

What are the characteristics of telehealth consultations that have been the subject of comparative studies?

The characteristics include:

  1. Clinical conditions addressed. These can include broad categories such as diagnosis and treatment of infectious disease or specific conditions (e.g., upper respiratory infection, hepatitis C, skin infections) or decisions (e.g., stewardship of antibiotics or antimicrobials, selection of treatments)
  2. Characteristics of the providers and patients involved
  3. Relationships among the providers and patients involved including whether these are new or ongoing relationships
  4. Telehealth modalities and/or methods for sharing patient data used
  5. Whether specifics in (d) meet Medicare's coverage and HIPAA requirements
  6. Settings including
    • Type of health care organization including the organizational structure (e.g., integrated delivery system, critical access) and the type of care (e.g., long-term care, inpatient, ambulatory care)
    • Country
    • Geographic and economic characteristics such as urban or rural areas, or areas with high vs. low socioeconomic resources
  7. Other circumstances (e.g., appropriate transportation, climate)
  8. Payment models or requirements or limits for payment including
    • The payer/insurance for the patient (e.g., Medicare, Medicaid, commercial)
    • Any parameters for payment (e.g., relative value units [RVUs]) or limits on visits
    • Any eligibility requirements for payment based on patient, provider, setting or context characteristics
KQ 5: Do clinical, economic, intermediate, or negative outcomes (i.e., the outcomes in Key Questions 1, 2, and 3) vary across telehealth consultation characteristics (Key Question 4)?

Decision Model Guiding Questions

Using decision modeling on selected topics where information is lacking:

  1. What is the predicted impact on clinical, economic and intermediate outcomes of telehealth consultations?
  2. What is the predicted effect of various proposed payment reforms on clinical, economic, and intermediate outcomes of telehealth consultations?

PICOTS

The population, intervention, comparator, outcomes, timing, and setting (PICOTS) for this review are outlined below. Additional detail in the form of inclusion and exclusion criteria is provided in Appendix A.

Populations
  • Patients of any age, with medical care needs for prevention, treatment, or management of chronic or acute conditions
  • Providers (clinicians or health care organizations)
  • Payers for health care services (public, private, insurers, patients)
Interventions
  • Telehealth consultations are defined as the use of telehealth designed to facilitate collaboration among providers, often involving a specialist, or between clinical team members, across time and/or distance, on the assessment, diagnosis, and/or clinical management of a specific patient or group of patients.
  • Telehealth consultations can be for any acute or chronic conditions. The search will be both general as well as focused on conditions identified as areas of growth and policy interest such as infection, disease, dermatology, and critical care.
  • Telehealth consultations can use any technology (e.g., real-time video, store and forward).
Comparator

Other locations, patients, or time periods that use in-person consultations or provide usual care (which could include no access to specific services)

Outcomes for Each Key Question
  • Key Question 1: Clinical and economic outcomes
    • Clinical outcomes such as mortality, morbidity, function, recovery, infection, and access to services
    • Economic outcomes such as return on investment, cost, volume of visits, and resource use
  • Key Question 2: Intermediate outcomes
    • Patient satisfaction, behavior, and decisions such as completion of treatment, or satisfaction with less travel to access health care
    • Provider satisfaction, behavior, and decisions such as choice of treatment or antibiotic stewardship
    • Time to diagnosis and time to treatment
    • Diagnostic concordance or other measures of agreement between in-person and telehealth consultations
  • Key Question 3: Adverse effects or unintended consequences
    • Loss of privacy or breech of data security
    • Misdiagnosis or delayed diagnosis
    • Inappropriate treatment
    • Increase in resource costs, negative return on investment
  • Key Question 4: Not applicable (this is a descriptive question)
  • Key Question 5: Clinical and economic outcomes (see Key Question 1), intermediate outcomes (see Key Question 2), and adverse effects or unintended consequences (see Key Question 3).
Timing
  • Telehealth consultations can be used at any point in the diagnosis, treatment, or management of a patient.
  • Outcome measurement needs to occur after the telehealth consultation.
Setting

The consultation can involve providers and patients in any location. These could include inpatient, outpatient, or long-term care, and could be in civilian, Veterans Administration, or military facilities.

Study Designs
  • Comparative studies, including trials and observational studies.
  • Descriptive studies may be used to inform the DM as needed but will not be included in the SR.

III. Analytic Framework

Below is the analytic framework, which represents the relationships among the elements of the Key Questions for the systematic review.

The figure represents the relationships among the elements in the Key Questions. Patients, providers, and payers are listed on the left as these are the groups that experience telehealth consultations, which is the intervention. An arrow connects telehealth consultation to clinical and economic outcomes representing Key Question 1.  Another arrow connects telehealth consultations to intermediate outcomes to depict Key Question 2. A curved arrow connects telehealth consultations to adverse effects or unintended consequences. Patient characteristics, provider characteristics, technology and setting, and payment system are all listed along the bottom as these are factors that may vary across telehealth consultations. Key Question 4 is a description of which factors have been studied. Arrows go from these factors and intersect the arrows connecting telehealth consultations to intermediate, clinical, and economic outcomes to depict Key Question 5, which asks how these factors influence the relationship between telehealth consultations and the outcomes.

IV. Methods

Overview

This project includes an SR and a DM and integrates tasks for these two complementary activities. After providing key definitions, the details related to SR and DM methods are provided separately in Parts A and B below.

A key difference is that the methods for the SR can be specified in detail in advance. For the DM, a general outline is provided. Specific details may need to be revised based on the specific scope for the DM and the topics selected.

Definition of Telehealth and Telehealth Consultation for This Project

Telehealth is defined as the use of information and telecommunications technology in health care delivery for a specific patient or group of patients, involving a provider across distance or time in regards to a particular diagnosis or health condition. The information can be transmitted live, be stored and forwarded, or be a hybrid of the two prior possibilities. This definition is similar to that used in the Evidence Map.8 How the inclusion and exclusion criteria are operationalized may differ given the scope and purpose for this review.

Telehealth Consultation is defined as the use of telehealth designed to facilitate collaboration among providers, often involving a specialist, or between clinical team members, across time and/or distance, on the assessment, diagnosis, and/or clinical management of a specific patient or group of patients. While the patient may or may not be involved in the consultation, the consultation is required to be about a specific patient or group of patients in order to differentiate this activity from training or education (which would not meet our definition of telehealth). Limited information provided by one clinician to another that does not contribute to collaboration (e.g., interpretation of an EEG, report on an x-ray or scan, or reporting the results of a diagnostic text) is not considered a consultation for this review.

Part A: Systematic Review Methods

Criteria for Inclusion/Exclusion of Studies in the Review are based on the Key Questions and are described in detail in Appendix A. Below are additional details.

Study Designs

We will include comparative studies of any design including trials and cohort studies, as well as pre-post designs (i.e., the comparison can be across time points). We will exclude descriptive studies with no outcomes data or studies that include only data from one point in time (post only). We will also exclude modeling studies or studies that use synthetic data. We will access existing systematic reviews, and include their results if appropriate. At a minimum, we will use systematic reviews to identify studies. We will also exclude commentaries, letters, and articles that describe telehealth systems or implementations but do not assess impact. We will consider whether an excluded article contains information that could be used in the DM even if the study will not be included in the systematic review.

Non-English-Language Studies

We will restrict inclusion to English-language articles, but will review English-language abstracts of non-English-language articles to identify studies that would otherwise meet inclusion criteria, in order to assess for the likelihood of language bias.

Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions.

The search strategies are included in Appendix B.

Publication Date Range

We will include studies published in the past 20 years (1997 to 2016, with an update search through part of 2017). Given the delay in publishing this will capture studies of systems that rely on more current technology. We will include information on the dates the studies were conducted and the technologies used as well the dates of publication.

Literature Databases

Ovid MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials (CCRCT), and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) will be searched to capture published literature. The search strategies were developed by a specialist librarian and peer reviewed by a second librarian.

Hand Searching

Reference lists of included articles and selected excluded articles (e.g., narrative reviews) will be reviewed for includable literature.

Scientific Information Packets

The AHRQ Evidence-based Practice Center (EPC) Scientific Resource Center will be asked to notify stakeholders about the opportunity to submit Scientific Information Packets via an announcement in the Federal Register.

Grey Literature

Sources for grey (unpublished) literature will include reports produced by government agencies, health care provider organizations, or others. With the help of AHRQ we will contact the federal government community of practice on telehealth, the American Telemedicine Association, and AcademyHealth to make initial inquires, and we will also follow up on any suggestions made by TEP members.

Contacting Authors

In the event that information regarding methods or results appears to be omitted from the published results of a study, or if we are aware of unpublished data, we will query the authors to obtain additional information.

Process for Selecting Studies

Pre-established criteria will be used to determine eligibility for inclusion and exclusion of abstracts in accordance with the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.17 To ensure accuracy, all abstracts will be independently reviewed by two team members. All citations deemed appropriate for inclusion by at least one of the reviewers will be retrieved. Each full-text article will be independently reviewed for eligibility by at least two reviewers. We will review the full text of any articles suggested by peer reviewers or that arises from the public posting or SIP processes. Any disagreements about inclusion or not will be resolved by discussion and consensus across the investigators.

Data Abstraction and Data Management

After studies are deemed to meet inclusion criteria, the following data will be abstracted: study design, year, setting, country, sample size, eligibility criteria, population, and clinical characteristics (e.g., age, sex, race, reason for presentation, diagnosis), intervention characteristics (e.g., duration, training/background of personnel engaged in the consultations), and results relevant to each Key Question as outlined in the previous PICOTS section. Information relevant for assessing applicability will include the number of patients randomized/eligible for inclusion in an observational study relative to the number of patients enrolled, and characteristics of the population, telehealth intervention, and administrating personnel. Sources of funding for all studies will also be recorded. All study data will be verified for accuracy and completeness by a second team member. A record of studies excluded at the full-text level with reasons for exclusion will be maintained and made available as part of the final report.

Assessment of Methodological Risk of Bias of Individual Studies

Predefined criteria will be used to assess the risk of bias for individual controlled trials and observational studies by using clearly defined templates and criteria consistent with the approach recommended in the chapter, Assessing the Risk of Bias of Individual Studies When Comparing Medical Interventions in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.17 Studies will be rated as "low risk of bias," "medium risk of bias," or "high risk of bias."

Studies rated "low risk of bias" are considered to have the least risk of bias, and their results are generally considered valid. "Low risk of bias" studies include clear descriptions of the population, setting, interventions, and comparison groups; a valid method for allocation of patients to treatment; low dropout rates and clear reporting of dropouts; appropriate means for preventing bias; and appropriate measurement of outcomes.

Studies rated "medium risk of bias" are susceptible to some bias, though not enough to invalidate the results. These studies may not meet all the criteria for a rating of low risk of bias, but no flaw is likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems. The "medium risk of bias" category is broad, and studies with this rating will vary in their strengths and weaknesses. The results of some medium risk of bias studies are likely to be valid, while others may be only possibly valid.

Studies rated "high risk of bias" have significant flaws that imply biases of various types that may invalidate the results. They have a serious or "fatal" flaw in design, analysis, or reporting; large amounts of missing information; discrepancies in reporting; or serious problems in the delivery of the intervention. In general observational studies that do not perform adjustment for potential confounders will be assessed as "high risk of bias." The results of these studies are at least as likely to reflect flaws in the study design as the true difference between the compared interventions. We will not exclude studies rated high risk of bias a priori, but high risk of bias studies will be considered to be less reliable than low or medium risk of bias studies when synthesizing the evidence, particularly if discrepancies between studies are present.

Each study evaluated will be independently reviewed for risk of bias by two team members. Any disagreements will be resolved by consensus. If consensus cannot be arrived at by the two reviewers, the principal investigator and the lead for the decision analysis will make a final determination. Team members who were involved in the conduct of a study will not be involved in data abstraction or risk of bias assessment for that study.

Data Synthesis

We will construct evidence tables identifying the study characteristics (as discussed above), results of interest, and risk of bias ratings for all included studies, and summary tables to highlight the main findings. We will review and highlight studies by using a hierarchy-of-evidence approach, where the best evidence is the focus of our synthesis for each key question.

Qualitative data will be summarized in summary tables and ranges and descriptive analysis and interpretation of the results will be provided.

If sufficient data are available, meta-analyses will be conducted to summarize data and obtain more precise estimates of outcomes for which studies are homogeneous enough to provide a meaningful combined estimate. The feasibility of a quantitative synthesis will depend on the number and completeness of reported outcomes and a lack of major heterogeneity. To determine whether meta--analysis could be meaningfully performed, we will consider the risk of bias for each of the studies and the heterogeneity among studies in design, patient population, interventions, and outcomes, and may conduct sensitivity analyses. If meta-analysis is performed, randomized controlled trials will be analyzed separately from observational studies. Meta-regression may be conducted to explore statistical heterogeneity using additional variables for methodological or other characteristics (e.g., risk of bias, randomization or blinding, outcome definition, and ascertainment) given enough number of studies.

Grading the Strength of Evidence for Major Comparisons and Outcomes

The strength of evidence (SOE) for each Key Question will be initially assessed by one researcher for each clinical outcome (see PICOTS) by using the approach described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.17 To ensure consistency and validity of the evaluation, the grades will be reviewed by the entire team of investigators for:

  • Study limitations (low, medium, or high level of study limitations)
  • Consistency (consistent, inconsistent, or unknown/not applicable)
  • Directness (direct or indirect)
  • Precision (precise or imprecise)
  • Reporting bias (suspected or undetected)

The strength of evidence will be assigned an overall grade of high, moderate, low, or insufficient according to a four-level scale by evaluating and weighing the combined results of the above domains:

  • High—Very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. The findings are stable (i.e., another study would not change the conclusions).
  • Moderate—Confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. The findings are likely to be stable, but some doubt remains.
  • Low—Limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). Additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
  • Insufficient—No evidence. Investigators are unable to estimate an effect, or have no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.
Assessing Applicability

Applicability will be considered according to the approach described in the Methods Guide for Effectiveness and Comparative Effectiveness Reviews.17 We will use the PICOTS framework to consider the applicability of the evidence base for each key question, for example, examining the characteristics of the patient populations (e.g., clinical condition) and study setting (e.g., inpatient or outpatient). Variability in the studies may limit the ability to generalize the results to other populations and settings.

Part B: Decision Analysis Methods

The purpose of the DM component of this project is to augment the SR by providing the best available information for questions that fall outside the bounds of the existing research. DM can provide information on how interventions are expected to perform with longer time horizons, in specific subgroups, or for additional outcomes. In this project DM will be used to estimate the clinical and cost outcomes of telehealth consultations for a selected clinical condition and to estimate the impact of different payment models for telehealth consultation compared with current standard care.

Criteria for selection of topics for DM

The DM will be limited to a small number of models (two to four) of telehealth consultation for specific patient populations and outcomes. As the purpose of the DM is to address questions the SR alone cannot answer, the scope is dependent on the findings of the SR. We will accelerate the SR search and triage in order to provide an overview of the data available and inform the DM.

We will set criteria for selection of the topics for models and solicit TEP and other stakeholder input before finalizing the DM topics. The proposed criteria are (1) that some data are available on the topic, (2) that the SR is unlikely to be able to answer the question, and (3) that the question is of high priority for current policy and practice.

The DM process will include specifying the perspective, populations to be included, treatment of missing data (cut-offs for inclusions, potential patterns of absence, and when data may be imputed), the time horizon, and the approach to sensitivity analysis (e.g., probabilistic and scenario analyses). The type of analysis will be decided based on the question and the underlying assumptions and findings from the review. For example, if the clinical outcomes are found to be equally effective as in-person consultations for assessment and management of a specific condition, a cost minimization analysis could be used to distinguish the least costly strategy. However, if the difference in outcomes is uncertain or there is an expectation of differences outcomes, we will undertake a full cost-effectiveness analysis comparing the use of telehealth consultation to current practice through construction of an incremental cost-effectiveness ratio (ICER). Other possible approaches include using Bayesian network meta-analysis to account for the full joint uncertainty of model parameters18-27 or quantifying evidence gaps using value of information (VOI) analysis, which can help provide decision makers with estimates of the value of waiting for further research.28,29

Once the topics are selected, a specific Modeling Analysis Plan will be constructed for each DM question. Once the DM question has been defined and the available input data identified (from the SR), a decision on the modeling framework will be made. The modeling will likely consist of either a decision tree or Markov Model, depending on the complexity of the scenario and outcomes being modeled, as well as the time horizon of the events and outcomes. Microsimulation (discrete event simulation) methods will also be considered, although it is unlikely that the SR will produce the level of data required to inform this type of modeling framework.

After the modeling technique has been decided, each model's analysis plan will specify the following elements:

  • Model structure presented as a graphic or text based description of the inputs, processes, events, and outcomes
  • Descriptions of comparators
  • Perspective from which the decision is analyzed. Perspectives may include those of the patient, a health care system, a specific health care provider, or a specific payer.
  • Time Horizon for the impact and resource requirements
    • Influenced by the perspective
  • Discount Rate (3% base case for costs and outcomes)
  • Assumptions, including
    • Inputs
    • Sensitivity ranges (variance) for inputs
    • Scenarios for consideration

V. References

  1. Castro D, Miller B, Nager A. Unlocking the Potential of Physician-to-Patient Telehealth Services. Washington, DC: Information Technology and Innovation Foundation; May 12, 2014. http://www.itif.org/publications/unlocking-potential-physician-patient-telehealth-services. Accessed February 3, 2017.
  2. Lustig T. The Role of Telehealth in an Evolving Health Care Environment - Workshop Summary. Washington, DC: Institute of Medicine; 2012.
  3. Bashshur R, Shannon G, Smith B. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health. 2014;20(9):769-800. PMID: 24968105.
  4. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform. 2010;79(11):736-71. PMID: 20884286.
  5. Hersh W, Helfand M, Wallace J, et al. Telemedicine for the Medicare Population. Evidence Report/Technology Assessment: Number 24. AHRQ Publication Number 01-E011. Rockville, MD: Agency for Healthcare Research and Quality; July 2001. http://www.ncbi.nlm.nih.gov/books/NBK33341/. Accessed February 3, 2017.
  6. Hersh W, Hickam D, Severance S, et al. Telemedicine for the Medicare Population: Update. Evidence Report/Technology Assessment Number 131. Rockville, MD: Agency for Healthcare Research and Quality; February, 2006. http://www.ncbi.nlm.nih.gov/books/NBK37953/. Accessed February 3, 2017.
  7. Hersh W, Wallace J, Patterson P, et al. Telemedicine for the Medicare Population: Pediatric, Obstetric, and Clinician-Indirect Home Interventions. Evidence Report/Technology Assessment: Number 24, Supplement. AHRQ Publication Number 01-E059. Rockville, MD: Agency for Healthcare Research and Quality; August 2001. https://www.ncbi.nlm.nih.gov/books/NBK36492/. Accessed February 3, 2017.
  8. Totten A, Womack D, Eden K, et al. Telehealth: Mapping the Evidence for Patient Outcomes From Systematic Reviews. Technical Brief No. 26. (Prepared by the Pacific Northwest Evidence-based Practice Center under Contract No. 290-2015-00009-I.) AHRQ Publication No.16-EHC034-EF. Rockville, MD: Agency for Healthcare Research and Quality; June 2016. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm
  9. Adler-Milstein J, Kvedar J, Bates DW. Telehealth among US hospitals: several factors, including state reimbursement and licensure policies, influence adoption. Health Aff (Millwood). 2014 Feb;33(2):207-15. PMID: 24493762.
  10. Broderick A, Lindeman D. Scaling Telehealth Programs: Lessons from Early Adopters. New York, NY: Commonwealth Fund; 2013. http://www.commonwealthfund.org/Publications/Case-Studies/2013/Jan/Telehealth-Synthesis.aspx. Accessed on July 8, 2016.
  11. Beck M. How telemedicine is transforming health care. The Wall Street Journal. June 27, 2016;Sect. R1-R2.
  12. Hasselberg M, Beer N, Blom L, et al. Image-based medical expert teleconsultation in acute care of injuries. A systematic review of effects on information accuracy, diagnostic validity, clinical outcome, and user satisfaction. PLoS One. 2014;9(6):e98539. PMID: 24887257.
  13. Wallace DL, Hussain A, Khan N, et al. A systematic review of the evidence for telemedicine in burn care: with a UK perspective. Burns. 2012 Jun;38(4):465-80. PMID: 22078804.
  14. Assimacopoulos A, Alam R, Arbo M, et al. A brief retrospective review of medical records comparing outcomes for inpatients treated via telehealth versus in-person protocols: is telehealth equally effective as in-person visits for treating neutropenic fever, bacterial pneumonia, and infected bacterial wounds? Telemed J E Health. 2008 Oct;14(8):762-8. PMID: 18954245.
  15. Mashru J, Kirlew M, Saginur R, et al. Management of infectious diseases in remote northwestern Ontario with telemedicine videoconference consultations. J Telemed Telecare. 2016 Jan 8PMID: 26748393.
  16. Parmar P, Mackie D, Varghese S, et al. Use of telemedicine technologies in the management of infectious diseases: a review. Clin Infect Dis. 2015 Apr 1;60(7):1084-94. PMID: 25516192.
  17. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication No. 10(14)-EHC063-EF. Rockville, MD:: Agency for Healthcare Research and Quality; January 2014. Chapters available at: www.effectivehealthcare.ahrq.gov.
  18. Cheng MM, Goulart B, Veenstra DL, et al. A network meta-analysis of therapies for previously untreated chronic lymphocytic leukemia. Cancer Treat Rev. 2012 Dec;38(8):1004-11. PMID: 22405931.
  19. Devine EB, Alfonso-Cristancho R, Sullivan SD. Effectiveness of biologic therapies for rheumatoid arthritis: an indirect comparisons approach. Pharmacotherapy. 2011 Jan;31(1):39-51. PMID: 21182357.
  20. Fu R, Dey DK, Holsinger KE. Bayesian models for the analysis of genetic structure when populations are correlated. Bioinformatics. 2005 Apr 15;21(8):1516-29. PMID: 15585534.
  21. Fu R, Dey DK, Holsinger KE. A Beta-mixture mod el for assessing genetic population structure. Biometrics. 2011 Sep;67(3):1073-82. PMID: 21114661.
  22. Jansen JP, Fleurence R, Devine B, et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health. 2011 Jun;14(4):417-28. PMID: 21669366.
  23. Lin VW, Ringold S, Devine EB. Comparison of ustekinumab with other biological agents for the treatment of moderate to severe plaque psoriasis: a Bayesian network meta-analysis. Arch Dermatol. 2012 Dec;148(12):1403-10. PMID: 23069736.
  24. Signorovitch J, Swallow E, Kantor E, et al. Everolimus and sunitinib for advanced pancreatic neuroendocrine tumors: a matching-adjusted indirect comparison. Exp Hematol Oncol. 2013;2(1):32. PMID: 24314093.
  25. Welton NJ, Sutton AJ, Cooper NJ, et al. Chapter 7: Evidence synthesis in a decision modeling framework. Evidence Synthesis for Decision Making in Healthcare. Chichester, West Sussex, UK: Wiley; 2012.
  26. McDonagh M, Peterson K, Carson S, et al. Drug Class Review: Atypical Antipsychotic Drugs: Final Update 3 Report. Portland, OR: Oregon Health & Science University; 2010.
  27. Fu R, Handel D. Two-part random effects model with right truncation with application to multisite ambulance diversion data. Acad Emerg Med. 2014;5(21):Suppl S27.
  28. Basu A, Meltzer D. Value of information on preference heterogeneity and individualized care. Med Decis Making. 2007 Mar-Apr;27(2):112-27. PMID: 17409362.
  29. Carlson JJ, Thariani R, Roth J, et al. Value-of-information analysis within a stakeholder-driven research prioritization process in a US setting: an application in cancer genomics. Med Decis Making. 2013 May;33(4):463-71. PMID: 23635833.

VI. Summary of Protocol Amendments

If we need to amend this protocol, we will give the date of each amendment, describe the change, and give the rationale in this section.

VII. Technical Experts

Technical Experts constitute a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes and identify particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design, and methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor do they contribute to the writing of the report. They have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The Task Order Officer and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

VIII. Peer Reviewers

Peer Reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. The EPC considers all peer review comments on the draft report in preparation of the final report. Peer Reviewers do not participate in writing or editing of the final report or other products. The final report does not necessarily represent the views of individual reviewers. The EPC will complete a disposition of all peer review comments. The disposition of comments for systematic reviews and technical briefs will be published 3 months after the publication of the evidence report.

Potential Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer Reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.

IX. EPC Team Disclosures

EPC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify EPC core team investigators.

X. Role of the Funder

This project was funded under Contract No. 290-2015-00009-I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.

XI. Registration

This protocol will be registered in the International Prospective Register of Systematic Reviews (PROSPERO).

Appendixes

Appendix A: Inclusion and Exclusion Criteria for the Systematic Review
PICOTS Inclusions / Exclusions
Study Designs Include: Comparative studies. Designs may include trials, cohort studies, natural experiments, and pre-post designs. Systematic reviews will be evaluated.
Exclude: Non-systematic reviews, narrative reviews, opinions, letters, descriptive articles, articles with no outcomes data. Modeling studies.
Populations Include: Patients (adult or pediatric) requiring consultations for prevention diagnosis, treatment, or management of acute or chronic conditions from a medical provider. Providers collaborating via telehealth about the clinical care of a specific patients (adult or pediatric) or group of patients.
Exclude: Patients receiving nonmedical services (e.g., social services, housing, and transportation).
Interventions Include: Telehealth consultation is defined as the use of telehealth designed to facilitate collaboration among providers, often involving a specialist, or between clinical team members, across time and/or distance, on the assessment, diagnosis, and/or clinical management of a specific patient or group of patients. While the patient may or may not be involved in the consultation, the consultation is required to be about a specific patient or group of patients in order to differentiate this from training or education.
Exclude: Other uses of telehealth. Consultations that do not involve telehealth. Any intervention that does not include an interaction between a health professional and patient, or between two health professionals. Training/education interventions that do not include a patient. Limited information provided by one clinician to another that does not contribute to collaboration (e.g., interpretation of an EEG, report on an x-ray or scan, or reporting the results of a diagnostic text) is not considered a consultation for this review.
Comparators Include: Usual care, consultations not using telehealth, non-receipt of services
Outcomes Include:
  • Clinical outcomes such as mortality, morbidity, function, recovery, infection, and access to care.
  • Economic outcomes such as return on investment, cost, volume of visits, and resource use.
  • Intermediate outcomes such as patient satisfaction, behavior, decisions; provider satisfaction, behavior, decisions; time to diagnosis, time to treatment; diagnostic accuracy or agreement between telehealth and in?person consultations.
  • Adverse effects or unintended consequences, such as loss of privacy, breech of data security, misdiagnosis or delayed diagnosis, or inappropriate treatment.
Timing/Setting Include:
  • Any setting, including rural or urban, home or community-based care, clinic, nursing home, or hospital-based care.
  • Any duration of follow-up.

Appendix B: Search Strategies

Database: Ovid MEDLINE(R) without Revisions 1996 to November Week 1 2016

Systematic reviews

  1. exp Telemedicine/
  2. Mobile Applications/
  3. telemedicine journal & e health.jn.
  4. "journal of telemedicine & telecare".jn.
  5. or/1-4
  6. limit 5 to (meta analysis or systematic reviews)
  7. meta-analysis.pt.
  8. meta-analysis/ or systematic review/ or meta-analysis as topic/ or "meta analysis (topic)"/ or "systematic review (topic)"/ or exp technology assessment, biomedical/
  9. ((systematic* adj3 (review* or overview*)) or (methodologic* adj3 (review* or overview*))).ti,ab.
  10. ((quantitative adj3 (review* or overview* or synthes*)) or (research adj3 (integrati* or overview*))).ti,ab.
  11. ((integrative adj3 (review* or overview*)) or (collaborative adj3 (review* or overview*)) or (pool* adj3 analy*)).ti,ab.
  12. (data synthes* or data extraction* or data abstraction*).ti,ab.
  13. (handsearch* or hand search*).ti,ab.
  14. (mantel haenszel or peto or der simonian or dersimonian or fixed effect* or latin square*).ti,ab.
  15. (met analy* or metanaly* or technology assessment* or HTA or HTAs or technology overview* or technology appraisal*).ti,ab.
  16. (meta regression* or metaregression*).ti,ab.
  17. (meta-analy* or metaanaly* or systematic review* or biomedical technology assessment* or bio-medical technology assessment*).mp,hw.
  18. (medline or cochrane or pubmed or medlars or embase or cinahl).ti,ab,hw.
  19. (cochrane or (health adj2 technology assessment) or evidence report).jw.
  20. (meta-analysis or systematic review).ti,ab.
  21. (comparative adj3 (efficacy or effectiveness)).ti,ab.
  22. (outcomes research or relative effectiveness).ti,ab.
  23. ((indirect or indirect treatment or mixed-treatment) adj comparison*).ti,ab.
  24. or/7-23
  25. 5 and 24
  26. 6 or 25
  27. limit 26 to yr="2016"

Randomized controlled trials and controlled observational studies – Broad search strategy

  1. exp Telemedicine/
  2. Mobile Applications/
  3. telemedicine journal & e health.jn.
  4. "journal of telemedicine & telecare".jn.
  5. or/1-4
  6. limit 5 to (clinical trial, all or comparative study or controlled clinical trial or pragmatic clinical trial or randomized controlled trial)
  7. 5 and (random* or control* or cohort).ti,ab.
  8. 6 or 7
  9. limit 8 to yr="2010 - 2016"

All study designs – Narrow search strategy

  1. exp Telemedicine/
  2. Mobile Applications/
  3. telemedicine journal & e health.jn.
  4. "journal of telemedicine & telecare".jn.
  5. exp Remote Consultation/
  6. consult*.mp.
  7. (or/1-4) and (5 or 6)
  8. limit 7 to yr="1996 - 2016"
  9. limit 8 to (meta analysis or systematic reviews)
  10. meta-analysis.pt.
  11. meta-analysis/ or systematic review/ or meta-analysis as topic/ or "meta analysis (topic)"/ or "systematic review (topic)"/ or exp technology assessment, biomedical/
  12. ((systematic* adj3 (review* or overview*)) or (methodologic* adj3 (review* or overview*))).ti,ab.
  13. ((quantitative adj3 (review* or overview* or synthes*)) or (research adj3 (integrati* or overview*))).ti,ab.
  14. ((integrative adj3 (review* or overview*)) or (collaborative adj3 (review* or overview*)) or (pool* adj3 analy*)).ti,ab.
  15. (data synthes* or data extraction* or data abstraction*).ti,ab.
  16. (handsearch* or hand search*).ti,ab.
  17. (mantel haenszel or peto or der simonian or dersimonian or fixed effect* or latin square*).ti,ab.
  18. (met analy* or metanaly* or technology assessment* or HTA or HTAs or technology overview* or technology appraisal*).ti,ab.
  19. (meta regression* or metaregression*).ti,ab.
  20. (meta-analy* or metaanaly* or systematic review* or biomedical technology assessment* or bio-medical technology assessment*).mp,hw.
  21. (medline or cochrane or pubmed or medlars or embase or cinahl).ti,ab,hw.
  22. (cochrane or (health adj2 technology assessment) or evidence report).jw.
  23. (meta-analysis or systematic review).ti,ab.
  24. (comparative adj3 (efficacy or effectiveness)).ti,ab.
  25. (outcomes research or relative effectiveness).ti,ab.
  26. ((indirect or indirect treatment or mixed-treatment) adj comparison*).ti,ab.
  27. or/10-26
  28. 8 and 27
  29. 9 or 28
  30. 8 not 29
  31. limit 30 to (english language and humans)
Database: EBM Reviews - Cochrane Central Register of Controlled Trials October 2016
  1. exp Telemedicine/
  2. (telemedicine or telehealth or teleconsult*).mp.
  3. 1 or 2
  4. 3 and (random* or control* or cohort).ti,ab.
  5. limit 4 to english language
Database: EBM Reviews - Cochrane Database of Systematic Reviews 2005 to November 09, 2016
  1. (telemedicine or telehealth or teleconsult*).mp.
  2. limit 1 to new reviews
Database - CINAHL Plus with Full Text
  • (MM "Telemedicine +")
  • consult*
  • S1 AND S2