I. Background and Objectives for the Evidence Product
Mobile health technology (mHealth) for diabetes was nominated to the Agency for Healthcare Research and Quality (AHRQ) Effective Health Care Program by a managed care pharmacist. The nominator is interested in the effectiveness of mHealth for diabetes self-management to inform the use of mHealth in clinical practice as well as third-party payer coverage policies. In early 2017, the AHRQ Scientific Resource Center (SRC) prepared a topic brief that identified a number of original research studies applicable to the nomination questions.1 In April 2017, the SRC identified several high-quality systematic reviews that could potentially address the questions from the nomination.2,3 However, these systematic reviews don't present information in a way that is useful for decision-makers who need to decide whether to use mHealth in clinical practice, and what type of technology to use.
The objective of this work is to develop and validate a product on a rapid timeline that will help decision-makers make informed choices about using mHealth for diabetes self-management. This type of product could promote the use of evidence about effectiveness of mHealth in decision-making, in addition to the mHealth's functions, usability, and cost.
Approximately 29 million Americans have some form of Diabetes Mellitus, which includes Type 1 diabetes, Type 2 diabetes, and gestational diabetes.4 Type 2 diabetes represents approximately 90 to 95% of all patients with diabetes, while Type 1 diabetes accounts for 5%. The Centers for Disease Control and Prevention (CDC) reports that in 2012, diabetes cost $245 billion due to related complications, medical costs, and lost wages.4 Diabetes was the seventh leading cause of death in the United States in 2013.4
For decades, diabetes self-management has been considered a cornerstone of diabetes care. Self-management is believed to play an important role in preventing diabetes and its complications. Components of self-management include: learning about diabetes; healthy eating, physical activity, medication and device usage, monitoring and using patient-generated data to adjust behaviors and medication doses; preventing, detecting, and treating acute and chronic complications; coping with psychosocial issues and concerns; and problem solving.5
Increasingly, clinicians, pharmacists and patients have started to use mHealth to assist with diabetes self-management. mHealth is defined as "the use of mobile and wireless technologies to support the achievement of health objectives."6 mHealth is typically patient-facing and are available on patients' mobile devices. mHealth overlaps with both telehealth and telemedicine; however, these terms are more broad and include all information and communication technologies to improve clinical care (telemedicine & telehealth) as well as public health, health administration and health-related education (telehealth).7-9 In this report, we define mHealth as any website, program, or application delivered through a mobile device (i.e., phone or tablet) for the purpose of diabetes self-management.
Like any educational or technological diabetes intervention, mHealth could potentially help patients implement self-management of diabetes by tracking and displaying data, providing educational resources, and offering support from peers and clinicians. Access to and use of these tools could help patients adhere to diet, exercise, and medication management plans, which could lead to improved health outcomes.
A wide range of mHealth technologies are available to individuals with diabetes. These technologies vary by the function they provide, including tracking blood glucose measurements, nutrition database and carbohydrate tracking, physical activity and weight tracking, sharing data with clinicians or peers, social support, messaging, and reminders.10 Some technologies only provide a single function, while others provide a group of functions. mHealth technologies are delivered on a variety of platforms, for example, through an application alone, through an application and online, or online only. mHealth technologies also vary by the types of device and operating systems required; some technologies are compatible across multiple devices and operating systems, others are not. mHealth technologies vary in the extent to which they connect to other aspects of a patient's care. For example, some mHealth technologies are designed to be used within an online patient portal, where patients and clinicians can exchange messages or other protected health information. In addition, some mHealth technologies connect directly to Food and Drug Administration (FDA)-regulated medical devices, such as a glucometer, which automatically upload information into an application.
This complexity of how mHealth technologies are designed and delivered, and the frequency with which these technologies are updated, makes it challenging to evaluate the literature and interpret results.
II. Guiding Questions
This list of guiding questions will be used to develop the literature search and inclusion/exclusion criteria.
- Which specific mHealth technologies for diabetes self-management have been researched?
- What are the characteristics (e.g., interoperability, functions, acceptability/usability, connection to electronic health records) of these specific mHealth technologies?
- What patient outcomes are associated with the use of these specific mHealth technologies?
- What are the harms and costs associated with these specific mHealth technologies?
A preliminary examination of the evidence suggests there are high-quality systematic reviews related to some of these questions. Therefore, we will leverage the information in existing systematic reviews to identify the best, most relevant studies, supplemented by a search for more recent studies, and present information on the effectiveness and features of specific mHealth technologies in a format that is useful for patients, clinicians, pharmacists and payers.
Types of translational products that fit this description include interactive decision aids; narrative summaries and product tables such as those published by the AHRQ Eisenberg Center11 and Consumer Report's Best Buy Drugs;12 and evidence maps.13 A recent AHRQ technical brief used both product tables and evidence maps to describe evidence about telehealth.14 The purpose of these products is to provide an overview of the evidence in a format that enables decision-makers to quickly understand the topic and make informed decisions.
III. Analytic Framework
Below is the analytic framework for this evidence product (Figure 1). This evidence product will examine adults with either Type 1 or Type 2 diabetes. It will summarize the characteristics of mHealth technologies, including functions (i.e., what does the technology do?), interoperability (i.e., is it compatible across devices, including both Apple and Android? Is it delivered on multiple platforms such as through a mobile application and online?), acceptability/usability (i.e., does the technology work? Is it easy to use? Do patients enjoy using it?) and connection to electronic health records (EHR) (i.e., does it sync to a patient's EHR? Does it connect to other aspects of a patient's health care?) or others based on our findings from the literature. The evidence product will examine three general categories of outcomes: interaction with the technology, patient-important outcomes, and health outcomes or others based on our findings from the literature.
Figure 1. Analytic Framework
Interviews with Key Informants
We will identify and invite decision-makers and stakeholders to serve as key informants on this evidence product. In general, stakeholders might disseminate the findings from the final evidence product, whereas decision-makers would use the product. We will invite representatives from the following organizations and agencies to serve as key informants: Academy of Managed Care Pharmacy (AMCP), Centers for Medicare and Medicaid Services (CMS), and the American Academy of Family Physicians (AAFP). We may invite representatives from additional organizations and agencies if we determine that additional perspectives are needed.
In addition to traditional interview methods that focus on defining the population, interventions, comparators, and outcomes (PICOs) of an evidence product, interviews with decision-makers will use principles of decision analysis methodology to 1) identify who the decision-makers are and how decisions are made, 2) determine what features of mHealth technologies are most important to evaluate, and 3) decide what the final product should be.15,16
In addition to these decision-makers, we will also interview experts in diabetes and mHealth to ensure our framework for evaluation of mHealth is aligned with current thinking on diabetes self-management and mHealth. All key informants will file Conflict of Interest (COI) and Confidentiality paperwork with the SRC.
Published Literature Search
We will search Ovid Medline and Cochrane Database of Systematic Reviews for high-quality systematic reviews or technology assessments published from 2008 to present. If we determine these reviews contain sufficient information to address the guiding questions, we will conduct a supplemental search for original research studies published since the end date of the reviews' literature searches. If these reviews only address some of the guiding questions, we will search for original research studies for questions not addressed by systematic reviews if possible.
2008 was chosen as the start date as this was the first year that mobile applications were available to consumers through the Apple and Android App stores.
We will request information from manufacturers, sponsors, and developers of mHealth. We may also search additional grey literature sources that are determined to be important based on key informant and topic expert interviews.
Process for Selecting Studies
The guiding questions will be used to determine eligibility for inclusion and exclusion of abstracts. All abstracts will be reviewed by a single reviewer. All citations determined to be appropriate for inclusion will be retrieved.
As discussed in the previous section, we will use a hierarchical approach to reviewing the evidence, beginning with systematic reviews and technology assessments; then other evidence reports, evidence syntheses, and original research studies. Identification of systematic reviews that fully address the guiding questions will preclude the need for lower-tier levels of evidence. To qualify as a systematic review/technology assessment, a study must include 1) a search of one or more citation databases, 2) include pre-specified inclusion and exclusion criteria, and 3) an assessment of the quality or risk of bias of identified studies. For questions where no systematic reviews are available, we will include primary research studies.
Data Organization and Presentation
Information management and data abstraction
As most systematic reviews on mHealth for diabetes examine a wide range of technologies, and our product aims to describe the evidence supporting specific technologies, we expect to extract information from individual studies. Identified studies and relevant study information, including type of study, length of study, study quality, characteristics of mHealth technology, study comparator, and evidence of effect on patient outcomes will be entered into an Excel spreadsheet. Study quality will be assessed either by extracting systematic reviews' quality assessments, or by conducting an assessment using a tool that adheres to AHRQ guidance for assessing quality. All study data will be verified for accuracy and completeness by a second reviewer.
The final evidence product will contain either a tabular presentation of the data on specific mHealth technologies or another type of graphical representation of data. The product will contain sufficient detail to allow end-users to understand the topic and the degree of evidence available on specific mHealth technologies.
V. Challenges with Evaluating and Translating Evidence on mHealth for Diabetes
We anticipate there will be considerable challenges in evaluating and translating the evidence on mHealth for diabetes. Examples of these challenges include issues surrounding the availability of mHealth technologies, data security, patient safety, complexity of app delivery, individual preferences, and consideration for particular sub-populations such as the elderly and those with diabetes-related disabilities. These challenges will be discussed with key informants and other topic experts to identify which challenges are most important to address, and to identify potential solutions.
- Veazie S, Relevo R, McKenna R, Winchell K, Helfand M. mHealth for Diabetes Topic Disposition. Agency for Healthcare Research and Quality Effective Health Care Program. 2017; /sites/default/files/tnd-0855-170515.pdf.
- Kitsiou S, Pare G, Jaana M, Gerber B. Effectiveness of mHealth interventions for patients with diabetes: An overview of systematic reviews. PloS one. 2017;12(3):e0173160.
- Wu Y, Yao X, Vespasiani G, et al. Mobile App-Based Interventions to Support Diabetes Self-Management: A Systematic Review of Randomized Controlled Trials to Identify Functions Associated with Glycemic Efficacy. JMIR mHealth and uHealth. 2017;5(3):e35.
- Centers for Disease Control and Prevention. Diabetes. 2015; http://www.cdc.gov/diabetes/basics/index.html. Accessed May 1, 2017.
- Beck J GD, Blanton L, Bollinger ST, Butcher MK et al. 2017 National Standards for Diabetes Self-Management Education and Support. http://professional.diabetes.org/sites/professional.diabetes.org/files/media/
2017_national_standards_for_dsmes_public_comment.pdf. Accessed May 12, 2017.
- World Health Organization. mHealth: New horizons for health through mobile technologies. 2011; http://www.who.int/goe/publications/goe_mhealth_web.pdf. Accessed Jan 10, 2017.
- U.S. Department of Health and Human Services, Federal Office of Rural Health Policy. Telehealth Programs. Nov 2015; http://www.hrsa.gov/ruralhealth/telehealth/. Accessed Jan 10, 2017.
- World Health Organization. Telemedicine: opportunities and developments in member states. 2010; http://www.who.int/goe/publications/goe_telemedicine_2010.pdf. Accessed Jan 10, 2017.
- van Dyk L. A Review of Telehealth Service Implementation Frameworks. International Journal of Environmental Research and Public Health. 01/23/2014;11(2):1279-1298.
- Ristau RA, Yang J, White JR. Evaluation and Evolution of Diabetes Mobile Applications: Key Factors for Health Care Professionals Seeking to Guide Patients. Diabetes Spectrum. 2013;26(4):211.
- Agency for Healthcare Research and Quality. Lung cancer screening tools for patients & clinicians. Mar 2016; AHRQ Publication No. 16-EHC007-012-A. Available at: https://effectivehealthcare.ahrq.gov/decision-aids/lung-cancer-screening/. Accessed May 15, 2017.
- Consumer Reports, Best Buy Drugs. Best treatments for depression. Apr 2017; http://www.consumerreports.org/cro/2013/09/best-treatments-for-depression/index.htm. Accessed May 15, 2017.
- Miake-Lye IM, Hempel S, Shanman R, Shekelle PG. What is an evidence map? A systematic review of published evidence maps and their definitions, methods, and products. Systematic Reviews. 2016;5(1):28.
- Totten AM, Womack DM, Eden KB, et al. Telehealth: Mapping the Evidence for Patient Outcomes From Systematic Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2016.
- Howard R, and Abbas, AE. Foundations of decision analysis. Pearson; 2015.
- Owens DK, Nease RF, Jr. Development of outcome-based practice guidelines: a method for structuring problems and synthesizing evidence. The Joint Commission journal on quality improvement. Jul 1993;19(7):248-263.
VII. Definition of terms
VIII. Summary of Protocol Amendments
In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.
IX. Key Informants
Within the Evidence Product process, Key Informants serve as a resource to offer insight into the clinical context of the technology/intervention, how it works, how it is currently used or might be used, and which features may be important from a patient of policy standpoint. They may include clinical experts, patients, manufacturers, researchers, payers, or other perspectives, depending on the technology/intervention in question. Differing viewpoints are expected, and all statements are crosschecked against available literature and statements from other Key Informants. Information gained from Key Informant interviews is identified as such in the report. Key Informants do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.
Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the SRC work to balance, manage, or mitigate any potential conflicts of interest identified.
X. Peer Reviewers
Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the SRC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers.
Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.
XI. SRC Team Disclosures
SRC core team members must disclose any financial conflicts of interest greater than $1,000 and any other relevant business or professional conflicts of interest. Related financial conflicts of interest that cumulatively total greater than $1,000 will usually disqualify SRC core team investigators.
XII. Role of the Funder
This project was funded by the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.