Draft Key Questions
Disposition of Comments Report
Notice: This protocol was amended on March 12, 2015. To view the amendments go to Summary of Protocol Amendments on this page.
Background and Objectives for the Systematic Review
The use of health information technology (IT) has been promoted as having promise in improving the efficiency, cost-effectiveness, quality, and safety of medical care delivery in the U.S. health care system.1 Health IT has the potential to support patient care related activities such as order communications, results reporting, care planning, and clinical or health documentation. Examples of health IT applications include the electronic health record (EHR), telemedicine, clinical alerts and reminders, computerized provider order entry, clinical decision support systems, electronic access to best practice guidelines and evidence databases, consumer health informatics applications, and electronic exchange of health information. In recent years, the Health Information Technology for Economic and Clinical Health (HITECH) Act has accelerated adoption of the EHR in ambulatory and hospital settings across the United States. The HITECH Act, part of the American Recovery and Reinvestment Act of 2009 (ARRA), is providing up to $29 billion in incentive funding for the adoption and “meaningful use” of health IT. Due to HITEC funding, over 80 percent of eligible hospitals and 50 percent of eligible professionals are now using certified EHR technology.2 The motivation to increase the use of health IT in health care is grounded in evidence that health IT can improve the quality, safety, satisfaction, and efficiency of health care, as reported in recent systematic reviews.3-6
A key challenge to effective use of health IT, however, is the fact that most Americans, especially those with multiple illnesses, receive care in multiple settings. In Massachusetts, out of 3.7 million patients hospitalized, 31 percent visited two or more hospitals over five years (57% of all visits) and 1 percent visited five or more hospitals (10% of all visits).7 Similarly, an analysis of 2.8 million emergency department patients in Indiana found that 40 percent had data at multiple institutions.8 This presents a challenge if we are to meet the goal stated by former AHRQ Director Dr. Carolyn Clancy that, “data should follow the patient” wherever they get their care.9
To enable patient records to follow patients wherever they receive care, increased attention has been paid to health information exchange (HIE), which has been defined as the reliable and interoperable electronic sharing of clinical information among doctors, nurses, pharmacists, other health care providers and patients across the boundaries of health care institutions, health data repositories, States and others, not within a single organization or among affiliated providers, while protecting the integrity, confidentiality, privacy and security of the information.10 The HITECH Act recognized that EHR adoption alone would not be sufficient to achieve the full value of health IT, allocating another $563 million for State-based HIE.11 In the meantime, there has been a growing number of private organizations undertaking HIE.12 HIE across provider organizations may help coordinate care transitions between settings, improve patient safety and reduce unnecessary duplicate testing.
The Office of the National Coordinator for Health IT (ONC) has defined three key forms of HIE.11
- Directed exchange – sending and receiving of secure information electronically between care providers
- Query-based exchange – providers being able to find or request information on a patient from other providers
- Consumer-mediated exchange – patients aggregating and controlling the use of their health information among care providers
An early successful example of HIE was the work of Clement McDonald, MD, who pioneered HIE in Indiana starting in the 1990s.13 The work led to the formation of the Indiana Health Information Exchange, which has been one of the largest and most successful HIE effort in the United States.14 Other early programs aiming to achieve HIE, including some high-profile ones, were not as successful.15 Although the rationale for HIE is critically important,16 the path to achieving it has been more difficult than the adoption of EHRs,17,18 in part due to the lack of sustainable business models.19-23
Evaluating the effectiveness of HIE is also challenging. HIE is an intermediate technology designed to improve health care delivery and is not specific to any disease. HIE implementations are often funded by one-time start-up awards, without longer-term support to sustain and evaluate the interventions. However, the promise for HIE to improve health care delivery is substantial. It is therefore critical to be able to determine if HIE does improve health or intermediate outcomes as well as to systematically assess comparative approaches, barriers, return on investment, and sustainability of HIE.
The review to be undertaken by the EPC is timely and necessary—our knowledge of and experience with the HIE literature indicates that the evidence base is scattered across disciplines and in various formats with only one previously published systematic review that focused exclusively on HIE.24 Meanwhile, AHRQ has funded a large portfolio of research in health IT and HIE.25 In requesting this review, AHRQ’s goal is a report that will systematically identify and synthesize evidence on the extent to which HIE is effective in improving a variety of outcomes and how the impact varies by different approaches to HIE. The report will also identify evidence on levels of use, and usability of HIE, as well as facilitators of and barriers to HIE. Additional aims are to determine what evidence is available or needed regarding the value of HIE for implementers and users, the extent to which system characteristics influence the effectiveness of HIE in improving outcomes, and what attributes influence sustainability of HIE.
The draft Key Questions (KQs) developed during Topic Refinement were available for public comment from February 6 to February 26, 2014. The comments did not lead to significant changes but were helpful in identifying additional factors of interest in KQ 4 and KQ 5, and for clarifying the wording of the questions.
Based on the public comments and subsequent discussions with AHRQ, the following changes of note were made to the KQs:
- KQ 4: Added “provider type” to KQ 4b. Added an additional sub question of “Do level of use and primary uses vary by data source?”
- KQ 5: Added an additional sub question of “How does usability vary by health care settings or systems?”
The revised KQs are as follows:
- Is health information exchange (HIE) effective in improving clinical (e.g., mortality and morbidity), economic (e.g., costs and resource use, the value proposition for HIE) and population (e.g., syndromic surveillance) outcomes?
- Does effectiveness vary by type of HIE?
- Does effectiveness vary by health care settings and systems?
- Does effectiveness vary by IT system characteristics?
- What evidence exists that the lack of HIE leads to poorer outcomes?
- What harms have resulted from HIE? (e.g., violations of privacy, errors in diagnosis or treatment from too much, too little or inaccurate information, or patient or provider concerns about HIE)?
- Do harms vary by type of HIE?
- Do harms vary by health care settings and systems?
- Do harms vary by the IT system characteristics?
- Is HIE effective in improving intermediate outcomes such as patient and provider experience, perceptions or behavior; health care processes; or the availability, completeness, or accuracy of information?
- Does effectiveness in improving intermediate outcomes vary by type of HIE?
- Does effectiveness in improving intermediate outcomes vary by health care settings and systems?
- Does effectiveness in improving intermediate outcomes vary by IT system characteristics?
- What evidence exists that the lack of HIE leads to poorer intermediate outcomes?
- What is the current level of use and primary uses of HIE?
- Do level of use and primary uses vary by type of HIE?
- Do level of use and primary uses vary by health care settings and systems, or provider type?
- Do level of use and primary uses vary by IT system characteristics?
- Do level of use and primary uses vary by data source?
- How does the usability of HIE impact effectiveness or harms for individuals and organizations?
- How usable are various types of HIE?
- What specific usability factors impact the effectiveness or harms from HIE?
- How does usability vary by health care settings or systems?
- What facilitators and barriers impact implementation of HIE?
- Do facilitators and barriers that impact implementation vary by type of HIE?
- Do facilitators and barriers that impact implementation vary by health care settings and systems?
- Do facilitators and barriers that impact implementation vary by IT system characteristics?
- What facilitators and barriers impact use of HIE?
- Do facilitators and barriers that impact use vary by type of HIE?
- Do facilitators and barriers that impact use vary by health care settings and systems?
- Do facilitators and barriers that impact use vary by IT system characteristics?
- What factors influence sustainability of HIE?
Any individual or group of health care providers, patients, managers, health care institutions, or regional organizations.
Heath Information Exchange (HIE). HIE is defined as the electronic sharing of clinical information among users such as health care providers, patients, administrators or policy makers across the boundaries of health care institutions, health data repositories, States and others, typically not within a single organization or among affiliated providers, while protecting the integrity, privacy, and security of the information.
- Time period prior to HIE implementation
- Locations (geographic or organizational without HIE)
- Situations in which HIE is not available, akin to "usual care" in a clinical study
- Comparisons across types of HIE
- Comparisons of the characteristics of the different settings, health care system, and IT systems in which HIE is used
Outcomes (specified for each Key Question)
KQ 1: Effectiveness is defined in terms of clinical outcomes (e.g., mortality and morbidity), economic outcomes (e.g., costs and resource use, the value proposition for HIE) and population outcomes (e.g., syndromic surveillance for the identification of trends or clusters).
KQ 2: Harms include unintended negative consequence or adverse events experienced by individuals, institutions, or organizations. Harms from HIE may include negative outcomes or the risk of negative outcomes resulting from information that is wrong, not provided in a timely manner, or in formats that inhibit its identification, comprehension, and use. Harms also may result from too much information as well as lack of information. Harms can also include negative impacts on attitudes (e.g., patients not trusting the privacy will be protected, clinicians' concerns about legal liability).
KQ 3: Intermediate outcomes include outcomes such as provider and patient experience and perceptions; changes in provider behavior and health care processes; and changes in the availability, completeness, or accuracy of information.
KQ 4: Level of use is the rate of HIE use by individuals, health care institutions, or regional organizations.
KQ 5: Usability focuses on the function of the HIE in terms of the interaction between users and HIE and their ability to navigate and accomplish tasks.
KQ 6: Implementation of HIE is defined as the realization of an HIE project such that the exchange of data is operational.
KQ 7: Use is the incorporation of the HIE into the workflow and decisions of patients, providers or organizations.
KQ 8: Sustainability is long-term maintenance, and improvement or expansion of HIE, after the implementation period.
No minimum duration of time lapsed from implementation of HIE to the measurement of outcomes.
Any aspect of the setting in which health information is exchanged for the purpose of improving health or health care decisions that is hypothesized to impact effectiveness, use, usability or sustainability. This may include the type(s) of clinical environments (e.g., ambulatory care, hospital, nursing home, etc.), payment/reimbursement model(s) (e.g., fee-for-service, managed care setting, risk/value-based model such as an accountable care organization, etc.), and legislative requirements (e.g., participation in HIE required to participate in Medicaid).
Figure 1. Analytic Framework for Health Information Exchange
A. Criteria for Inclusion/Exclusion of Studies in the Review
The criteria for inclusion and exclusion of studies will be based on the KQs and the PICOTS in the previous section. As the questions are broad and diverse, we anticipate that different types of evidence will be included. However, included studies must respond to a KQ, which implies that they include relevant or comparative data about HIE as it is defined for this review.
For efficacy or effectiveness, a "best evidence" approach will be used. Randomized controlled clinical trials (RCTs) and systematic reviews of such trials will be included as the top-tier evidence. If insufficient evidence is found of this type, we will explore observational study evidence (defined as cohort studies comparing at least two HIE systems, case-control studies, and time-series studies).
For harms, in addition to RCTs, observational studies (defined as cohort studies comparing at least two concurrent HIE systems, case-control studies, and time-series studies) will be included.
Given that several of the key questions expand the scope of inquiry beyond effectiveness and harms to topics such as use, usability and sustainability, observational studies and qualitative research will be included.
We will also evaluate systematic reviews and at a minimum, all systematic reviews will be considered as sources of studies to be reviewed for possible inclusion. Whether the synthesis from existing reviews (the results) can be used by us depends on whether any review we locate contains information directly relevant to our KQs and whether the review is of high quality. High quality reviews will be defined as those assessed as being at low risk of bias, according to the AMSTAR quality assessment tool.26,27 For this review of HIE, the prior systematic reviews we have evaluated to date do not directly address the KQs. If relevant, high quality reviews are identified, we will consult with our TEP, AHRQ TOO, and methods experts who have worked in this area to determine the best way to use the results and integrate them with evidence from individual studies.
Modeling and Planning Studies: We will not include studies that model the potential impact of HIE or that present, discuss, or evaluate hypothetical situations of HIE that has not yet been implemented. For a study to be included it has to be about HIE that has actually occurred.
Non-English Language Studies: We will restrict to English-language articles, but will review English language abstracts of non-English language articles to identify studies that would otherwise meet inclusion criteria, in order to assess for the likelihood of language bias.
B. Searching for the Evidence: Literature Search Strategies and Identification of Relevant Studies to Answer the Key Questions
Publication Date Range
Searches will include articles published between January 1990, which reflects the timing of initial implementations of HIE in the United States, and May 2014.
Library searches will be updated while the draft report is posted for public comment and out for peer review to include articles published in June through December 2014 in order to capture any new publications. Literature identified during the update search will be assessed by following the same process of dual review as all other studies considered for inclusion in the report. If any pertinent new literature is identified for inclusion in the report, it will be incorporated before the final submission of the report.
Ovid MEDLINE, PsycINFO, CINHAL, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Health Technology Assessment, National Health Sciences Economic Evaluation Database, and Database of Abstracts of Reviews of Effects will be searched to capture both published and gray literature. The search strategies developed in Ovid MEDLINE (Appendix 1) have been peer reviewed by another librarian who offered suggestions and confirmed accuracy.
During our literature scan we reviewed two additional databases: Business Premier and Institute of Electrical and Electronics Engineers, both of which were determined to produce low yield of relevant articles and therefore will not be searched for this systematic review.
Scientific Information Packets
An equivalent of Scientific Information Packets (SIPs) for this topic is data from research or evaluations of HIE that have not been published or indexed in citation databases. While we search gray literature sources ourselves, including looking for reports and analyses on the web sites of key organizations, we will also contact organizations and individuals (via the Scientific Resource Center) and request any unpublished reports or data. The request letter describes the project and specifies how this information will be used.
The following are examples of organizations that will be contacted:
- American Electronic Health Record
- American Health Information Management Association
- American Medical Informatics Association
- Center for Studying System Change
- Commonwealth Fund
- eHealth Initiative
- EHR Intelligence
- Healthcare Information and Management Systems Society
- Henry J. Kaiser Family Foundation
- HHS Office of Disability, Aging and Long-term care Policy
- Health Level 7
- Institute of Electrical and Electronics Engineers
- Inland Northwest Health Services
- John and Mary R. Markle Foundation
- Mathematica Policy Research
- National Academy of Social Insurance
- National Governors Association Center for Best Practices, State Alliance for e-health
- Office of the National Coordinator for Health Information Technology, such as State HIE Bright Spots
- PriceWaterhouse Coopers
- Public Health Informatics Institute
- Rhode Island Quality Initiative (Laura Adams)
- Robert Wood Johnson Foundation
- Shared Care Plan in Whatcom
- United Hospital Fund of New York
- Washington and Idaho Regional Extension Center – Qualis Health
- Washington State DOH, HIE
- Workgroup for Electronic Data Exchange
Reference lists of included articles will also be reviewed for includable literature.
In the event that information regarding methods or results appears to be omitted from the published results of a study, or if we are aware of unpublished data, we will contact the authors to request this information.
Abstract and Article Review Procedures
Initial review of abstracts will be done to determine if a citation is relevant. At this stage the only criteria are that an abstract presents data about HIE and that there is an English-language abstract if the article is in a foreign language. To ensure accuracy, all excluded abstracts will be reviewed by a second person. All citations deemed potentially eligible for the review by at least one of the reviewers will be retrieved for full-text review. Each full-text article will be independently reviewed for eligibility by two team members using the inclusion/exclusion criteria outlined above. Any disagreements will be resolved by consensus.
A record of studies excluded at the full-text level with reasons for exclusion will be maintained.
C. Data Abstraction and Data Management
After studies are selected for inclusion, data will be abstracted into categories that include but are not limited to: a) general information such as study design, year, setting, geographic location, and duration; b) characteristics of the HIE such as the form (directed exchange, query-based exchange, consumer-mediated exchange), the number and types of participating organizations, the type of user interface (e.g., push versus pull), and the types of information included; and c) key contextual information that will be used to identify facilitators and barriers to HIE use as well as assess applicability of the results. At a minimum, these may include details about the type(s) of clinical environments (e.g., ambulatory care, hospital, nursing home, etc.), payment/reimbursement model(s) (e.g., fee-for-service, managed care setting, risk/value-based model such as an accountable care organization, etc.), and legislative requirements (e.g., participation in HIE required to participate in Medicaid). We will abstract the data for the outcomes for each KQ as outlined in the PICOTS section.
D. Assessment of Methodological Risk of Bias of Individual Studies
Our assessment of risk of bias will be based on the recommendations in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter, Methods Guide).26 Included studies will be classified according to type of design (e.g., randomized trial, nonrandomized trial, observational study, etc.) as part of the data abstraction phase, and each major type of study will be assessed for bias according to relevant criteria. These criteria included questions to assess selection bias, performance bias, detection bias, attrition bias, and reporting bias (i.e. those about adequacy of randomization, similarity of groups at baseline, appropriateness of the comparators, consideration of concurrent interventions or unintended exposures, quantity of missing data, methods of handling missing data, identification and assessment of important confounding variables, use of intention-to-treat analysis, reliability and validity of outcome measures, and reporting of pre specified outcomes).
Quality ratings of all articles will be made by two raters. Differences will be resolved by discussion and, involvement of a third rater as needed.
Individual studies will be rated as "good," "fair," or "poor." or as specified by the particular criteria.
Studies rated "good" will be considered to have the least risk of bias, and their results will be considered valid. Good quality studies include clear descriptions of the population, setting, interventions, and comparison groups clear reporting of missing data; appropriate means for preventing bias; and appropriate measurement of outcomes.
Studies rated "fair" will be susceptible to some bias, though not enough to necessarily invalidate the results. These studies may not meet all the criteria for a rating of good quality, but do not have flaws likely to cause major bias. The study may be missing information, making it difficult to assess limitations and potential problems. The fair quality category is broad, and studies with this rating will vary in their strengths and weaknesses. The results of some fair quality studies are likely to be valid, while others may be only possibly valid.
Studies rated "poor" will have significant flaws that imply biases of various types that may invalidate the results. They will have a serious or "fatal" flaw in design, analysis, or reporting; large amounts of missing information; or discrepancies in reporting. The results of these studies will be least as likely to reflect flaws in the study design as the true difference between the compared interventions. We will not exclude studies rated as being poor in quality a priori, but poor quality studies will be considered to be less reliable than higher quality studies when synthesizing the evidence, particularly if discrepancies between studies are present.
E. Data Synthesis
We will construct evidence tables identifying the study characteristics, results of interest, and quality ratings for all included studies and summary tables to highlight the main findings. We will review and highlight studies by using a hierarchy of evidence approach, where the best evidence is the focus of our synthesis for each key question.
If several studies are identified that are similar in terms of the intervention (HIE), outcomes, and study design for a specific key question, we will consider a quantitative meta-analysis. Given that there is considerable heterogeneity in HIE in terms of structures of systems, how they are implemented and how they are actually used, we anticipate that quantitative meta-analysis many not be advisable. If this proves to be the case, we will use qualitative groups to identify trends in study findings.
F. Grading the Strength of Evidence (SOE) for Individual Comparisons and Outcomes
The strength of evidence for each key question will be initially assessed for the range of outcomes (see PICOTS) by using the approach described in the AHRQ Methods Guide.26 To ensure consistency and validity of the evaluation, the grades will be reviewed by the entire team of investigators for:
- Study limitations (low, medium, or high level of study limitations)
- Consistency (consistent, inconsistent, or unknown/not applicable)
- Directness (direct or indirect)
- Precision (precise or imprecise)
- Reporting bias (suspected or undetected) for trials
The strength of evidence will be assigned an overall grade of high, moderate, low, or insufficient according to a four-level scale by evaluating and weighing the combined results of the above domains:
- High: We are very confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has few or no deficiencies. We believe that the findings are stable, i.e., another study would not change the conclusions.
- Moderate: We are moderately confident that the estimate of effect lies close to the true effect for this outcome. The body of evidence has some deficiencies. We believe that the findings are likely to be stable, but some doubt remains.
- Low: We have limited confidence that the estimate of effect lies close to the true effect for this outcome. The body of evidence has major or numerous deficiencies (or both). We believe that additional evidence is needed before concluding either that the findings are stable or that the estimate of effect is close to the true effect.
- Insufficient: We have no evidence, we are unable to estimate an effect, or we have no confidence in the estimate of effect for this outcome. No evidence is available or the body of evidence has unacceptable deficiencies, precluding reaching a conclusion.
G. Assessing Applicability
Applicability will be estimated by considering the characteristics of the population and setting as defined in the PICOTS above. This may include differences in the organizations (e.g., payment/reimbursement model, range of services provided, governance structure, IT systems etc.) and people (profession, type of relationship with the organization, tenure, etc.) affected by the creation and implementation of the HIE that was the subject of study, the scope of the HIE, the clinical settings involved and the geographic area (e.g., states, regions or countries) in which the studies are performed.
Variability in the studies may limit the ability to generalize the results to other populations and settings.
- Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
- Department of Health and Human Services. Doctors and hospitals' use of health IT more than doubles since 2012. Washington, DC. Available at: http://www.hhs.gov/news/press/2013pres/05/20130522a.html. Accessed April 18, 2014.
- Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144(10). PMID: 16702590
- Goldzweig CL, Towfigh A, Maglione M, et al. Costs and benefits of health information technology: new trends from the literature. Health Aff. 2009;28(2). PMID: 19174390
- Buntin MB, Burke MF, Hoaglin MC, et al. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff. 2011;30(3). PMID: 21383365
- Jones SS, Rudin RS, Perry T, et al. Health information technology: an updated systematic review with a focus on meaningful use. Ann Intern Med. 2014;160(1). PMID: 24573664
- Bourgeois FC, Olson KL, Mandl KD. Patients treated at multiple acute health care facilities: quantifying information fragmentation. Arch Intern Med. 2010;170(22). PMID: 21149756
- Finnell JT, Overhage JM, Grannis S. All health care is not local: an evaluation of the distribution of emergency department care delivered in Indiana. AMIA Annu Symp Proc. 2011;2011. PMID: 22195094
- Clancy C. Keynote Adress AMIA 2007 Anual Symposium. 2007.
- What is HIE (Health Information Exchange)? Department of Health and Human Services. Washington, DC. Available at: http://www.healthit.gov/providers-professionals/health-information-exchange/what-hie. Accessed April 18, 2014.
- Williams C, Mostashari F, Mertz K, et al. From the Office of the National Coordinator: the strategy for advancing the exchange of health information. Health Aff. 2012;31(3). PMID: 22392663
- Health Information Exchange Roadmap: The Landscape and a Path Forward. National eHealth Collaborative. Washington, DC. Available at: http://www.nationalehealth.org/hie-roadmap. Accessed April 18, 2014.
- McDonald CJ. The barriers to electronic medical record systems and how to overcome them. J Am Med Inform Assoc. 1997;4(3). PMID: 9147340
- McDonald C, Overhage J, Barnes M, et al. The Indiana Network for Patient Care: a working local health information infrastructure. Health Aff. 2005;24. PMID: 16162565
- Frohlich J, Karp S, Smith MD, et al. Retrospective: lessons learned from the Santa Barbara project and their implications for health information exchange. Health Aff. 2007;26(5). PMID: 17670777
- Kuperman GJ. Health-information exchange: why are we doing it, and what are we doing? J Am Med Inform Assoc. 2011;18(5). PMID: 21676940
- McGowan J, Kuperman G, Olinger L, et al. Strengthening Health Information Exchange: Final Report HIE Unintended Consequences Work Group. Rockville, MD: Westat; 2012. Available at: http://www.healthit.gov/sites/default/files/hie_uc_workgroup_final_report.pdf. Accessed January 7, 2014.
- Yasnoff WA, Sweeney L, Shortliffe EH. Putting health IT on the path to success. JAMA. 2013;309(10). PMID: 23483170
- Adler-Milstein J, Landefeld J, Jha AK. Characteristics associated with regional health information organization viability. J Am Med Inform Assoc. 2010;17(1). PMID: 20064803
- Adler-Milstein J, Bates DW, Jha AK. A survey of health information exchange organizations in the United States: implications for meaningful use. Ann Intern Med. 2011;154(10). PMID: 21576534
- Adler-Milstein J, Jha AK. Sharing clinical data electronically: a critical challenge for fixing the health care system. JAMA. 2012;307(16). PMID: 22535851
- Adler-Milstein J, Bates DW, Jha AK. Operational health information exchanges show substantial growth, but long-term funding remains a concern. Health Aff. 2013;32(8). PMID: 23840051
- Grossman JM, Kushner KL, November EA. Creating sustainable local health information exchanges: can barriers to stakeholder participation be overcome? Res Briefs. 2008(2). PMID: 18496926
- Hincapie A, Warholak T. The impact of health information exchange on health outcomes. Appl Clin Inform. 2011;2(4). PMID: 23616891
- Yi R, Samarth A, Dearfield C, et al. Synthesis of Lessons Learned in the First 5 Years of State and Regional Demonstration Health Information Exchange Projects. AHRQ Publication No. 11-0050-EF. Rockville, MD: Agency for Healthcare Quality and Research; March, 2011. Available at: http://healthit.ahrq.gov/sites/default/files/docs/citation/synthesis_of_lessons_learned.pdf. Accessed April 18, 2014.
- Methods Guide for Effectiveness and Comparative Effectiveness Reviews. AHRQ Publication Number 10(14)-EHC062-EF. Rockville, MD: Agency for Healthcare Research and Quality; January 2014. Available at: www.effectivehealthcare.ahrq.gov. Accessed April 18, 2014. PMID: 21433403
- Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10). PMID: 19230606
- McDonagh MS, Jonas DE, Gartlehner G, et al. Methods for the drug effectiveness review project. BMC Med Res Methodol. 2012;12. PMID: 22970848
- Whitlock EP, Lin JS, Chou R, et al. Using existing systematic reviews in complex systematic reviews. Ann Intern Med. 2008;148(10). PMID: 18490690
- White CM, Ip S, McPheeters M, et al. Using Existing Systematic Reviews To Replace De Novo Processes in Conducting Comparative Effectiveness Reviews. 2008. PMID: 21433402
- Common industry specification for usability requirements NISTIR 7432. Publication number IUSR:CISU-R v0.90. Gaithersburg, MD: National Institute of Standards and Technology; 2007.
Definition of Terms
- Health information exchange (HIE): the electronic sharing of clinical information among users such as health care providers, patients, administrators or policy makers across the boundaries of health care institutions, health data repositories, States and others, typically not within a single organization or among affiliated providers, while protecting the integrity, privacy, and security of the information.
- Sustainability: refers to the endurance of systems and processes.
- Usability: the “...effectiveness, efficiency and satisfaction with which the intended users can achieve their tasks in the intended context of product use.”31
Summary of Protocol Amendments
|Date||Section||Original Protocol||Revised Protocol||Rationale|
|3/12/15||Key Questions||6. What facilitators and barriers impact implementation of HIE?
6a. Do facilitators and barriers that impact implementation vary by type of HIE?
6b. Do facilitators and barriers that impact implementation vary by health care settings and systems?
6c. Do facilitators and barriers that impact implementation vary by IT system characteristics?
7. What facilitators and barriers impact use of HIE?
7a. Do facilitators and barriers that impact use vary by type of HIE?
7b. Do facilitators and barriers that impact use vary by health care settings and systems?
7c. Do facilitators and barriers that impact use vary by IT system characteristics?
|Revised order of Key Questions 6 and 7.
Key Question 7 re-numbered to be Key Question 6, and Key Question 6 to be Key Question 7.
|This was done because studies often measured data on usability (Key Question 5) and barriers and facilitators to use (old Key Question 7, new Key Question 6) in the same study, and the report flows better with these presented together. Similarly, studies often measured implementation barriers and facilitators (old Key Question 6, new Key Question 7) along with measures of sustainability (Key Question 8) such that the flow of the report is better with these questions considered together.|
Review of Key Questions
For all EPC reviews, KQs were reviewed and refined as needed by the EPC with input from Key Informants and the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed. In addition, for Comparative Effectiveness reviews, the KQs were posted for public comment and finalized by the EPC after review of the comments.
Key informants are the end-users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the key informant role is to provide input into identifying the KQs for research that will inform health care decisions. The EPC solicits input from key informants when developing questions for systematic review or when identifying high-priority research gaps and needed new research. Key informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Key informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as key informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
Technical experts comprise a multi-disciplinary group of clinical, content, and methodologic experts who provide input on how to identify, synthesize and present the evidence for a review.in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicting opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore, study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical experts do not do analysis of any kind nor do they contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Technical experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as technical experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified. Potential conflicts of interest are also managed by not releasing the names of technical experts until publication of the final report.
Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published three months after the publication of the Evidence report.
Potential reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited peer reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism. Potential conflicts of interest are also managed by not releasing the names of peer reviewers until publication of the final report.
EPC Team Disclosures
The EPC project team has no conflicts of interest to report.
Role of the Funder
This project was funded under Contract No. HHSA 290201200014I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements and quality. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.
Database: Ovid MEDLINE(R) and Ovid OLDMEDLINE(R) Search Strategy:
- (health information adj5 exchang$).mp. (445)
- hie.mp. (953)
- exp Medical Records/ (85293)
- exp Systems Analysis/ (34998)
- exp Medical Informatics/ (320040)
- Information Dissemination/ (10118)
- 3 or 4 or 5 or 6 (411857)
- 2 and 7 (218)
- 1 or 8 (533)
- health information organization$.mp. (70)
- 7 and 10 (66)
- (hio or hios or rhio or rhios).mp. (195)
- 7 and 12 (116)
- ((clinical$ or health$) adj5 (data adj3 exchang$)).mp. (192)
- 7 and 14 (153)
- (patient$ adj2 match$).mp. (12897)
- 7 and 16 (588)
- ((query or querie$) adj3 (base or based or bases or basing) adj5 exchang$).mp. (1)
- 7 and 18 (1)
- directed exchang$.mp. (9)
- 7 and 20 (0)
- ((consumer$ or patient$) adj5 mediat$ adj7 exchang$).mp. (12)
- 7 and 22 (0)
- ((health information adj5 tech$) and exchang$).mp. (127)
- 7 and 24 (117)
- (health information adj7 network$).mp. (433)
- 7 and 26 (333)
- ((health information or ((electronic$ or computer$) adj2 (health or medic$ or patient$) adj2 record$) or ehr or emr) adj7 exchang$).mp. (527)
- 7 and 28 (455)
- (exchang$ adj5 network$).mp. (463)
- 7 and 30 (116)
- (interoperab$ adj7 standard$).mp. (320)
- 7 and 32 (270)
- ((inter or between or across) adj3 (organization$ or systems) adj7 network$).mp. (159)
- 7 and 34 (44)
- 9 or 11 or 13 or 15 or 17 or 19 or 21 or 23 or 25 or 27 or 29 or 31 or 33 or 35 (2045)
- Medical Record Linkage/ (3570)
- exp systems integration/ (8043)
- 37 and 38 (331)
- exp Cooperative Behavior/ (28784)
- 37 and 40 (62)
- exp Medical Informatics Applications/ (314549)
- 37 and 42 (1741)
- 10 or 12 or 14 or 16 or 18 or 20 or 22 or 24 or 26 or 28 or 30 or 32 (14873)
- 43 and 44 (180)
- 36 or 39 or 41 or 45 (2359)
- 6 and 38 and 42 (102)
- 6 and 38 and 40 (11)
- 4 and 37 and 40 (9)
- 4 and 37 and 42 (281)
- 6 and 37 and 42 (57)
- 6 and 37 and 40 (3)
- 4 and 38 and 40 (178)
- 46 or 47 or 48 or 49 or 50 or 51 or 52 or 53 (2644)
- limit 54 to english language (2581)
- limit 54 to abstracts (2185)
- 55 or 56 (2639)