Significant variation exists in both the types and definitions of outcome measures used in patient registries, even within the same clinical area. This variation reduces the utility of registries, making it difficult to compare, link, and aggregate data across the spectrum of clinical care and reporting. To address these limitations, the Agency for Healthcare Research and Quality (AHRQ) developed the Outcome Measures Framework (OMF), a conceptual model for classifying outcomes that are relevant to patients and providers across most conditions; it is intended to serve as a content model for developing harmonized outcome measures for specific clinical areas.
AHRQ is assessing the feasibility of using the OMF to develop standardized libraries of outcome measures in five clinical areas, including (1) Atrial fibrillation, (2) Asthma, (3) Depression, (4) Lung cancer, and (5) Lumbar spondylolisthesis.1 These clinical areas represent diverse populations and care settings, different treatment modalities, and varying levels of harmonization. For each clinical area, the relevant registries and observational studies are identified, and registry sponsors, informaticists, and clinical subject matter experts are invited to participate in a registry group that focuses on harmonizing outcome measures through a series of in-person and web-based meetings. A stakeholder group, including payers, patient representatives, Federal partners and health system leaders, is also assembled to discuss challenges and provide feedback on the harmonization effort.
A key goal of this effort is to standardize the definitions of the components that make up the outcome measures, so users can understand the level of comparability between measures across different systems and studies. As a final step in the harmonization process, clinical informaticists map the narrative definitions (generated by the workgroups) to standardized terminologies to produce a library of common data definitions.
This document describes the technical approach used to prepare the Library of Common Data Definitions for the minimum set of outcome measures produced by the Atrial Fibrillation Workgroup. For reference, the narrative definitions are included in Appendix A.
Approach to Representing an Outcome Definition
For each measure, the accompanying workbook contains the narrative definition and recommended reporting period (timeframe), the initial population for measurement (e.g., all atrial fibrillation patients, atrial fibrillation patients undergoing procedures), the outcome focused population (patients who experienced the outcome of interest), and the data criteria and value sets.
Electronic Health Record (EHR) data often will not contain all the requisite components of an outcome definition that would allow for the computational confirmation of that outcome. The approach used for this project is to gather the clinician’s assertion of an outcome condition and as much supporting evidence as possible, so that even where the expression logic cannot computationally confirm an outcome, some structured evidence might still be available.
Relationships between events raise a challenge because relationships are often not directly asserted in an EHR. Thus, where possible, relationships have been inferred based on time stamps and intervals. Where this is not possible (e.g., cause of death), the logic requires an asserted relationship.
For each outcome, the following have been defined:
- An object representing the outcome condition itself: In many cases, the only structured data will be an assertion of an outcome, with all the supporting evidence being present in the narrative.
- Fast Healthcare Interoperability Resources (FHIR) for evidence for the outcome: These include labs, diagnostic imaging, etc.
- FHIR for additional relevant events: These might include procedures, encounters, etc.
- Temporal aspects for all events: These allow for inferred relationships.
Approach to Identifying Overlaps
A key goal of this project is to leverage existing resources and build connections across initiatives, where possible. To support that goal, the following sources were searched for overlap:
- https://ecqi.healthit.gov/: Primarily looking for overlapping criteria
- https://vsac.nlm.nih.gov/: Primarily looking for overlapping value sets
- C-CDA: Primarily looking for overlapping data representations
- https://www.nlm.nih.gov/cde/: Primarily looking for overlapping data element definitions
- https://ecqi.healthit.gov/; https://vsac.nlm.nih.gov/:
- We were unable to identify any quality measures that laid out specific criteria for any of the AFib outcomes. Quality measures generally look to the EHR problem list for an assertion that an outcome exists and generally do not attempt to define the detailed criteria for an outcome. For example, considering TIA, quality measures will look for the presence of TIA on the problem list and do not require detailed supporting observations. Quality measures do create value sets (e.g., for TIA), and we did identify several overlapping value sets. All such value sets are housed at VSAC.
- eCQMs are based on the National Quality Forum’s Quality Data Model, as expressed as HL7 QRDA templates, whereas this project is based on FHIR version 1.8.0 objects.2 The HL7 Clinical Quality Improvement committee is actively harmonizing QDM and FHIR resources, and a FHIR-based quality reporting format is expected to be balloted soon.
- VSAC does not currently provide intensionally-defined value sets. Therefore, comparisons are done based on enumerated lists.
- There are no AFib or cardiac-specific templates or value sets in C-CDA.
- We were unable to identify any data elements that laid out specific criteria for any of the AFib outcomes. CDEs generally look for presence/absence of a condition and may associate a condition with a code system or value set. Therefore, there was minimal overlap between any AFib outcomes and existing CDEs.
Challenges and Lessons Learned
Some challenges were encountered in translating the text definitions produced by the workgroup into standardized definitions and value sets. Of note, two outcome measure definitions (procedure-related death and major complications) included the concept of complications related to a procedure or treatment. Within the EHR setting, it is often not feasible to attribute causality. Some events may be recorded as procedure complications. For other events, it is feasible to identify an event that occurs in a specified time window after a procedure or treatment. However, these events are not definitively linked to the procedure as a complication. Causality therefore is assumed based on the nature of the event and the timing.
Related to the major complications outcome measure, the ability to distinguish whether a complication was ‘major’ is difficult and may not be possible in some cases. The criteria that distinguish a major complication, as specified in the definition, are 'permanent injury or death, requires intervention for treatment, or prolongs or requires hospitalization for more than 48 hours or results in re-hospitalization within 30 days.' Within the EHR setting, ‘permanent’ injury is not indicated, although it may be assumed for events with no end date. It is also not feasible in most cases to determine if a specific complication prolonged a hospital stay.
Lastly, the definitions produced by the workgroup lacked some specificity that is necessary for producing the standardized definitions. Some definitions use lists that begin with 'such as' or end with 'etc.' (e.g., symptomatic bleeding in a critical area or organ, such as intracranial, intraspinal, intraocular, retroperitoneal, intraarticular, pericardial, or intramuscular with compartment syndrome). It is important to specify the full list as part of the definition to support appropriate translation into a standardized definition. Timeframes also must be clearly specified. For example, the major bleeding definition references a hemoglobin drop of 2 g/dL, but does not specify the timeframe in which the two measurements must occur in order for the drop to count as major bleeding.
The project team will apply these lessons learned in subsequent workgroups.
Appendix A. Harmonized Definitions for Atrial Fibrillation Outcome Measures
|OMF Category||Outcome Measure||Definition||Reference|
|Survival||All-cause mortality||All-cause mortality||Workgroup recommendation|
|Survival||Cardiovascular death||Cardiovascular death indicates cause of death was sudden cardiac death, MI, unstable angina, or other coronary artery disease; vascular death (e.g., stroke, arterial embolism, pulmonary embolism, ruptured aortic aneurysm, or dissection); congestive heart failure; or cardiac arrhythmia.||2004 ACC/AHA Key Data Elements (1)|
|Survival||Procedure-related death||All-cause mortality within 30 days of the procedure or during the index procedure hospitalization (if the postoperative length of stay is > than 30 days). Procedure-related deaths include those related to a complication of the procedure or treatment for a complication of the procedure.||VARC Statement (2)|
|Clinical Response||AF/AFL/AT Recurrence||Recurrent AF/AFL/AT is defined as AF/AFL/AT of at least 30 seconds' duration that is documented by an ECG or device recording system and occurs following catheter ablation or drug therapy. In the setting of catheter ablation, recurrent AF/AFL/AT may occur within or following the post ablation 3-month blanking period. Recurrent AF/AFL/AT that occurs within the post ablation blanking period is not considered a failure of AF ablation.||2017 HRS Consensus Statement (3)|
|Clinical Response||AF progression (paroxysmal to persistent AF)||
AF should be classified as paroxysmal or persistent in accordance with the 2014 AHA/ACC/HRS joint committee guidelines on the management of patients with AF:
|Modified from 2014 AHA/ACC/HRS joint committee guidelines on the management of patients with AF (4)
Definition of progression is adapted
|Clinical Response||Thromboembolic events (with respect to persistent AF management)||
|Events of Interest||Stroke||An acute episode of focal or global neurological dysfunction caused by brain, spinal cord, or retinal vascular injury as a result of hemorrhage or infarction. Symptoms or signs must persist ≥24 hours, or if documented by CT, MRI or autopsy, the duration of symptoms/signs may be less than 24 hours. Stroke may be classified as ischemic (including hemorrhagic transformation of ischemic stroke), hemorrhagic, or undetermined. Stroke disability measurement should be performed using the modified Rankin Scale (mRS) at discharge and 6 months post-discharge. The mRS scores should be recorded.||2014 ACC/AHA Key Data Elements (6)|
|Events of Interest||TIA||Transient episode of focal neurological dysfunction caused by brain, spinal cord, or retinal ischemia without acute infarction and with signs and symptoms lasting less than 24 hours.||2014 ACC/AHA Key Data Elements (6)|
|Events of Interest||Systemic embolism||Acute arterial insufficiency or occlusion of the extremities or any non-CNS organ associated with clinical, imaging, surgical/autopsy evidence of arterial occlusion in the absence of other likely mechanism (e.g., trauma, atherosclerosis, or instrumentation).||Modified from the Munich Consensus Statement (7)|
|Events of Interest||Major bleeding at 12 month interval of interest (no peri-procedural association)||Fatal bleeding AND/OR symptomatic bleeding in a critical area or organ, such as intracranial, intraspinal, intraocular, retroperitoneal, intraarticular, pericardial, or intramuscular with compartment syndrome AND/OR bleeding causing a fall interest (no peri-in hemoglobin level of 2 g/dL (1.24 mmol/L) or more, or leading to transfusion of procedural two or more units of blood.||ISTH definition (8)|
|Events of Interest||Periprocedural bleeding (any bleeding during 12-month interval which occurs within 30d of procedure)||
||ISTH definition (8)|
|Events of Interest||Myocardial infarction||The term acute myocardial infarction (MI) should be used when there is evidence of myocardial necrosis in a clinical setting consistent with acute myocardial ischemia. Under these conditions any one of the following criteria meets the diagnosis for MI:
||Third universal definition of myocardial infarction (9)|
|Events of Interest||Myocardial infarction (as a complication of ablation procedure)||MI, in the context of catheter or surgical ablation, is defined as the presence of any one of the following criteria: (1) detection of ECG changes indicative of new ischemia (new ST-T changes or new LBBB) that persist for more than 1 hour; (2) development of new pathological Q waves on an ECG; (3) imaging evidence of new loss of viable myocardium or new regional wall motion abnormality.||2017 HRS Consensus statement (3)|
|Events of Interest||Heart Failure||Heart failure is defined as physician documentation or report of any of the following clinical symptoms of heart failure described as unusual dyspnea on light exertion, recurrent dyspnea occurring in the supine position, fluid retention; or the description of rales, jugular venous distention, or pulmonary edema on physical examination. A low ejection fraction without clinical presentation does not qualify as heart failure. Studies that wish to classify heart failure should use the New York Heart Association (NYHA) Functional Classification.||2013 ACCF/AHA key data elements and definitions (9)|
|Events of Interest||Other major complications of the procedure||
A major complication is a complication results in permanent injury or death, requires intervention for treatment, or prolongs or requires hospitalization for more than 48 hours or results in re-hospitalization within 30 days. Because early recurrences of AF/AFL/AT are to be expected following AF ablation, recurrent AF/AFL/AT within 3 months that requires or prolongs a patient’s hospitalization should not be considered to be a major complication of AF ablation.
Because early recurrences of AF/AFL/AT following antiarrhythmic drug therapy are a failure of therapy and not a complication, they should be excluded from this measure as it relates to prolonged hospitalization or readmission within 30 days.
|2017 HRS Consensus Statement (3)|
|Patient Reported||AF-related quality of life||AF-related quality of life should be measured using an AF-specific quality of life instrument that is validated and commonly used, such as AFEQT.||Workgroup recommendation|
|Patient Reported||Generic quality of life||General quality of life should be measured using a quality of life instrument that is validated and commonly used.||Workgroup recommendation|
|Resource utilization||All-cause hospitalization||All-cause hospitalization||Workgroup recommendation|
|Resource Utilization||Cause-specific hospitalization||Hospitalization for which the primary admitting diagnosis was for heart failure, stroke, bleeding, atrial fibrillation, repeat AF-ablations, periprocedural complication, other cardiovascular causes.||Workgroup recommendation|
|Resource Utilization||Other resource utilization related to treatment or management of AF or associated complications||Other resource utilization related to treatment or management of AF or associated complications, e.g., because-hospitalization doesn't include office visits, emergency room visits, drug costs, etc.||Workgroup recommendation|
- This work was supported by the Office of the Secretary Patient-Centered Outcomes Research Trust Fund under Interagency Agreement #16-566R-16.
- McNamara RL, Brass LM, Drozda JPJ, Go AS, Halperin JL, Kerr CR, et al. ACC/AHA key data elements and definitions for measuring the clinical management and outcomes of patients with atrial fibrillation: A report of the American College of Cardiology/American Heart Association Task Force on Clinical Data Standards (Writing Commitee to Develop Data Standards on Atrial Fibrillation). J Am Coll Cardiol. 2004 Jul 21;44(2):475–95.
- Kappetein AP, Head SJ, Genereux P, Piazza N, van Mieghem NM, Blackstone EH, et al. Updated standardized endpoint definitions for transcatheter aortic valve implantation: the Valve Academic Research Consortium-2 consensus document (VARC-2). Eur J Cardio-Thorac Surg Off J Eur Assoc Cardio-Thorac Surg. 2012 Nov;42(5):S45–60.
- Calkins H, Hindricks G, Cappato R, Kim Y-H, Saad EB, Aguinaga L, et al. 2017 HRS/EHRA/ECAS/APHRS/SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation. Heart Rhythm. 14(10):e275–444.
- January CT, Wann LS, Alpert JS, Calkins H, Cigarroa JE, Cleveland JCJ, et al. 2014 AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the Heart Rhythm Society. J Am Coll Cardiol. 2014 Dec 2;64(21):e1–76.
- Padfield GJ, Steinberg C, Swampillai J, Qian H, Connolly SJ, Dorian P, et al. Progression of paroxysmal to persistent atrial fibrillation: 10-year follow-up in the Canadian Registry of Atrial Fibrillation. Heart Rhythm. 2017 Jun;14(6):801–7.
- Hicks KA, Tcheng JE, Bozkurt B, Chaitman BR, Cutlip DE, Farb A, et al. 2014 ACC/AHA Key Data Elements and Definitions for Cardiovascular Endpoint Events in Clinical Trials. Circulation. 2015 Jul 27;132(4):302.
- Tzikas A, Holmes DRJ, Gafoor S, Ruiz CE, Blomstrom-Lundqvist C, Diener H-C, et al. Percutaneous left atrial appendage occlusion: the Munich consensus document on definitions, endpoints and data collection requirements for clinical studies. EuroIntervention J Eur Collab Work Group Interv Cardiol Eur Soc Cardiol. 2016 May 17;12(1):103–11.
- Schulman S, Kearon C. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost JTH. 2005 Apr;3(4):692–4.
- Cannon CP, Brindis RG, Chaitman BR, Cohen DJ, Cross JTJ, Drozda JPJ, et al. 2013 ACCF/AHA key data elements and definitions for measuring the clinical management and outcomes of patients with acute coronary syndromes and coronary artery disease: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Clinical Data Standards (Writing Committee to Develop Acute Coronary Syndromes and Coronary Artery Disease Clinical Data Standards). Circulation. 2013 Mar 5;127(9):1052–89.