To (1) conduct a systematic literature review of systems used to standardize language and definitions for outcome measures and other data elements, including systems for registries, clinical trials, electronic health records (EHRs), and quality reporting systems; and (2) use those findings to develop the Outcome Measures Framework (OMF) information model.
We searched the PubMed and Google Scholar databases for articles published in English since 2004. Articles were identified through a review of bibliographies of included articles. In addition, relevant materials were identified through discussions with a Technical Expert Panel (TEP) and key informants.
The literature review focused on four major areas: harmonizing data elements, key components of outcome measures, information models that describe outcome measures, and governance plans for existing models. Search terms were pilot-tested for sensitivity, specificity, and feasibility, and were refined through two iterations. Identified publications were reviewed for relevancy by a senior researcher using pre-determined key words. Data were extracted into a standard abstraction form and summarized qualitatively. Relevant data harmonization efforts identified through the literature or recommended by the TEP or key informants were examined, and data on these efforts were summarized in a table.
We identified 61 relevant publications. Many of the articles described efforts to harmonize existing data elements or create new standardized data elements within a defined condition area; these efforts may provide useful models for developing standardized outcome measures through a consensus-driven process. The literature search yielded relatively few examples of systems to catalog or organize outcome measures (or data elements generally). Little information was found on governance models or other issues applicable in the development and management of the OMF catalog. Some reports discussed the potential for using data captured in EHRs during routine clinical care to evaluate outcomes in research studies. These reports, along with input from the TEP and key informants, suggest that EHR-derived data are neither sufficiently harmonized nor detailed enough to support reliable outcome measurement currently.
The results of the literature search and the review of related initiatives demonstrate the number and varying scopes of existing projects. Based on these findings, this report presents different potential approaches for the OMF: a fully curated model, a community-sourced model, and a hybrid community-curated model. The fully curated model encompasses both a structured effort led by working groups of clinical experts to develop standardized outcome measures in specific condition areas as well as a repository of the standardized outcome measures created by these working groups. The community-sourced model enables the research community to organically identify the existing leading outcome measures through the provision of usage statistics and related qualitative comments from the Registry of Patient Registries (RoPR). A hybrid community-curated model enables the community to contribute and identify leading outcome measures, while also providing information on which measures are recommended or endorsed by other organizations.
Existing initiatives provide a wealth of opportunity for collaboration and shared learning. However, there is no existing effort identified through this review that attempts to achieve the same goals as the OMF, which is intended to support both development of new consensus-driven outcome measures for use in registries as well as provide a repository for existing outcome measures to facilitate reuse. Thus, the OMF would fill a unique need. This report presents three potential approaches for the OMF: a fully-curated model, a community- sourced model, and a hybrid community-curated model. Both the community-sourced and hybrid community-curated model have the advantage of providing robust content to users immediately upon launch and are therefore more likely to attract users. While all have strengths and limitations, the hybrid community-curated model represents the approach that is more likely to be scalable and sustainable, while still encouraging innovation within the research community.
Regardless of the approach used for the OMF, the system must prioritize ease of use and search capabilities to appeal to its voluntary user base, and should incorporate pilot testing to ensure that the system meets user needs. The OMF should also promote the use of data standards by establishing a data architecture and encouraging common data definitions. Additionally, it should provide links to other sources of standards or common data elements as part of a broader effort to generate common understanding of information used within the broad registry community. Lastly, the OMF, which must demonstrate value in order for the system to become widely used, would benefit from the development of case studies to support its value proposition.