This methods guide is available in PDF (1.2 MB) only.
This project was performed under a contract from the Agency for Healthcare Research and Quality (AHRQ) in collaboration with the Centers for Medicare & Medicaid Services (CMS) through the Developing Evidence to Inform Decisions about Effectiveness (DEcIDE) Network of AHRQ’s Effective Health Care (EHC) Program. The purpose of the project was to produce a handbook that would serve as a reference for establishing, maintaining, and evaluating the success of registries created to collect data about patient outcomes.
Following award of the project on September 29, 2005, we created a draft outline for the document that was posted for public comment on AHRQ’s Effective Health Care Program website from January through March 2006. During that same period, we worked with AHRQ to create a process for selecting contributors and reviewers. We broadly solicited recommendations from a range of stakeholders, including government agencies, industry groups, medical professional societies, and other experts in the field; conducted a review of the pertinent literature; and contacted the initial list of contributors to confirm their interest and area of expertise and to seek further recommendations. Through that process and in collaboration with AHRQ and CMS, we arrived at a set of contributors and reviewers based on subject/content expertise, practical experience, and interest and availability, with balanced representation from key stakeholder groups for nearly all chapters. In addition, a request for submission of real-world case examples that could be used in the handbook to illustrate issues and challenges in implementing registries was posted on the Effective Health Care Program website. The primary selection criteria for these examples were their utility in illustrating a practical challenge and its resolution.
An initial meeting of contributors was convened in February 2006. A second meeting including contributors and chapter reviewers was held in June 2006, following creation of an initial draft document and focused review by the reviewers. The collaborative efforts of contributors, reviewers, and editors resulted in a draft document that was posted for public comment on the Effective Health Care Program website in October and November 2006. In all, 39 contributors and 35 individual reviewers participated. These contributors and reviewers participated as individuals and not necessarily as representatives of their organizations. We are grateful to all those who contributed to the document and who reviewed it and shared their comments.
To begin the discussion of registries, we would like to clarify some distinctions between registries and clinical trials. While this is further discussed in Chapter 1, from a high-level perspective, we offer the following distinctions: The clinical trial is an experiment in which an active intervention intended to change a human subject’s outcome is implemented, generally through a randomization procedure that takes decision making away from the practitioner. The research protocol describes inclusion and exclusion criteria that are used to select the patients who will participate as human subjects, focusing the experiment on a homogeneous group. Human subjects and clinical researchers agree to adhere to a strict schedule of visits and to conduct protocol-specific tests and measurements.
In contrast, registries use an observational study design that does not specify treatments (although a specific treatment may be an inclusion criterion) and observe without requiring any therapies intended to change patient outcomes. There are generally few inclusion and exclusion criteria in an effort to study a broad range of patients to make the results more generalizable. Patients are typically observed as they present for care and the data collected generally reflect whatever tests and measurements a provider customarily uses.
Patient registries represent a useful tool for a number of purposes. Their ideal use and their role in evidence development, design, operations, and evaluation resemble but differ from clinical trials in a number of substantive ways, and therefore they should not be evaluated with the same constructs. This handbook presents what the contributors and reviewers consider to be good registry practices. Many registries today may not meet even the basic practices described. On the whole, registry science is in an active state of development. This document is an important step in developing this field.
This book is divided into three sections: Creating, Operating, and Evaluating Registries. The first two sections provide basic information on key areas of registry development and operations, highlighting the spectrum of practices in each of these areas and their potential strengths and weaknesses. Section I, “Creating Registries,” includes six chapters. “Patient Registries” defines and characterizes types of registries, their purposes, and uses, and describes their place within the scope of this document. “Planning a Registry” focuses on the recommended steps in planning a registry, from determining if a registry is the right option to describing goals and objectives. “Registry Design” examines the specifics of designing a registry once the goals and objectives are known. “Data Elements for Registries” provides an approach to selecting data elements that is both scientific and practical. “Data Sources for Registries” addresses how existing data sources (administrative, pharmacy, other registries, etc.) may be used to enhance the value of patient registries. “Principles of Registry Ethics, Data Ownership, and Privacy” reviews several key legal and ethical issues that should be considered in creating or operating a registry.
Section II, “Operating Registries,” provides a practical guide to the day-to-day operational issues and decisions for producing and interpreting high-quality registries. “Patient and Provider Recruitment and Management” describes strategies for recruiting and retaining providers and patients. “Data Collection and Quality Assurance” reviews key areas of data collection, cleaning, storing, and quality assurance for registries. “Adverse Event Detection, Processing, and Reporting” examines relevant practical and regulatory issues. “Analysis and Interpretation of Registry Data To Evaluate Outcomes” addresses key considerations in analyzing and interpreting registry data.
Interspersed throughout the first two sections of the handbook are case examples. As discussed above, the choice of examples was limited to those submitted for consideration during the public submission period. The purpose of their inclusion is solely to illustrate specific point(s) in the text from real-world examples, regardless of whether the source of the example is within the scope of the handbook as described in Chapter 1. Inclusion of a case example in this handbook is not intended as an endorsement of the quality of the particular registry, nor do the case examples necessarily present registries that meet all the criteria described in Chapter 11 as basic elements of good practice. Rather, case examples are introduced to provide the reader with a richer description of the issue or question being addressed in the text. In some cases, we have no independent information on the registry other than what has been provided by the contributor.
Section III is “Evaluating Registries.” This final chapter summarizes key points from the earlier chapters in a manner that can be used to review the structure, data, or interpretations of patient registries. It describes good registry practice in terms of “basic elements” and “potential enhancements.” This information might be used by a person developing a registry, or by a reviewer or user of registry data or interpretations derived from registries.
Richard E. Gliklich
Nancy A. Dreyer