Skip to main content
Effective Health Care Program
Home » Products » Closing the Quality Gap Series: The Patient-Centered Medical Home » Closing the Quality Gap Series: The Patient-Centered Medical Home

Closing the Quality Gap Series: The Patient-Centered Medical Home

Research Protocol Archived
Download PDF493.6 KB

Background and Objectives for the Systematic Review

Almost half of patients in the United States have one or more chronic medical conditions.1 Outcomes for such patients appear to be enhanced through a sustained, ongoing partnership between the patient and his or her health care team.2 This is especially true for the more than 20 percent of Americans who have multiple chronic illnesses.1 Despite two decades of intense research and identification of successful strategies for enhancing outcomes for patients with chronic illness,2 numerous remaining opportunities to improve the delivery of chronic illness care have been noted.3-8 For example, McGlynn et al. found that 45 percent of primary care patients in the United States did not receive evidence-based preventive and chronic illness services.3 There appears to be evidence that redesigning the practice of primary care to provide proactive chronic illness management may lead to improved care process and outcomes.2,9

The patient-centered medical home (PCMH) is a model of primary care transformation that seeks to meet the variety of health care needs of patients as a way of improving patient experiences, outcomes, safety, and system efficiency.10-12 It is based on 40 years of previous efforts to redesign primary care to provide the highest quality of care possible.13 Comprehensive PCMH interventions hold promise as a pathway to improved primary health care quality, safety, efficiency, and effectiveness. As defined by physician and consumer groups, the core principles of the PCMH are: wide-ranging team-based care; patient-centered orientation toward the whole person; care that is coordinated across all elements of the health care system and the patient’s community; enhanced access to care that utilizes alternative methods of communication; and a systems-based approach to quality and safety.11

Multiple organizations have begun to implement PCMH-based programs. These include health maintenance organizations (HMOs), networks of Medicaid providers, community health centers, private integrated delivery systems, private practices, and the Veterans Affairs (VA) health care system.14-16 The goal is to improve the care of patients across the continuum of chronic and acute illness, while potentially improving both patient and provider experiences with the health care system. Further, it has been hypothesized that in combination with accountable care organizations (ACOs), PCMH may bring improvements in patient care while at the same time making health care organizations more efficient.14

Existing systematic reviews show that certain individual organizational components or elements encompassed in the definition of the PCMH are associated with improvements in selected care process and outcomes for individual conditions.9,17-20 To the best of our knowledge, however, there has not been a high-quality systematic review of whether implementing comprehensive PCMH interventions—with the combination of all components included in the definition of the model—leads to better care processes or outcomes. (Specific PCMH components are described in the section on PICOTS, B: Interventions, below.) Further, it is uncertain if organizing overall primary care in accordance with the PCMH model (i.e., combining the use of PCMH components for multiple conditions) improves overall care processes and clinical outcomes.

For this review, we will examine results of studies focusing on changing care for all or most patients serviced by a health care organization, not just a specific group of patients such as those with a given illness or set of illnesses. As part of the Closing the Quality Gap series of Evidence-based Practice Center (EPC) reviews, the purpose of the systematic review will be to identify completed and ongoing efforts to evaluate the comprehensive PCMH model, summarize current evidence for this model, and identify gaps in the evidence. Because the PCMH model is being implemented widely but the number of completed studies is expected to be small, the identification of ongoing studies is an important goal of this review. This “horizon scan” component of the review will help to identify forthcoming studies that may address gaps in the currently available evidence.

The PCMH is a cross-cutting topic, relevant to broad areas of health care and patient populations. We anticipate a number of important challenges:

  • We recognize that multiple definitions of the PCMH model have been proposed by various professional and patient organizations.21 We have identified components of comprehensive PCMH interventions that must be included for studies to be included in this review (see PICOTS, B). These components are based on the PCMH definition proposed by the Agency for Healthcare Research and Quality (AHRQ).10
  • Based on a preliminary review of the literature, we anticipate few randomized controlled trials (RCTs) and a diverse range of study designs. Because RCTs, quasi-experimental designs, and observational designs vary in their risk of bias, we will prioritize RCTs. However, we will include the other study designs when necessary, analyzing these studies separately.

The Key Questions

For clarification, Key Questions (KQs) 1-3 concern published studies, while KQ 4 is a “horizon scan” question that relates to unpublished studies now in progress.

KQ 1: In published, primary care–based evaluations of comprehensive PCMH interventions, what are the effects of the PCMH on patient and staff experiences, process of care, clinical outcomes, and economic outcomes?

  1. Are specific PCMH components associated with greater effects on patient and staff experiences, process of care, clinical outcomes, and economic outcomes?
  2. Is implementation of comprehensive PCMH associated with unintended consequences (e.g., decrease in levels of indicated care for non-priority conditions) or other harms?

KQ 2: In published, primary care–based evaluations of comprehensive PCMH interventions, what individual PCMH components have been implemented?

KQ 3: In published, primary care–based evaluations of comprehensive PCMH interventions, what financial models and implementation strategies have been used to support uptake?

KQ 4: What primary care–based studies evaluating the effects of comprehensive PCMH interventions on patient and staff experiences, process of care, clinical outcomes, or economic outcomes are currently under way? In these ongoing studies, what are the study designs, PCMH components, comparators, settings, financial models, and outcomes to be evaluated?

PICOTS (Population, Intervention, Comparator, Outcome, Timing, Setting) Framework for the Key Questions

A. Population(s):

  1. Adult, primary care patients, selected to represent the practice rather than on the basis of a particular chronic illness
  2. Children with special health care needs according to the Health Resources and Services Administration (HRSA) definition. The broad definition of children with special health care needs includes those who have or are at increased risk for chronic physical, developmental, behavioral, or emotional conditions that require health and related services of a type or amount beyond those required by children generally.

B. Interventions:

The PCMH is a broad-based strategy aimed at improving chronic illness care or provision of preventive services. Using the AHRQ definition of the PCMH (items marked with an asterisk [*] below),10 we operationalize the concept of a PCMH intervention as a comprehensive intervention that includes items 1, 3, and 4, along with at least two elements of item 2. The comprehensive PCMH intervention is the combination of the components described below, not the individual components themselves. The components are:

  1. Team*-based care. The team may be virtual.
  2. The intervention includes ≥ 2 of the following 4 elements:
    1. Enhanced access* to care (e.g., advanced electronic communications such as Internet or telephone visits, open access scheduling, group visits, 24/7 coverage).
    2. Coordinated* care (care coordinated across settings such as inpatient and outpatient, or across specialty and non-specialty care [such as mental health], or subspecialty medicine and primary care; care management; or referral tracking).
    3. Comprehensiveness,* (care that is accountable for addressing a large majority of personal health needs; for example, preventive care, acute care, chronic disease care, and mental health).
    4. A systems-based approach to improving quality and safety. * For example, an approach that includes a care planning process, evidence-based medicine/clinical guidelines, point-of-care resources, electronic prescribing, test-tracking, performance measurement, self-management support, accountability, and shared decisionmaking.
  3. A sustained partnership* and personal relationship over time oriented towards the whole person.* For example, a designated primary point of contact who coordinates care, a personal physician, and shared decisionmaking.
  4. The intervention involves structural changes to the traditional practice, reorganizing care delivery. Examples include new personnel, new role definitions, functional linkages with community organizations and/or other health care entities such as hospitals, specialists or other service providers, and disease registries.

C. Comparators:

  1. Usual care.
  2. Programs aimed at improving the quality of care, process outcomes, or clinical outcomes that do not meet the operational definition of a comprehensive PCMH intervention given above. These comparator programs may include some components of the PCMH model, but not enough to qualify as a comprehensive PCMH intervention.

D. Outcomes:

KQ 1:

  1. Patient experiences, assessed using a validated measure.
  2. Staff experiences, including retention rates, job satisfaction, and “burnout.” The last two should be assessed using validated measures.
  3. Process of care, including:
    1. Access to care (e.g., timeliness of response to patient inquiry, appointment timeliness, and availability of telephone or email appointments).
    2. Guideline-concordant care processes, or scores on broadly accepted performance measures (e.g., rate of depression screening) such as those endorsed by the National Quality Forum or National Committee on Quality Assurance).
  4. Clinical outcomes:
    1. Intermediate patient outcomes (e.g., change in HbA1c for diabetic patients).
    2. Clinical outcomes (e.g., mortality, symptom scale).
  5. Economic outcomes:
    1. Overall health care utilization.
    2. Specific categories of utilization (e.g., decreased emergency department utilization for asthma).
    3. Incremental cost-effectiveness.

KQ 2: PCMH components as listed in intervention section of the PICOTS framework. We will describe the use of specific PCMH components and related activities reported in the reviewed studies, for example:

  1. Team-based care: defined as team-based structure with specified roles. We will provide a description, including disciplines represented.
  2. Enhanced access (components): programs that provide additional avenues for patient access, such as telephone appointments, open access scheduling, Internet visits, group visits, and expanded hours.
  3. Coordinated care across settings: We will describe programs that coordinate care across settings such as inpatient and outpatient, home health care, community services, or specialty and non-specialty care.
  4. Comprehensive care (yes/no): We will describe whether or not a primary care practice is accountable for addressing a large majority of personal health needs, including prevention and wellness, acute care, and chronic care.
  5. A systems-based approach to quality, safety, and delivery of evidence-based care. Components of such programs could include clinical decision support tools (clinical guidelines, point-of-care resources), electronic prescribing, test-tracking, performance measurement, self-management support or shared decisionmaking, or public reporting of quality and/or safety data.
  6. Sustained partnership (yes/no): Are efforts made to ensure sustained partnerships between patients and identifiable providers/teams?
  7. Reorganizing care delivery (components): Have there been changes to the structure of the organization through which care is provided? (e.g., new personnel, new role definitions, functional linkages with community organizations and/or other health care entities, and/or with disease registries?)

KQ 3:

  1. Financial models (e.g., bundled payments, fee-for-service, performance-based incentives).
  2. System-change strategies (e.g., plan-do-study-act (PDSA) cycles,22 academic detailing23), along with any theoretical basis provided for these strategies.
  3. Organizational learning strategies (e.g., quality improvement collaboratives22), and any theoretical basis provided for these strategies.

KQ 4. Because KQ 4 is a horizon scan of ongoing studies, we anticipate that many study details will not be available, but we will examine data sources for the following information:

  1. Study designs, including patient or cluster RCTs, non-randomized clustered controlled trials, and controlled before-and-after studies.
  2. PCMH components (as defined in intervention PICOTS) and comparators.
  3. Settings (e.g., practice size, geographic location).
  4. Financial models (e.g., bundled payments, fee-for-service, performance-based incentives).
  5. Types of outcomes assessed:
    1. Patient experiences, assessed using a validated measure.
    2. Staff experiences, including retention rates, job satisfaction, and “burnout.” The last two will be assessed using validated measures.
    3. Process of care reported as study outcomes:
      1. Access to care; for example, timeliness of response to patient inquiry, appointment timeliness.
      2. The degree to which care process are guideline-concordant; and reports of scores on broadly accepted performances measures such as those developed by the National Quality Forum or National Committee on Quality Assurance).
    4. Clinical outcomes:
      1. Intermediate patient outcomes (e.g., change in HbA1c for diabetic patients).
      2. Clinical outcomes (e.g., mortality, symptom scale).
    5. Economic outcomes:
      1. Overall health care utilization.
      2. Specific categories of utilization (e.g., decreased emergency department utilization for asthma).
      3. Incremental cost-effectiveness.

E. Timing:

  1. Studies must have at least 6 months’ longitudinal followup.

F. Settings:

  1. Primary care. That is, we will not consider studies in specialty care settings such as infectious disease for patients with HIV/AIDS. Primary care includes:
    1. General internal medicine.
    2. Family medicine.
    3. Primary care pediatrics.
    4. Primary care clinics directed by mid-level providers.
    5. Terms commonly used for primary care outside the United States (e.g., general practice/practitioner).
  2. KQ 4 will be further restricted specifically to studies under way in the United States. We will impose this restriction on the horizon scan to identify ongoing studies that are most relevant to the U.S. health care system. Also, we will be able to identify more reliably studies conducted in the United States.

G. Type of studies:

  1. KQ 1: We will focus on studies of comprehensive PCMH interventions that include a comparison group. Specific study designs are based on guidance from the Cochrane Effective Practice and Organisation of Care Review Group (EPOC) and include:24
    1. Patient or cluster randomized controlled trials (RCTs).
    2. Non-randomized clustered controlled trials: experimental studies in which practices or clinicians are allocated to different interventions using methods that are not random.
    3. Controlled before-and-after studies: studies in which observations are made before and after the implementation of an intervention, both in a group that receives the intervention and in a comparison group that does not. These studies include observational studies of “natural experiments.”
  2. KQ 2-3: To answer these questions, we will include all of the designs listed above and uncontrolled studies that include a pre- and post-intervention assessment. We include uncontrolled studies for these questions because the aims of the questions are descriptive. By including uncontrolled studies, we will give a more comprehensive description of the PCMH components, financial models, and implementation strategies examined to date.
  3. KQ 4: Same as KQ1. Because this question represents a horizon scan of ongoing and/or yet-to-be-published literature, we will be seeking ongoing longitudinal studies, including pilot and demonstration projects, with comparison groups. Given the large number of organizations conducting ongoing evaluation of PCMH, we will prioritize studies from major Federal funders (e.g. CMS, AHRQ, and VA) and large non-Federally funded studies that are most likely to yield high quality data and address gaps in existing evidence.

Analytic Framework

The figure depicts the key questions within the context of the analytic framework based on the elements described in the previous section. The figure illustrates how comprehensive PCMH interventions (the combination of PCMH elements taken as a group, not just the individual components) and their comparators (usual care or programs aimed at improving the quality of care, process outcomes, or clinical outcomes that do not meet the operational definition of a comprehensive PCMH intervention) have been shown in the published literature to impact outcomes of interest (KQ 1), including patient and staff experiences, process of care, clinical outcomes, and economic outcomes. In addition, the association of PCMH with unintended consequences or other harms is demonstrated. The individual components of PCMH and their incorporation and/or implementation in PCMH evaluations are demonstrated (KQ 2), as well as the financial models and system change or organizational learning strategies used to support uptake (KQ 3). Finally, the figure illustrates the way in which the above-mentioned outcomes and moderators were identified in ongoing studies (KQ 4).

Methods

A. Overall Approach

Our approach will be guided by the AHRQ’s Methods Guide for Effectiveness and Comparative Effectiveness Reviews (hereafter referred to as the General Methods Guide).25 and methodology from the original “Closing the Quality Gap Series,” drawing particularly on Volume 1, Series Overview and Methodology,26 and Volume 7, Care Coordination.27 Consistent with these earlier works, we will adopt the EPOC framework for relevant study designs, as described above: patient or cluster RCTs (KQs 1-4), non-randomized cluster controlled trials (KQs 1-4), controlled before-and-after studies (KQs 1-4), and uncontrolled studies that include a pre- and post-intervention assessment (KQs 2-3). These designs can yield valid evidence about quality improvement interventions. Other key methodological decisions from this series include a focus on outpatient care and the inclusion of studies where the intervention seeks to improve outcomes for a broad and relatively unselected group of patients. The key questions were refined via discussions with the EPC coordinating the “Closing the Quality Gap” series and with AHRQ. A Technical Expert Panel (TEP) will be assembled to provide input, during the protocol review process, with experts knowledgeable in the PCMH as primary care.

B. Criteria for Inclusion/Exclusion of Published Studies (KQs 1-3) and Gray Literature (KQ 4)

An article will be included if all of the following criteria apply:

  • For KQs 1-3, a published study must examine a comprehensive PCMH intervention as defined in the section on PICOTS, B: Interventions.
  • For the horizon scan (KQ 4), the intervention should meet the definition of a comprehensive PCMH intervention as specified in PICOTS B. The degree to which this can be verified for yet-to-be-published studies will vary and be dependent on the specific source form which the horizon scan information comes.
  • For KQs 1-3, published studies of the same type with at least 6 months’ longitudinal followup. For the horizon scan (KQ 4), an ongoing longitudinal study including pilot and demonstration projects with a comparison group.
  • A study must be one of the following designs:
    • KQ1, KQ4: Patient or cluster RCT; non-randomized clustered controlled trial; controlled before-and-after study.
    • KQ2, KQ3: Patient or cluster RCT; non-randomized clustered controlled trial; controlled before-and-after study; uncontrolled pre- and post-intervention study.
  • A study must be of primary care settings (e.g., family medicine, general internal medicine, primary care pediatrics, general medical clinics such as Federally Qualified Health Centers, general medical clinics primarily staffed by mid-level providers, general practice/practitioner).
  • The comparator must be usual care or programs that are aimed at improving the quality of care process or clinical outcomes, but that do not meet the operational definition of a comprehensive PCMH intervention provided in PICOTS B.
  • For KQ 1-3, a study must report a relevant outcome. (See the section on PICOTS, D.)
  • We include English-language publications only. We exclude non-English-language publications for two reasons: (a) we are most interested in health care systems that are similar to U.S. health care, and reports from these countries are likely to be published in English; and (b) the expense of identifying and translating non-English publications is prohibitive.
  • The study must have been conducted in a high-income economy as defined by the World Bank.28 We restrict the study to high-income economies—countries that have greater cultural and health care system similarities to the United States—to improve applicability of the study results to the United States. KQ 4 will be further restricted specifically to studies under way in the United States.

An article will be excluded if any of the following criteria apply:

  • Studies where PCMH transformation was focused on a small proportion of patients being cared for in the practice; for example, studies restricted to patients with diabetes.

C. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions

To identify the relevant published literature, we will search MEDLINE, the Cumulative Index to Nursing & Allied Health Literature (CINAHL), and the Cochrane Database of Systematic Reviews (CDSR). Where possible, we will use existing validated search filters (such as the Clinical Queries Filters in PubMed), and will draw on other groups’ experience in searching for quality improvement studies (e.g., EPOC). We will supplement the electronic searches with a manual search of citations from a set of key primary and review articles.29 To identify ongoing or recently completed studies, we will search the following sources: databases of PCMH demonstration programs (e.g., the Patient-Centered Primary Care Collaborative [www.pcpcc.net]; primary care professional societies sponsoring PCMH demonstration projects (e.g., American College of Physicians, at www.acponline.org/running_practice/pcmh/); clinical trials databases (e.g., Clinicaltrials.gov); Web sites of non-Federal PCMH funders (e.g., Commonwealth Fund); databases of state-sponsored PCMH studies (e.g., National Academy for State Health Policy), and databases of Federally funded studies (e.g., AHRQ, HSRProj, NIH Reporter (NIH Research Portfolio Online), HRSA, and the Centers for Medicare & Medicaid Services [CMS]). In addition, we will contact experts in the PCMH field (including the TEP), and will search key articles manually, as described above. All searches will be completed in collaboration with an experienced search librarian.

For Web sites and databases of PCMH studies, two reviewers will screen the entries for studies of potential relevance to the research questions. For MEDLINE, CINAHL, and CDSR, two reviewers, using prespecified inclusion/exclusion criteria, will review titles/abstracts for potential relevance to research questions. Articles included by either reviewer will undergo full-text screening. At the full-text screening stage, two independent reviewers must agree on a final inclusion/exclusion decision. Articles meeting eligibility criteria will be included for data abstraction. All results will be tracked in a Distiller SR database. While the draft report is under peer review, we will update our search of the published literature.

D. Data Abstraction and Data Management

Data abstraction forms will be developed separately for KQ 1-3 and KQ 4 (horizon scan). For KQ 1, we will focus on data elements to evaluate the association between independent variables of interest and study outcomes, study quality, and factors affecting applicability. Outcomes include patient and staff experiences, process of care, clinical outcomes, and economic outcomes. We will extract data specified above for KQ 2-3 with an emphasis on capturing detail on the comprehensive PCMH intervention. For KQ 4 (horizon scan), we expect that the available study details will be limited, and we will restrict our data abstraction to major variables that have a higher likelihood of being reported in study registries: basic study design, geographic location, study setting including health care system, number of practices/physicians, population/patient characteristics, stated objective, payment reform/financial model, major components of the intervention/PCMH model, implementation approach and types of outcomes being assessed, study dates, stage of study, and source of funding. A trained researcher will complete data abstraction, and the abstraction will be over-read by a second researcher. Disagreements will be resolved by discussion or a third researcher when agreement cannot be reached.

E. Assessment of Methodological Quality of Individual Studies

The included studies that will be utilized to address KQ 1-3 will be assessed on the basis of the quality of their reporting of relevant data. We will evaluate the quality of individual studies using the approach described in AHRQ’s General Methods Guide.25 To assess quality, we will employ the strategy to (1) classify the study design, (2) apply predefined criteria for quality and critical appraisal, and (3) arrive at a summary judgment of the study’s quality. To evaluate methodological quality, we will apply criteria for each study type derived from core elements described in the General Methods Guide. To indicate the summary judgment of the quality of the individual studies, we will use the summary ratings of good, fair, and poor, based on the studies’ adherence to well-accepted standard methodologies and the adequacy of the reporting.

For RCTs, we will use the key criteria described in AHRQ’s General Methods Guide,25 adapted to this specific topic. These criteria include adequacy of randomization and allocation concealment, the comparability of groups at baseline, blinding, the completeness of followup and differential loss to followup, whether incomplete data were addressed appropriately, the validity of outcome measures, and conflict of interest. These general criteria will be customized to each major outcome. After considering each individual quality element, we will assign the article a global quality rating of good, fair, or poor, using definitions from the General Methods Guide.

We anticipate that this review will identify and include non-randomized clinical trials (see section IIG for eligible study designs). Because of the complexity of PCMH-based interventions, studies may have included an observational control group that was not randomized. Per the AHRQ General Methods Guide,25,30 threats to internal validity of systematic review conclusions based on observational studies will be made through assessment of the body of observational literature as a whole, with an examination of characteristics of individual studies. Study-specific issues that will be considered include: potential for selection bias (i.e., degree of similarity between intervention and control patients); performance bias (i.e., differences in care provided to intervention and control patients not related to the study intervention); attribution and detection bias (i.e., whether outcomes were differentially detected between intervention and control groups); and magnitude of reported intervention effects (see the section on “Selecting Observational Studies for Comparing Medical Interventions” in AHRQ’s General Methods Guide.)25

F. Data Synthesis

We will begin by summarizing key features of the included studies. Summary tables will be constructed and organized by key question. To the degree that data are available, we will abstract information on: study design, components of the PCMH, participating practices, study population, comparator groups, length of followup, financial models or payment reform, and the types of outcomes being assessed. We anticipate greater detail for the published studies evaluated in KQ 1-3 than for ongoing studies described in KQ 4. For published studies, we anticipate the following summary tables: overview table of basic study characteristics, intervention table giving details of the intervention, and a summary table of implementation strategies. For KQ 1, we will conduct quantitative meta-analyses if there are sufficient studies with similar designs, interventions, and outcomes. If possible, we will conduct separate meta-analyses for each outcome category (e.g., patient experiences, process of care) and an analysis of the overall effect, where de-identified data are considered and overall effects (the impact factor) are rated on an ordinal scale (high, medium, low, none). If possible, we will stratify the analysis according to length of followup, using the following definitions: short term (6 months to 1 year), medium term (>1 year to 2 years), and long term (>2 years). We will classify outcomes into those hypothesized to be affected over the short term (e.g., enhanced access to care), medium term, or long term. If there is a large amount of evidence, we will consult with the TEP to prioritize the most important outcomes for analyses and summary presentations. For studies with a large number of outcomes, we anticipate focusing on outcomes related to particular “tracer conditions” such as diabetes mellitus, congestive heart failure, or depressive disorders, where comprehensive PCMH interventions may have particular value and where quality gaps are large. If meta-analyses are not indicated, we will use cross-case analyses to evaluate the association between independent variables (e.g., specific components of comprehensive PCMH) and study effect, using methods based on Miles and Huberman.31 These exploratory analyses would be considered hypothesis generating. To carry out these analyses, we will rank studies by impact level, based on an overall impact measure that considers all relevant reported outcomes. We will develop scenarios based on our hypotheses for types of interventions, features, and their relationships to impact. We will create a series of tables around these scenarios as antecedent matrices related to our conceptual model of PCMH care. We will look for patterns and verified scenarios for similar and contrasting outcomes.

G. Grading the Evidence for Each Key Question

Grading the strength of evidence will be outcome-specific; thus, a given study may be graded to be of different quality for two individual outcomes reported within that study. The strength of evidence for the highest priority outcomes in KQ 1 will be assessed using the approach described in AHRQ’s General Methods Guide.25,32 In brief, the General Methods Guide recommends assessment of four domains: risk of bias, consistency, directness, and precision. Additional domains are to be used when appropriate: coherence, dose-response association, impact of plausible residual confounders, strength of association (magnitude of effect), and publication bias. These domains will be considered qualitatively, and a summary rating will be assigned, after discussion by two reviewers, as “high,” “moderate,” or “low” strength of evidence. In some cases, high, moderate, or low ratings will be impossible or imprudent to make; for example, when no evidence is available or when evidence on the outcome is too weak, sparse, or inconsistent to permit any conclusion to be drawn. In these situations, a grade of “insufficient” will be assigned. This four-level rating scale consists of the following definitions:

High: High confidence that the evidence reflects the true effect. Further research is very unlikely to change our confidence in the estimate of effect.

Moderate: Moderate confidence that the evidence reflects the true effect. Further research may change our confidence in the estimate of effect and may change the estimate.

Low: Low confidence that the evidence reflects the true effect. Further research is likely to change the confidence in the estimate of effect and is likely to change the estimate.

Insufficient: Evidence either is unavailable or does not permit estimation of an effect.

H. Assessing Applicability

Systematic evidence reviews are conducted to summarize knowledge and to support clinicians, patients, and policymakers in making informed decisions. “Does this information apply?” is the core question for decisionmakers weighing the usefulness and value of a specific intervention or choosing among interventions. Interventions that work well in one context may not in another. The primary aim of assessing applicability is to determine whether the results obtained under research conditions are likely to reflect the results that would be expected in broader populations under “real-world” conditions. In this particular instance, we are focused on application to primary care populations.

We will assess applicability using methods described in the General Methods Guide. In brief, this method uses the PICOTS (Population, Intervention, Comparator, Outcome, Timing, Setting) framework as a way to organize information relevant to applicability. The most important issue with respect to applicability is whether the outcomes are different across studies that recruit different populations, use different intensities or forms of the intervention or comparator, or differ in duration. That is, important characteristics are those that affect baseline (comparison/control group) rates of events, intervention group rates of events, or both. We will use a checklist to guide the assessment of applicability. We will use these data to evaluate the applicability to clinical practice, paying special attention to study eligibility criteria, demographic features of the enrolled population (such as age, ethnicity, and gender) in comparison to the target population, characteristics of the intervention used in comparison with care models currently in use, and clinical relevance and timing of the outcome measures. We will summarize issues of applicability qualitatively. We seek to (1) provide organizations wishing to implement comprehensive PCMH interventions with information on the degree to which the PCMH has been implemented in settings like theirs; (2) provide payers and policymakers with information on components they may wish to encourage; and (3) provide patients with information on the types of PCMH-based organizations from which they may wish to seek care.

References

  1. Anderson G, Horvath J. The growing burden of chronic disease in America. Public Health Rep 2004;119(3):263-70.
  2. Coleman K, Mattke S, Perrault PJ, et al. Untangling practice redesign from disease management: how do we best care for the chronically ill? Annu Rev Public Health 2009;30:385-408.
  3. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med 2003;348(26):2635-45.
  4. Jencks SF, Huff ED, Cuerdon T. Change in the quality of care delivered to Medicare beneficiaries, 1998-1999 to 2000-2001. JAMA 2003;289(3):305-12.
  5. Saaddine JB, Engelgau MM, Beckles GL, et al. A diabetes report card for the United States: quality of care in the 1990s. Ann Intern Med 2002;136(8):565-74.
  6. Saaddine JB, Cadwell B, Gregg EW, et al. Improvements in diabetes processes of care and intermediate outcomes: United States, 1988-2002. Ann Intern Med 2006;144(7):465-74.
  7. Berlowitz DR, Ash AS, Glickman M, et al. Developing a quality measure for clinical inertia in diabetes care. Health Serv Res 2005;40(6 Pt 1):1836-53.
  8. Grant RW, Buse JB, Meigs JB. Quality of diabetes care in U.S. academic medical centers: low rates of medical regimen change. Diabetes Care 2005;28(2):337-442.
  9. Coleman K, Austin BT, Brach C, et al. Evidence on the Chronic Care Model in the new millennium. Health Aff (Millwood) 2009;28(1):75-85.
  10. Agency for Healthcare Research and Quality. Patient Centered Medical Home Resource Center. Available at: http://pcmh.ahrq.gov/. Accessed January 24, 2011.
  11. Scholle S, Torda P, Peikes D, et al. Engaging Patients and Families in the Medical Home (Prepared by Mathematica Policy Research under Contract No. HHSA290200900019ITO2.) AHRQ Publication No. 10-0083-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2010.
  12. Moreno L, Peikes D, Krilla A. Necessary But Not Sufficient: The HITECH Act and Health Information Technology's Potential to Build Medical Homes. (Prepared by Mathematica Policy Research under Contract No. HHSA290200900019ITO2.) AHRQ Publication No. 10-0080-EF. Rockville, MD: Agency for Healthcare Research and Quality. June 2010.
  13. Kilo CM, Wasson JH. Practice redesign and the patient-centered medical home: history, promises, and challenges. Health Aff (Millwood) 2010;29(5):773-8.
  14. Shortell SM, Gillies R, Wu F. United States innovations in healthcare delivery. Public Health Rev 2010;32(10):190-212.
  15. 15. Rittenhouse DR, Thom DH, Schmittdiel JA. Developing a policy-relevant research agenda for the patient-centered medical home: a focus on outcomes. J Gen Intern Med 2010;25(6):593-600.
  16. Crabtree BF, Chase SM, Wise CG, et al. Evaluation of patient centered medical home practice transformation initiatives. Med Care 2011;49(1):10-6.
  17. Shojania KG, Ranji SR, McDonald KM, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA 2006;296(4):427-40.
  18. Walsh JME, McDonald KM, Shojania KG, et al. Quality improvement strategies for hypertension management: a systematic review. Med Care 2006;44(7):646-57.
  19. Gilbody S, Whitty P, Grimshaw J, et al. Educational and organizational interventions to improve the management of depression in primary care: a systematic review. JAMA 2003;289(23):3145-51.
  20. Ouwens M, Wollersheim H, Hermens R, et al. Integrated care programmes for chronically ill patients: a review of systematic reviews. Int J Qual Health Care 2005;17(2):141-6.
  21. Vest JR, Bolin JN, Miller TR, et al. Medical homes: "where you stand on definitions depends on where you sit". Med Care Res Rev 2010;67(4):393-411.
  22. Kilo CM. Improving care through collaboration. Pediatrics 1999;103(1 Suppl E):384-93.
  23. Solomon DH, Van Houten L, Glynn RJ, et al. Academic detailing to improve use of broad-spectrum antibiotics at an academic medical center. Arch Intern Med 2001;161(15):1897-902.
  24. Reeves BC, Deeks JJ, T. HJP, et al. Chapter 13: Including nonrandomized studies. In: Higgins JPT, Green S (editors).Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009. Available from www.cochrane-handbook.org. Accessed February 7, 2011.
  25. Agency for Healthcare Research and Quality. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality. Available at: https://effectivehealthcare.ahrq.gov/topics/cer-methods-guide/overview/. Accessed January 24, 2011.
  26. Shojania K, McDonald K, Wachter R, et al. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies. Volume 1 - Series Overview and Methodology. Technical Review 9 (Contract No. 290-02-0017 to the Stanford University-UCSF Evidence-based Practice Center). AHRQ Publication No. 04-005101. Rockville, MD: Agency for Healthcare Research and Quality. August 2004.
  27. McDonald KM, Sundaram V, Bravata DM, et al. Care Coordination. Vol. 7 of: Shojania KG, McDonald KM, Wachter RM,Owens, DK, editors. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies. Technical Review 9 (Prepared by the Stanford University-UCSF Evidence-based Practice Center under contract 290-02-0017). AHRQ Publication No. 04(07)-0051-7. Rockville, MD: Agency for Healthcare Research and Quality. June 2007.
  28. The World Bank. Country and lending groups. Available at: http://data.worldbank.org/about/country-classifications/country-and-lending-groups. Accessed March 18, 2011.
  29. Chapman AL, Morgan LC, Gartlehner G. Semi-automating the manual literature search for systematic reviews increases efficiency. Health Info Libr J 2010;27(1):22-7.
  30. Viswanathan M, Berkman ND. Assessing the Risk of Bias and Precision of Observational Studies of Intervention or Exposures: Development, Validation, and Reliability Testing of the RTI Item Bank. RTI International–University of North Carolina Evidence-based Practice Center, Contract No. 290200710056I. Rockville, MD: Agency for Healthcare Research and Quality. In press.
  31. Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. 2nd edition. Thousand Oaks: Sage Publications; 1994.
  32. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions-Agency for Healthcare Research and Quality and the Effective Health Care Program. J Clin Epidemiol 2010;63(5):513-23.

Definitions of Terms

We will adopt AHRQ’s basic definition of the PCMH.10 AHRQ defines a medical home not simply as a place but as a model of the organization of primary care that delivers the core functions of primary health care. The medical home encompasses five functions and attributes: patient-centered, comprehensive care, coordinated care, superb access to care, and a systems-based approach to quality and safety.

Abbreviations and acronyms used in this protocol are:

ACO Accountable care organization


AHRQ Agency for Healthcare Research and Quality


CDSR Cochrane Database of Systematic Reviews


CER Comparative Effectiveness Review


CMS Centers for Medicare & Medicaid Services


EPC Evidence-based Practice Center


EPOC Cochrane Effective Practice and Organisation of Care Review Group


HbA1c Glycated hemoglobin


HMO Health maintenance organization


HRSA Health Resources and Services Administration


KQ Key Question


NIH National Institutes of Health


PCMH Patient-centered medical home


PDSA Plan-do-study-act


PICOTS Population, Intervention, Comparator, Outcome, Timing, Setting


QI Quality improvement


RCT Randomized controlled trial


TEP Technical Expert Panel


TOO Task Order Officer


VA United States Department of Veterans Affairs

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale.

Review of Key Questions

For all EPC reviews, key questions were reviewed and refined as needed by the EPC with input from the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed.

Technical Experts

Technical Experts comprise a multi-disciplinary group of clinical, content, and methodologic experts who provide input in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore, study questions and design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues, as requested by the EPC. Technical Experts do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Individuals are invited to serve as Technical Experts because of their unique clinical or content expertise, and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for Comparative Effectiveness Reviews (CERs) and Technical Briefs, be published three months after the publication of the Evidence Report.

Potential reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.