Background and Objectives for the Comparative Effectiveness Review

Objective:

To determine the effectiveness of case management in adult patients with medical illness(es) and complex care needs.

Summary of Nomination:

The original topic nomination proposed a comparative effectiveness review of case management (performed by certified nurse case managers) for improving utilization and costs of health services. The original nomination specified a broad population of interest (“all patients”) and did not further specify the outcomes of interest. Because a literature scan identified diverse populations, interventions, and outcomes, the nomination was further scoped during topic refinement to produce more specific key questions.

This review will focus on a specific clinical strategy—case management. We define case management as a process in which a person (alone or in conjunction with a team) manages multiple aspects of a patient’s care. Key components of case management include planning and assessment, coordination of services, patient education, and clinical monitoring.¹

Given that case management interventions and outcomes differ substantially for medical illness when compared with mental illness, we decided to limit the scope of the report to case management for adults with medical illness. This limited scope would not exclude patients with medical illness and coexisting mental illness, but it would exclude patients for whom case management is used primarily to manage mental illness.

Background and Clinical Context:

Patients with complex health care needs because of multiple chronic conditions, multiple treatments, and/or multiple providers may be at increased risk of experiencing inadequate quality of care.² In addition, health care expenditures are substantially higher for patients with complex care needs when compared with those who have few or no chronic conditions.² Care coordination refers to a variety of strategies that have the potential to improve care and reduce costs for patients with complex health care needs.³ Care coordination has been defined as “the deliberate organization of patient care activities between two or more participants (including the patient) involved in a patient’s care to facilitate the appropriate delivery of health care services.”³ As reported by Bodenheimer and Berry-Millett,² a lack of care coordination for patients with complex care needs can lead to preventable problems, such as higher rates of hospitalization. Case management is a strategy that potentially can improve patient outcomes.³ Case management has been used in a wide variety of clinical settings, and case managers typically are registered nurses or social workers. There are organizations that offer certification for case managers, but there have been few studies examining whether certification is associated with improved clinical performance.

Multiple systematic reviews have evaluated the effectiveness of case management for a variety of patient groups, medical conditions, settings, and outcomes.³ The findings and conclusions of these diverse reviews have varied. Some systematic reviews have assessed case management for individual chronic diseases, including a review that found improved glycemic control among patients with diabetes⁴ and a review of congestive heart failure with mixed results across multiple disease-related and resource-utilization outcomes.⁵ Another recent review of nurse-led case management for complex patients in general health care, which excluded patients with only one specific disease, found improved patient satisfaction but no effect on emergency department visits The individual studies of case management have examined a variety of outcomes. The most common justification for case management is to improve quality of care, and various health outcomes can serve as measures of quality. There also has been interest in case management as a means to prevent adverse clinical events, such as falls, preventable hospitalizations, or adverse drug events. Reducing adverse events potentially could make health care more efficient, so measures of utilization and cost are other relevant outcome measures.

Investigators from the Stanford University–UCSF Evidence-based Practice Center reviewed previously published systematic reviews on a wide variety of care-coordination interventions, including case management. In a technical report prepared for AHRQ in 2007, these investigators reported that case management may improve patient outcomes for heart failure and diabetes.³ Their review, however, evaluated only previously published systematic reviews of the broader topic of care coordination and did not synthesize evidence from the primary studies of case management.

PICOs and Key Questions

Key questions (KQs) were drafted with input from Key Informants and posted for public comment. Based on the input received, we made minor changes. Many responders suggested patient outcomes and intervention characteristics that were not included in the key questions. Therefore, we reformatted the KQs and included the phrase “including but not limited to.”

Population:

Adults with medical illness and complex care needs

Interventions:

Case management– defined as a process in which a person (alone or in conjunction with a team) manages multiple aspects of a patient’s care.

Comparators:

Case management will be compared with usual care (i.e., care without a case management component). If a study compares two or more different types of case management, then the comparator will be the alternative type of case management.

Outcomes:

1. Patient health outcomes (include but are not limited to): overall quality of care; disease-specific quality of care; disease-specific health outcomes; quality of life; patient satisfaction with care; morbidity; and mortality.
2. Resource utilization outcomes (include but are not limited to): overall financial cost; hospitalization rates; rehospitalization rates; emergency department use; and number of clinic visits.
3. Process measure outcomes (include but are not limited to): adherence to therapy; missed appointments; patient self-management; change in health behavior; disease-specific processes of care; patient and family perceptions of participation in decisions; medication adherence; and physician/case manager satisfaction.

Timing:

No minimum duration of intervention or follow-up. However, study duration will be considered in the assessment of the quality and applicability of the study.

Settings:

Primary care, specialty care, and home care. No geographic limitations will be applied.

KQ 1:

In adults with medical illness and complex care needs, does case management improve patient health outcomes (as listed above) when compared with usual care or other models of case management?

1. Does the effectiveness of case management for patient health outcomes differ according to patient characteristics?
   
   Patient characteristics include, but are not limited to: particular medical conditions; number or type of comorbidities; patient age and socioeconomic status; social support; and/or level of formally assessed health risk.
2. Does the effectiveness of case management for patient health outcomes differ according to intervention characteristics?
   
   Intervention characteristics include, but are not limited to: practice or health care system setting; case manager experience, training, or skills; case management tools, techniques or information systems; and complexity of the case management program.

KQ 2:

In adults with medical illness and complex care needs, does case management affect resource utilization outcomes (as listed above) when compared with usual care or other models of case management?

1. Does the effectiveness of case management for resource utilization outcomes differ according to patient characteristics?
2. Does the effectiveness of case management for resource utilization outcomes differ according to intervention characteristics?

KQ 3:

In adults with medical illness and complex care needs, does case management affect process measure outcomes (as listed above) when compared with usual care or other models of case management?

1. Does the effectiveness of case management for process measure outcomes differ according to patient characteristics?
2. Does the effectiveness of case management for process measure outcomes differ according to intervention characteristics?

Analytic Framework

Abbreviations: KQ = key question; QOL = quality of life.

Methods

A. Criteria for Inclusion/Exclusion of Studies in the Review

Included literature included will meet the PICO criteria outlined above. We will include observational studies, systematic reviews, and clinical trials. We will exclude case studies and small case series. Non-English-language articles will be included in this review and translated when it is feasible to do so. Grey literature will be identified by searching clinical trial registries (ClinicalTrials.gov, Current Controlled Trials, Clinical Trial Results, WHO Trial Registries), grants databases (NIHRePORTER, HSRProj, AHRQ GOLD), and individual funders' Web sites.

B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies To Answer the Key Questions

Results from previously conducted meta-analyses and systematic reviews on these topics will be sought and used where appropriate and updated when necessary. To identify systematic reviews, we will search the Cochrane Databases of Systematic Reviews and Controlled Trials and CINAHL (EBSCO) in addition to MEDLINE.

To identify articles relevant to each KQ, we will search the Cochrane Database of Systematic Reviews, the Cochrane Central Register of Controlled Trials, EBM Reviews, and Ovid MEDLINE®. We will use broad searches by combining terms for case management with terms for relevant research designs while limiting studies to those that focused on adults with medical illness(es) and complex care needs.

Our research team will use appropriate procedures to reduce bias and enhance consistency in our study-selection process. These procedures include using dual reviewers to review abstracts and full-text articles for inclusion and exclusion for each KQ. After finalizing our literature searches, we will review titles and abstracts by using our pre-established inclusion/exclusion criteria to determine potential eligibility for inclusion in the evidence synthesis. We will initially review 100 citations in triplicate and calculate kappa values to estimate inter-reviewer reliability. After discussing and reconciling disagreements between reviewers, we will review an additional 100 citations in triplicate. We will continue this process until the kappa values reach >0.50 for each pair of reviewers. We then will review all remaining citations in duplicate. All citations that are judged to meet the inclusion criteria by at least one reviewer will be retrieved for full-text review.

All retrieved studies will be reviewed in duplicate. For the studies that meet inclusion criteria at this stage, key data from each eligible study will be extracted and entered into an electronic database. A data file of excluded studies with reasons for exclusion will be maintained. We will use a consensus process to resolve conflicts. Searches will be updated while the report is posted for public comment and peer review to capture any new publications. Literature identified during the update search will go through the same process of dual review as all other studies considered for inclusion in the report. If any pertinent new literature is identified for inclusion in the report, it will be incorporated before the final submission of the report.

C. Data Abstraction and Data Management

The following data will be extracted from included trials: study design and setting; population characteristics (including sex, age, ethnicity, diagnosis); eligibility and exclusion criteria; case management intervention and comparisons; the outcome-ascertainment method, if available; and results for each study outcome. We will record intention-to-treat results if available.

D. Assessment of Methodological Quality of Individual Studies

We will assess the quality of systematic reviews, randomized trials, and cohort and case-control studies based on predefined criteria. We will adapt criteria from the Assessment of Multiple Systematic Reviews (AMSTAR) tool (systematic reviews),⁷ methods proposed by Downs and Black (observational studies),⁸ and methods developed by the U.S. Preventive Services Task Force.⁹ Results from poor-quality studies will most likely be excluded from data syntheses, though these data will still be included in evidence tables. The criteria we will use are consistent with the approach recommended by AHRQ in the draft Methods Reference Guide for Effectiveness and Comparative Effectiveness Reviews.¹⁰

Systematic Reviews: Included systematic reviews will also be rated for quality based on predefined criteria by assessing whether they had a clear statement of the questions(s), reported inclusion criteria, used an adequate search strategy, assessed validity, adequately reported the details of included studies, and used appropriate methods to synthesize the evidence. We will include systematic reviews and meta-analyses that included unpublished data inaccessible to the public; but because the results of such analyses are not verifiable, we will consider this a methodological shortcoming.

Trials: We will rate the internal validity of each trial based on: the methods used for randomization, allocation concealment, and blinding; the similarity of compared groups at baseline; maintenance of comparable groups; adequate reporting of dropouts, attrition, crossover, adherence, and contamination; loss to followup; and the use of intention-to-treat analysis. Trials that had a fatal flaw in one or more categories will be rated as poor quality; trials that met all criteria will be rated good quality; and the remainder will be rated fair quality. Because the fair-quality category is broad, studies with this rating vary in their strengths and weaknesses: the results of some fair-quality studies are likely to be valid, while others are only probably valid. A poor-quality trial is not valid—the results are at least as likely to reflect flaws in the study design as the true difference between the compared interventions.

Observational Studies: For assessing the internal validity of observational studies, we will evaluate: whether nonbiased selection methods were used; whether rates of loss to followup were acceptable; whether predefined outcomes were specified; whether they used appropriate methods for ascertaining exposures, potential confounders, and outcomes; and whether appropriate statistical analyses of potential confounders were performed. Although many tools exist for quality assessment of nonrandomized trials, there is no consensus on optimal quality-rating methods.^{10, 11}Therefore, we will not use a formal scoring system to rate the quality of the observational studies included in this review but will note methodological deficiencies in any of the areas listed above when present.

E. Data Synthesis

We will construct evidence tables showing study characteristics and quality ratings for all included studies. To determine the appropriateness of meta-analysis, we will consider the clinical and methodological diversity and assessed statistical heterogeneity. Appropriate measures will be chosen based on the type of data available for meta-analysis. We will assess the presence of statistical heterogeneity among studies by using standard X² tests and the magnitude of heterogeneity by using the I² statistic. When appropriate, studies will be combined by using a random-effects model while accounting for variation among studies. We will use a fixed-effects model for combining rare binary outcomes. When there is no variation among studies, the random-effects model yields the same results as a fixed-effects model. Statistical heterogeneity will be explored by using subgroup analysis or meta-regression. Subgroups will be defined by demographic and clinical criteria. Demographic criteria include age groups, urban/rural residence, household income, and racial/ethnic groups. Clinical criteria include principal diagnosis and medication complexity.

When statistical meta-analysis is not possible, we will group studies by similarity of intervention characteristics and plot trends in the study findings. Wherever possible, similar outcome measures will be grouped across the studies to make preliminary estimates of effect sizes. The outcomes will be compared among patient groups who receive different types of case management. Direct comparisons will be made when head-to-head trials are available. Otherwise, indirect comparisons will be considered, that is, if outcome measures in the nonintervention or usual-care arms are similar across the studies evaluated.

F. Grading the Evidence for Each Key Question

We will use the chapter, Grading the Strength of a Body of Evidence when Comparing Medical Interventions, in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.¹⁰

Definition of Terms

Case management: a specific clinical strategy in which a person (alone or in conjunction with a team) manages multiple aspects of a patient’s care.

References

1. Tahan HA, Campagna A. Case management roles and functions across various settings and professional disciplines. Prof Case Manag 2010;15:245-77..
2. Bodenheimer T, Berry-Millett R. Care management of patients with complex health care needs. Princeton, NJ: Robert Wood Johnson Foundation; 2009. Research Synthesis Report No. 19.
3. McDonald KM, Sundaram V, Bravata DM, et al. Care coordination. Vol. 7 of: Shojania KG, McDonald KM, Wachter RM and Owens DK, eds. Closing the quality gap: a critical analysis of quality improvement strategies. Technical Review No. 9 (Prepared by Stanford University–UCSF Evidence-based Practice Center under Contract No. 290-02-0017). Rockville, MD: Agency for Healthcare Research and Quality; June 2007. AHRQ Publication No. 04(07)-0051-7.
4. Norris SL, Nichols PJ, Caspersen CJ, et al. The effectiveness of disease and case management for people with diabetes: a systematic review. Am J Prev Med 2002;22(4 Suppl):15-38.
5. Windham BG, Bennett RG, Gottlieb S. Care management interventions for older patients with congestive heart failure. Am J Manag Care 2003;9:447-59.
6. Latour CH, van der Windt DA, de Jonge P, et al. Nurse-led case management for ambulatory complex patients in general health care: a systematic review. J Psychosom Res 2007;62:385-95.
7. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BCM Med Res Methodol 2007 Feb 15;7:10. DOI: 10.1186/1471-2288-7-10.
8. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies in health care interventions. J Epidemiol Community Health 1998;52:377-84.
9. Harris RP, Helfand M, Woolf SH, et al, for the Methods Work Group of the Third US Preventive Services Task Force. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(3 Suppl):21-35.
10. Agency for Healthcare Research and Quality. Methods guide for effectiveness and comparative effectiveness reviews, Version 1.0. Rockville, MD: Agency for Healthcare Research and Quality. Draft Posted October 2007. Available at: /sites/default/files/pdf/cer-methods-guide_research-2008.pdf.
11. Deeks JJ, Dinnes J, D'Amico R, et al. Evaluating non-randomized intervention studies. Health Technol Assess 2003;7(27):1-192, iii-x.
    

Summary of Protocol Amendments

The original version of the Key Questions has the same scope as the revised version. However, many of the studies included in the report address outcomes in more than one category, and the revised Key Questions now group the outcome categories as parts of Key Question 1.

Few of the studies include sub-group analyses. Thus, most evidence about patient sub-groups is indirect (comparisons across separate studies). Thus, it is more appropriate to address patient sub-groups as a separate Key Question (KQ2 in the protocol amendment). In addition, most studies compare a single model of case management to a usual care condition, and relatively few studies directly compare more than one model of case management. Thus, comparisons among intervention characteristics are usually indirect, and these analyses also are included in a separate Key Question (KQ3 in the protocol amendment).

With the revision of the Key Questions of this evidence review, the analytic framework was also revised to reflect these changes.

Revised Key Questions:

Key Question 1:

In adults with chronic medical illness and complex care needs, is case management (CM) effective in improving:

1. Patient-centered outcomes, including mortality, quality of life (QOL), disease-specific health outcomes, avoidance of nursing home placement, and patient satisfaction with care?
2. Quality of care, as indicated by disease-specific process measures, receipt of recommended health care services, adherence to therapy, missed appointments, patient self-management, and changes in health behavior?
3. Resource utilization, including overall financial cost, hospitalization rates, days in the hospital, emergency department use, and number of clinic visits (including primary care and other provider visits)?

Key Question 2:

Does the effectiveness of CM differ according to patient characteristics, including but not limited to: particular medical conditions, number or type of comorbidities, patient age and socioeconomic status, social support, and/or level of formally assessed health risk?

Key Question 3:

Does the effectiveness of CM differ according to intervention characteristics, including but not limited to: practice or health care system setting; case manager experience, training, or skills; CM intensity, duration, and integration with other care providers; and the specific functions performed by case managers?

Revised Analytic Framework

Abbreviations: KQ = key question; QOL = quality of life.

NOTE: The following protocol elements are standard procedures for all protocols.

Review of Key Questions

For all EPC reviews, key questions are reviewed and refined as needed by the EPC with input from Key Informants and the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed. In addition, for Comparative Effectiveness reviews, the key questions are posted for public comment and finalized by the EPC after review of the comments.

Key Informants

Key Informants are the end users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the Key Informant role is to provide input into identifying the Key Questions for research that will inform healthcare decisions. The EPC solicits input from Key Informants when developing questions for systematic review or when identifying high priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Technical Expert Panel (TEP)

Technical Experts comprise a multi-disciplinary group of clinical, content, and methodologic experts who provide input in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as healthy scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Review

Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodologic expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published 3 months after the publication of the Evidence report.

Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.