Skip to main content
Effective Health Care Program

Effectiveness of Practice-based Interventions Addressing Concomitant Mental Health and Chronic Medical Conditions in the Primary Care Setting

Research Protocol
Download PDF382.6 KB

Background and Objectives for the Systematic Review

As outlined in the World Health Organization report on integrating mental health into primary care, “[M]ental health disorders are prevalent in all societies. They create a substantial personal burden for affected patients and their families, and they produce significant economic and social hardships that affect society as a whole.”1 Because primary care is the setting where most people receive care for such conditions,2 there has been considerable interest in improving the recognition and management of mental health conditions, especially depression, within primary care.3, 4

There is emerging literature addressing whether improved treatment of depression in primary care can also improve chronic medical outcomes, such as for diabetes.5-7 The field of mental health and primary care is moving from consideration of single conditions and their outcomes to more real-world complex-care paradigms.8, 9 However, to date, there has not been a synthesis of the evidence in a way that accounts for the primary care patient with “multiple chronic conditions”10 and examines both mental health and chronic medical outcomes. One recent study illustrates an example of such an intervention in which primary care patients with depression and poorly controlled diabetes, poorly controlled coronary artery disease, or both, received care management of multiple conditions at once.11 A review of similar studies could help address the clinical uncertainty about whether such collaborative interventions can make a difference in more than one domain and inform policy decisions about the potential benefit of adopting such guidance.

According to the National Comorbidity Survey Replication, the 12-month prevalence of any mental health disorder in the United States is 26.2 percent, with more than half of these cases (14.4 percent) meeting criteria for only one disorder and smaller proportions for two (5.9 percent) or more (5.9 percent) disorders.12 Anxiety disorders are by far the most prevalent class of disorders (18.1 percent), followed by mood disorders (9.5 percent). Worldwide, over 150 million people suffer from unipolar depressive disorders.13 Depression can be seen in up to 10 percent of all primary care patients,14 and patients with a chronic general medical condition are two to three times more likely to suffer from depression than healthy individuals.15 Approximately 40 million American adults ages 18 or older, or about 18.2 percent of people in this age group, have an anxiety disorder in a given year.12 Patients with generalized anxiety disorder often have multiple medical comorbidities; among the most frequently reported are migraine, rheumatoid arthritis, peptic ulcer disease, irritable bowel syndrome, coronary heart disease, hyperthyroidism, diabetes, asthma, and chronic obstructive pulmonary disease.16 Among primary care outpatients with hypertension, diabetes, and/or heart disease, between 26 percent and 28 percent of patients have reported a diagnosis of anxiety disorder at some point in their lives.2 The overall prevalence of mental health disorders appears to affect both men and women equally. However, women have a higher prevalence of depression and most anxiety disorders.1 The groups most likely to have unmet mental health care needs include the elderly, children and adolescents, members of ethnic minorities, the uninsured, low-income individuals, and individuals who predominantly complain of physical symptoms as a manifestation of their mental health problem.9

In terms of social and economic costs, anxiety and depression top the list of mental health conditions causing high societal burden.17 Mental disorders are currently the leading cause of disability in the United States for ages 15 to 44.13 Worldwide, depression makes a large contribution to the burden of disease, ranking third worldwide, eighth in low-income countries, and first in middle- and high-income countries.13 Comorbid depression among people with chronic physical illness has been linked to an increase in health care utilization, disability, and work absenteeism when compared with those without comorbid depression, even after controlling for the varying burden of the physical health condition.18 In 2000, the U.S. economic burden of depressive disorders was estimated to be $83.1 billion. More than 30 percent of these costs are attributable to direct medical expenses.19 The economic cost of anxiety disorders has been estimated to be as much as $54.9 billion per year.20

In 2006, heart disease, stroke, chronic obstructive pulmonary disease and its allied conditions (including asthma), and diabetes were among the 10 leading causes of death in the United States.21 In 2006, 631,636 people died of heart disease, accounting for 26 percent of deaths in the United States. An estimated 23.6 million people (7.8 percent of the U.S. population) have diabetes.22 Roughly 24 million U.S. adults have chronic obstructive pulmonary disease, and an additional 23 million have asthma.21 In 2007, the leading causes of activity limitation included arthritis, back/neck pain, heart disease, diabetes, hypertension, and lung conditions.

Chronic medical conditions are significant drivers of health care costs. In a study conducted in 2001 and 2002,23 an estimated 13 percent of the total U.S. workforce experienced a loss in productive time during a 2-week period because of pain (including arthritis), costing an estimated $61.2 billion per year. In 2010, cardiovascular, lung, and blood diseases were projected to cost $486 billion in health care expenditures, not including lost productivity or costs attributed to them as secondary causes of morbidity and mortality.21 Direct costs for diabetes and arthritis were projected to be $116 billion22 and $194 billion,24, 25 respectively. In 2010, heart disease was estimated to cost the United States $311.1 billion in direct and indirect costs, accounting for the highest costs of any condition.

According to a 2003 report for the President’s Commission on Mental Health, half of the care for common mental disorders in the United States is delivered in general medical settings.9 Primary care providers, therefore, play a vital role in the diagnosis and treatment of mental disorders, although there is evidence to suggest that the quality of depression care provided by primary care providers is often suboptimal.26 Options primary care providers can use to address mental health problems in primary care settings include 1) referral to specialty mental health providers, 2) treatment by primary care providers using guidelines, 3) stepped care approaches, 4) collaborative care, and 5) various mechanisms of integrated care. Despite poor adoption of these options, the latter two have been well studied and have shown convincing effectiveness.3

Repeated evidence reviews show the benefits of integrated and collaborative care models—as compared to usual care—on the outcomes of depression in the general health setting.3, 15, 27 Although literature on the treatment of other mental health conditions such as anxiety in primary care is only beginning to emerge, data suggest that those conditions may also be successfully treated in primary care.28 Numerous studies have begun to address specific conditions—such as diabetes5 and chronic pain29—as they relate to depression, but the impact of treating mental health conditions in primary care on chronic medical outcomes has yet to be fully examined.

Two recent reports have particular relevance to this topic: a 2008 Agency for Healthcare Research and Quality (AHRQ) report examining the integration of mental health/substance abuse treatment and primary care27 and a 2009 National Institute for Health and Clinical Excellence (NICE) guideline for depression in adults with a chronic physical health problem.15 These reports neither specified primary care as the setting of inclusion nor examined disease-specific chronic medical outcomes, both of which our proposed review would include.


The 2008 AHRQ Evidence-based Practice Center (EPC) report titled Integration of Mental Health/Substance Abuse and Primary Care27 examined the evidence for integrating mental health services into primary care settings and primary services into specialty mental health settings. This report focused on four areas: specifying what integration is and is not, detailing the process through which integrated care may affect clinical outcomes, expanding previous reviews to include multiple illnesses and patient populations, and specifying the conditions under which various models of integrated care are likely to work in real-world settings. The AHRQ report found that, in general, integrated care achieved positive outcomes but that there is a lack of consistency among integrated care models. Our review will incorporate the relevant findings of the AHRQ report and attempt to examine the elements of practice-based interventions that affect outcomes.

The 2009 NICE guideline on depression in adults with a chronic physical health problem15 remains very relevant to this topic. However, the NICE guideline considered only adults with depression (not anxiety) and did not focus on the primary care setting. In addition, it excluded studies that were aimed at improving a concomitant physical health condition, which is a key component of this review. Chronic physical conditions examined in the NICE guideline included diabetes, cancer, asthma, stroke, arthritis, hypertension, and general medical illness. The studies identified in the NICE report provided consistent evidence that collaborative care had benefits on depression outcomes, particularly remission and response. However, with the exception of pain intensity and general physical functioning, the studies did not provide comparable data on physical health outcomes. Limited evidence suggests that collaborative care improved adherence to medication regimens for chronic physical conditions. Recommendations from the NICE guideline include additional research to examine the effects of collaborative care on physical health outcomes for patients with moderate to severe depression and a chronic physical health problem, which is something we intend to address with our review.

This protocol includes revisions to the Key Questions (KQs) and PICOTS (specifically the change from “service-level” to “practice-based” interventions) suggested during the public comment period and revisions suggested by members of our Technical Expert Panel (TEP), including additional search terms (refined by the Research Librarian; see Section IV) and outcomes (see Sections II and IV).

Key Questions

Question 1

  1. a. Among adults with chronic medical conditions and concomitant depression and/or anxiety (such as patients with diabetes and depression) treated in the primary care setting, what is the comparative effectiveness of practice-based interventions aimed at improving the mental health condition or both the mental health and chronic medical conditions (when compared to similar interventions or usual care) on intermediate depression/anxiety outcomes (e.g., symptom improvement)?
  2. b. Among adults with chronic medical conditions and concomitant depression and/or anxiety (such as patients with diabetes and depression) treated in the primary care setting, what is the comparative effectiveness of practice-based interventions aimed at improving the mental health condition or both the mental health and chronic medical conditions (when compared to similar interventions or usual care) on other mental health outcomes (e.g., depression-related quality of life) and mental health-related utilization?

When possible, we will include subgroup analyses that focus on a relevant subset of participants for both KQ 1a and KQ 1b.

Question 2

  • a. Among adults with chronic medical conditions and concomitant depression and/or anxiety (such as patients with diabetes and depression) treated in the primary care setting, what is the comparative effectiveness of practice-based interventions aimed at improving the mental health condition or both the mental health and chronic medical conditions (when compared to similar interventions or usual care) on intermediate chronic medical outcomes (e.g., HbA1c for patients with diabetes)?
  • b. Among adults with chronic medical conditions and concomitant depression and/or anxiety (such as patients with diabetes and depression) treated in the primary care setting, what is the comparative effectiveness of practice-based interventions aimed at improving the mental health or both the mental health and chronic medical conditions (when compared to similar interventions or usual care) on general health outcomes (e.g., diabetes-related morbidity)?

When possible, we will include subgroup analyses that focus on a relevant subset of participants for both KQ 2a and KQ 2b.


The following will be addressed in the context of the studies examined for KQs 1 and 2:

Question 3

What harms are associated with practice-based interventions for primary care patients with chronic medical conditions and concomitant depression and/or anxiety?

Question 4

What are the characteristics of the practice-based interventions addressing concomitant mental health and chronic medical conditions used in the primary care setting with regard to specific components and/or intensity (e.g., visit frequency, total number of contacts, provider discipline, self-management)?

Question 5

What are the specific characteristics of the practice setting where the interventions were delivered with regard to such variables as organizational characteristics (e.g., decision support, level of integration, information technology, electronic medical record, on-site mental health services, payer and service mix, practice size, and practice location/setting) or the relationship between elements of the system in which the practice operates (e.g., coordination, financing of care, payment arrangements)?

The PICOTS criteria for the KQs above are as follows:

Population

The population for these KQs is adults (ages 18 or older) with one or more chronic medical conditions and concomitant depression and/or anxiety (mental health condition). An example is patients with diabetes and depression.

  • Chronic medical conditions were identified as priorities by AHRQ30 and the Institute of Medicine31 and narrowed by a preliminary search for available evidence to the following:
    • Arthritis
    • Diabetes
    • Asthma or chronic obstructive pulmonary disease
    • Cancer
    • Chronic pain
    • Stroke
    • HIV/AIDS
    • Heart disease, heart failure, myocardial ischemia, coronary artery bypass graft, post–myocardial infarction, and coronary artery disease
    • “Complex” patients with multiple comorbidities
    • Frailty due to old age
  • Depression and anxiety are defined as threshold-level conditions, meeting criteria for a disorder as determined by valid and reliable measures with established cut points to exclude subthreshold symptoms and minor depression. Examples include:
    • Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, Text Revision (DSM-IV-TR)
    • Patient Health Questionnaire (PHQ-9)
    • Hamilton Rating Scales for Depression (HAM-D) and Anxiety (HAM-A)
    • Generalized Anxiety Disorder 7 (GAD-7)
    • Montgomery-Åsberg Depression Rating Scale (MADRS)
    • Geriatric Depression Scale (GDS)
    • Beck Depression Inventory (BDI and BDI-II)
    • Center for Epidemiologic Studies Depression Scale (CES-D)

Interventions

The interventions for the KQs are practice-based interventions aimed at improving the mental health condition or both the mental health and chronic medical conditions.

The term “practice-based intervention” is used many different ways in the health literature, but the working definition for the purposes of our review is “any intervention that targets the care process within a system of care.” Examples of practice-based interventions include coordinated care, integrated care, and collaborative care. Medication-only, device, and psychotherapy-only interventions and studies (e.g., efficacy trials comparing a medication to a placebo) will be excluded. Practice-based interventions can also include person-level components.

As the evidence allows, we will address interventions delivered at the various levels, including payer, practice, and patient.

Comparators

Comparators include practice-based interventions (as above and including usual care) aimed at improving the mental health condition or both the mental health and chronic medical conditions.

Usual care can include no treatment, prescription of medications, or referral to a mental health specialist.

Outcomes Measures for Each Key Question

  • Intermediate mental health outcomes include the following:
    • Symptom improvement, response rates, and remission and/or recurrence as measured by scores on reliable and valid instruments (to include self-rated instruments)
    • Treatment adherence
    • Satisfaction with care
  • Intermediate chronic medical condition outcomes include the following:
    • Symptom improvement and remission
    • Response to treatment (e.g., HbA1c)
    • Treatment adherence
    • Satisfaction with care
  • Other mental health outcomes include the following:
    • Disease-related mortality
    • Disease-related morbidity
    • Disease-related functional status
    • Mental health-related quality of life
    • Mental health care utilization
    • Sick days due to mental health
    • Employment stability
  • General health outcomes include the following:
    • All-cause mortality
    • Disease-related mortality
    • Disease-related morbidity
    • Disease-related functional status
    • General health-related quality of life
    • Disease-specific outcomes
    • General health care utilization
    • Total sick days and sick days due to general health condition
    • Employment stability
    • Individual and system costs, as reported by the individual study
  • Harms include the following:
    • Adverse effects of pharmacotherapy
    • Other harms as reported

Timing

Studies must be at least 6 months in duration. Our Key Informants felt that results from studies less than 6 months in duration would be difficult to interpret and not clinically meaningful, given our specified outcomes.

Settings

Settings will include traditional primary care settings (e.g., family medicine, internal medicine, obstetrics/gynecology, and geriatrics) and settings with a primary care-type relationship (e.g., oncology clinics for those with cancer, infectious disease clinics for those with HIV).


* An example of a “person-level” component is a particular psychotherapy, such as problem-solving therapy — that while part of a broader practice-based intervention, (e.g., coordinated care) would be delivered directly to the patient by a trained practitioner in the primary care setting; another example is antidepressant medication, as part of the larger intervention, but delivered at the patient level. While we are selecting for practice-based interventions that target an overall process of care, we will not be excluding components of such interventions because they occur at the patient level.

Analytic Framework

Figure 1 is titled “Effectiveness of Practice-based Interventions Addressing Concomitant Mental Health and Chronic Medical Conditions in the Primary care Setting.” The framework begins on the left with our patient population of interest: adults with chronic medical conditions and concomitant depression and/or anxiety. We also specify the following subgroups of interest: older adults, veterans, racial / ethnic minorities, women. An arrow with the word “intervention” starts from the population on the left and splits into four arrows, each leading to a box on the right. Each of these arrows has a specific KQ listed above it. The four boxes are labeled as: “Intermediate mental health outcomes” (KQ 1a); “Intermediate chronic medical outcomes” (KQ 2a); “Other mental health outcomes” (KQ 1b); and “General health outcomes” (KQ 2b). A curvy arrow labeled “KQ 3” descends from the initial “intervention” arrow to an elliptical shape with the words “Harms of intervention” to illustrate the focus of KQ3. A dashed line starts at the “Intermediate mental health outcomes” box and goes to the right, connecting to the “Other mental health outcomes” box. Similarly, a dashed line starts at the “Intermediate chronic medical outcomes” box and goes to the right, connecting to the “General health outcomes” box. A vertical dashed line connects the “Other mental health outcomes” and “General health outcomes” boxes. A diamond-shaped ‘kite’ containing the words “Characteristics of intervention” is located near the top of the figure and is connected to the main intervention arrow with two dashed vertical lines to illustrate the foci of KQs 4 (intervention components and intensity) and 5 (practice setting factors).

Methods

A. Criteria for Inclusion/Exclusion of Studies in the Review

Table 1 presents the inclusion/exclusion criteria for this review. We do not repeat all of the PICOTS information related to inclusion/exclusion criteria.

Table 1. Inclusion/exclusion criteria
Category Criteria
Inclusion Exclusion
* If evidence from controlled trials fails to address one or more KQs, we will consider expanding our search to include observational studies according to the methodology set forth in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews32 and training module.33




Abbreviations: AHRQ = Agency for Healthcare Research and Quality; KQ = key question; PICOTS = population, interventions, comparators, outcomes, timing, and settings.
Population
  • Adults (18 years of age or older)
  • One or more chronic medical conditions as listed above in the PICOTS
  • Threshold-level depression and/or anxiety as described above in the PICOTS
  • Children and adolescents (under 18 years of age)
  • Subthreshold depression or anxiety; minor depression
  • Mental health condition other than depression or anxiety
  • Chronic medical condition other than those listed in the PICOTS
Geography
  • No limits
None
Time period
  • 1990 to present; searches to be updated after draft report goes out for peer review
  • Articles published before 1990
Length of followup
  • At least 6 months (24 weeks)
  • Fewer than 6 months
Settings Traditional primary care settings
  • Settings with a primary care-type relationship (as described in the PICOTS)
  • All other settings
Interventions
  • As defined above in the PICOTS
  • Studies without a practice-based component (e.g., medication only, referral only)
Outcomes
  • As defined above in the PICOTS:
  • Intermediate mental health outcomes
  • Intermediate chronic medical outcomes
  • Other mental health outcomes
  • Chronic medical and general health outcomes
  • Costs of interventions
  • Harms
Publication language
  • English
  • All other languages
Admissible evidence (study design and other criteria)
  • Original research; eligible study designs include:
    • Randomized controlled trials
    • Nonrandomized controlled trials with concurrent eligible controls
    • Systematic reviews with or without meta-analyses
    • Subgroup and/or posthoc analyses of data from relevant controlled trials
  • For KQs 3, 4, and 5, we will evaluate the information within the studies included for KQs 1 and 2.
  • Case series
  • Case reports
  • Nonsystematic/narrative reviews
  • Editorials
  • Letters to the editor
  • Articles rated poor during quality assessment
  • Studies with historical, rather than concurrent, control groups
  • Observational studies*

B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies To Answer the Key Questions

We will systematically search, review, and analyze the scientific evidence for each KQ. The steps that we will take to accomplish the literature review are described below.

To identify articles relevant to each KQ, we will begin with a focused MEDLINE® search on concomitant mental health and chronic medical conditions using a variety of terms, medical subject headings (MeSH®), and major headings, limited to English-language (due to time and resources) and human-only studies. Relevant terms are listed in Table 2. We will also search the Cochrane Library, the Cumulative Index to Nursing and Allied Health Literature (CINAHL®), EMBASE®, and PsycINFO® using analogous search terms. We will conduct quality checks to ensure that the known studies (i.e., studies identified during Topic Nomination and Refinement and relevant studies from the AHRQ27 and NICE15 reports) are identified by the search. If they are not, we will revise and rerun our searches.

The authors of the AHRQ report27 comprehensively searched the literature published since 1950 and did not find any references that met all inclusion criteria and were published before 1992. Similarly, the authors of the NICE guideline15 did not identify any trials related to practice-based interventions that were published before 1996. Those authors noted that the growth of interest in the development of systems of care for managing depression in people with chronic medical conditions began around 1990, due largely to organizational developments in the health care system and the recognition of mental health conditions as chronic and disabling disorders. As a result, we will limit our database search to articles published from 1990 through the present.

We will search the “gray literature” for unpublished studies relevant to our review; we will include studies that meet all the inclusion criteria and report enough methodological information to assess internal validity/quality. Potential sources of gray literature include AHRQ evidence reports, NICE guidelines, the Health Technology Assessment Database (University of York), the Veterans Affairs Technology Assessment Program (VATAP) and Evidence-based Synthesis Program (ESP) reports, National Institutes of Health Consensus Statements, clinicaltrials.gov, and guidelines.gov.

We reviewed our search strategy with the TEP and supplemented it as needed according to their recommendations. In addition, to attempt to avoid retrieval bias, we will manually search the reference lists of landmark studies and background articles on this topic to look for any relevant citations that might have been missed by electronic searches.

We will also conduct an updated literature search (of the same databases searched initially) concurrent with the peer review process. Any literature suggested by peer reviewers or public comment respondents will be investigated and, if appropriate, incorporated into the final review. Appropriateness will be determined by the same methods listed above.

Table 2. Literature search terms
Population “anxiety disorders”[MeSH Terms] OR “anxiety disorders”[tiab] OR “anxiety disorder”[tiab] OR “anxiety”[MeSH Terms] OR “depressive disorder”[MeSH Terms] OR “depressive disorder”[tiab] OR “depression”[MeSH Terms]


AND at least one of the following:


“Arthritis” [MeSH Terms]


(“Diabetes Mellitus” [MeSH Terms] OR “diabetes mellitus”[tiab])


(chronic[tiab] AND “pain”[MeSH Terms]) OR “chronic pain”[tiab]


“cancer”[tiab]


“asthma”[MeSH Terms] OR “asthma”[tiab] OR “pulmonary disease, chronic obstructive”[MeSH Terms] OR “chronic obstructive pulmonary disease”[tiab] OR “COPD”[tiab]


“Stroke” [MeSH Terms]


“HIV” [MeSH Terms]


“Heart Failure” [MeSH Terms] OR “heart failure”[tiab]


“Myocardial Ischemia” [MeSH Terms] OR “myocardial ischemia”[tiab] OR “myocardial ischaemia”[tiab]


“coronary artery bypass”[tiab] OR “CABG”[tiab] OR “status post” AND myocardial


“status post” AND myocardial


“frail elderly”[MeSH Terms] OR “frail elderly”[All Fields]


complex patient*
Interventions At least one of the following:


“Intervention Studies”[MeSH Terms] OR intervention*[tiab]


“collaborative care”[tiab] OR “integrated treatment”[tiab] OR “clinical integration”[tiab] OR “integrated services”[tiab] OR “integrated care”[tiab] OR “integrated behavioral model”[tiab] OR “integrated health care”[tiab] OR “coordinated care”[tiab] OR (“integrated”[tiab] AND “behavioral model”[tiab]) OR “service coordination” OR “chronic disease management”
Limits Humans


English language


AND at least one of the following:


Clinical Trial; Meta-Analysis; Randomized Controlled Trial; Review; Clinical Trial, Phase I; Clinical Trial, Phase II; Clinical Trial, Phase III; Clinical Trial, Phase IV; Comparative Study; Controlled Clinical Trial; Multicenter Study; (“Randomized Controlled Trial”[Publication Type]; “Randomized Controlled Trials as Topic”[MeSH]); “Single-Blind Method”[MeSH]; “Double-Blind Method”[MeSH]; “Random Allocation”[MeSH]; “meta-analysis”[Publication Type]; “meta-analysis as topic”[MeSH Terms]; “meta-analysis”[All Fields]; “review”[Publication Type]; “review literature as topic”[MeSH Terms]; “systematic review”[All Fields]; “Comparative Study”[Publication Type]

C. Data Abstraction and Data Management

All titles and abstracts identified through searches will be independently reviewed for eligibility against our inclusion/exclusion criteria by two trained members of the research team. Studies marked for possible inclusion by either reviewer will undergo a full-text review. For studies without adequate information to determine inclusion or exclusion, we will retrieve the full text and then make the determination. All results will be tracked in an EndNote® database.

We will retrieve and review the full text of all articles included during the title/abstract review phase. Each full-text article will be independently reviewed by two trained members of the research team for inclusion or exclusion based on the eligibility criteria described above. If both reviewers agree that a study does not meet the eligibility criteria, the study will be excluded. If the reviewers disagree, conflicts will be resolved by discussion and consensus or by consulting a third member of the review team. As described above, all results will be tracked in an EndNote database. We will record the reason why each excluded full-text publication did not satisfy the eligibility criteria so that we can later compile a comprehensive list of such studies.

For studies that meet inclusion criteria, we will abstract important information into evidence tables. We will design data abstraction forms to gather pertinent information from each article, including characteristics of study populations, settings, interventions, comparators, study designs, methods, and results. Trained reviewers will extract the relevant data from each included article into the evidence tables. All data abstractions will be reviewed for completeness and accuracy by a second member of the team.

D. Assessment of Methodological Quality of Individual Studies

To assess the quality (internal validity) of studies, we will use predefined criteria based on those presented in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.34 In general, a “good” study has a strong design with reduced risk of bias, measures outcomes appropriately, uses appropriate statistical and analytical methods, reports low attrition, and reports methods and outcomes clearly and precisely. “Fair” studies are those that do not meet all criteria required for “good” quality but do not have flaws that are likely to cause major bias. Missing information often leads to ratings of “fair” as opposed to “good.” Studies of “poor” quality are those with at least one major flaw that is likely to cause significant bias. Examples of such major flaws include errors in design, analysis, or reporting; large amounts of missing information; and discrepancies in reporting. Poor-quality studies will be considered for inclusion in this review only if we are unable to answer KQs with the available good- and fair-quality studies. We will perform sensitivity analyses for all quantitative evaluations in order to assess the effects of omitting poor-quality studies and discuss the potential consequences thereof.

Two independent reviewers will assign quality ratings for each study. Disagreements between the two reviewers will be resolved by discussion and consensus or by consulting a third member of the team.

E. Data Synthesis

Prioritization or categorization of outcomes, or both, was determined by the research team with input from TEP members. If we find three or more similar studies for a comparison of interest, we will consider quantitative analysis (i.e., meta-analysis) of the data from those studies.

To determine whether quantitative analyses are appropriate, we will evaluate clinical heterogeneity using the PICOTS framework and following established guidance.34 We will consider similarities and differences across study populations in demographic factors, coexisting conditions, severity of depression or anxiety, study duration, and setting. We will evaluate the statistical heterogeneity of pooled analysis using the chi-squared statistic and the I2 statistic (the proportion of variation in study estimates due to heterogeneity).

When quantitative analyses are not appropriate (e.g., due to heterogeneity, insufficient numbers of similar studies, or insufficiency or variation in reporting), we will synthesize the data qualitatively.

We plan to stratify analyses and/or perform subgroup analyses when possible and appropriate. Planned stratifications or categories for subgroup analyses include mental health condition (depression vs. anxiety), chronic medical condition, age, and type of intervention.

F. Grading the Evidence for Each Key Question

We will grade the strength of evidence based on the guidance established for the Evidence-based Practice Center (EPC) Program.35 Developed to grade the overall strength of a body of evidence, this approach incorporates four key domains: risk of bias (including study design and aggregate quality), consistency, directness, and precision of the evidence. It also considers other optional domains that may be relevant for some scenarios, such as dose-response association, plausible confounding that would decrease the observed effect, strength of association (i.e., magnitude of effect), and publication bias.

Table 3 describes the grades of evidence that can be assigned. Grades reflect the strength of the body of evidence to answer the KQs on the comparative effectiveness, efficacy, and harms of the interventions in this review. Two reviewers will assess each domain for each key outcome, and differences will be resolved by consensus.

Table 3. Definitions of the grades of overall strength of evidence
Grade Definition
Source: Owens et al., 201035
High High confidence that the evidence reflects the true effect: Further research is very unlikely to change our confidence in the estimate of effect.
Moderate Moderate confidence that the evidence reflects the true effect: Further research may change our confidence in the estimate of the effect and may change the estimate.
Low Low confidence that the evidence reflects the true effect: Further research is likely to change our confidence in the estimate of the effect and is likely to change the estimate.
Insufficient Evidence either is unavailable or does not permit estimation of an effect.

We will grade the strength of evidence for the outcomes deemed to be of greatest importance to decisionmakers and those most commonly reported in the literature. We expect these to include intermediate outcome measures of depression/anxiety symptoms, intermediate chronic medical outcomes (e.g., HbA1c in diabetics), and health care utilization.

G. Assessing Applicability

We will assess the applicability of individual studies as well as the applicability of a body of evidence. For individual studies, we will examine conditions that may limit applicability based on the PICOTS structure. Such conditions may be associated with heterogeneity of treatment effect, measurement of absolute (rather than relative) benefits and harms, and the ability to generalize the effectiveness of an intervention to use in everyday practice. Examples include the following:

  • Population: narrow eligibility criteria
  • Intervention: intensity and delivery of interventions
  • Comparator: use of substandard comparators
  • Outcomes: use of composite outcomes that mix outcomes of different significance to patients
  • Timing: studies of different duration that may have various implications for applicability
  • Setting: standards of care that differ markedly from setting of interest (e.g., varying practice standards from country to country)

We will abstract and report key characteristics that may affect applicability into evidence tables. To assess the applicability of a body of evidence, we will consider the consistency of results across studies that represent an array of different populations. If the data allow, we will perform subgroup analyses to explore the influence of specific factors (e.g., age, race/ethnicity, gender). We will also describe the limitations of the aggregate evidence with regard to inclusion of relevant populations, interventions, comparisons, outcomes, and settings.

References

  1. World Health Organization. Integrating mental health into primary care - a global perspective. Geneva: World Health Organization; 2008. http://www.who.int/mental_health/policy/services/mentalhealthintoprimarycare/en/ Accessed on October 13, 2010.
  2. Sherbourne CD, Jackson CA, Meredith LS, et al. Prevalence of comorbid anxiety disorders in primary care outpatients. Arch Fam Med. 1996 Jan;5(1):27-34. PMID 8542051.
  3. Katon W, Unutzer J, Wells K, et al. Collaborative depression care: history, evolution and ways to enhance dissemination and sustainability. Gen Hosp Psychiatry. 2010 Sep-Oct;32(5):456-64. PMID 20851265.
  4. Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002 Dec;288(22):2836-45. PMID 12472325.
  5. Katon WJ, Von Korff M, Lin EH, et al. The Pathways Study: a randomized trial of collaborative care in patients with diabetes and depression. Arch Gen Psychiatry. 2004 Oct;61(10):1042-9. PMID 15466678.
  6. Bogner HR, Morales KH, Post EP, et al. Diabetes, depression, and death: a randomized controlled trial of a depression treatment program for older adults based in primary care (PROSPECT). Diabetes Care. 2007 Dec;30(12):3005-10. PMID 17717284.
  7. Ciechanowski PS, Russo JE, Katon WJ, et al. The association of patient relationship style and outcomes in collaborative care treatment for depression in patients with diabetes. Med Care. 2006 Mar;44(3):283-91. PMID 16501401.
  8. Bogner HR, de Vries HF. Integrating type 2 diabetes mellitus and depression treatment among African Americans: a randomized controlled pilot trial. Diabetes Educ. 2010 Mar-Apr;36(2):284-92. PMID 20040705.
  9. Unutzer J, Schoenbaum M, Druss BG, et al. Transforming mental health care at the interface with general medicine: report for the presidents commission. Psychiatr Serv. 2006 Jan;57(1):37-47. PMID 16399961.
  10. Schneider KM, O'Donnell BE, Dean D. Prevalence of multiple chronic conditions in the United States' Medicare population. Health Qual Life Outcomes. 2009;7:82. PMID 19737412.
  11. Katon WJ, Lin EH, Von Korff M, et al. Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010 Dec;363(27):2611-20. PMID 21190455.
  12. Kessler RC, Chiu WT, Demler O, et al. Prevalence, severity, and comorbidity of 12-Month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatr. 2005;62(6):617-27. PMID 15939839.
  13. World Health Organization. The global burden of disease: 2004 update. Geneva: World Health Organization; 2004. http://www.who.int/healthinfo/global_burden_disease/2004_report_update/en/index.html. Accessed on October 25, 2010.
  14. Schulberg HC, Katon WJ, Simon GE, et al. Best clinical practice: guidelines for managing major depression in primary medical care. J Clin Psychiatry. 1999;60 Suppl 7:19-26. PMID 10326871.
  15. National Institute for Health and Clinical Excellence. National Clinical Practice Guideline Number 91: Depression in Adults with a Chronic Physical Health Problem: treatment and management. London: National Institute for Health and Clinical Excellence; 2009. http://www.nice.org.uk/nicemedia/live/12327/45909/45909.pdf. Accessed on October 13, 2010.
  16. Culpepper L. Generalized anxiety disorder and medical illness. J Clin Psychiatry. 2009;70 Suppl 2:20-4. PMID 19371503.
  17. Ustun TB, Ayuso-Mateos JL, Chatterji S, et al. Global burden of depressive disorders in the year 2000. Br J Psychiatry. 2004 May;184:386-92. PMID 15123501.
  18. Stein MB, Cox BJ, Afifi TO, et al. Does co-morbid depressive illness magnify the impact of chronic physical illness? A population-based perspective. Psychol Med. 2006 May;36(5):587-96. PMID 16608557.
  19. Greenberg PE, Kessler RC, Birnbaum HG, et al. The economic burden of depression in the United States: how did it change between 1990 and 2000? J Clin Psychiatry. 2003 Dec;64(12):1465-75. PMID 14728109.
  20. Kessler RC, Greenberg PE. The economic burden of anxiety and stress disorders. In: Davis KL, Charney D, Coyle JT, Nemeroff C, eds. Neuropsychopharmacology: the fifth generation of progress. Philadelphia: Lippincott Williams & Wilkins; 2002:p. 981-92.
  21. National Heart L, and Blood Institute,. Morbidity & mortality: 2009 chart book on cardiovascular, lung, and blood diseases. Bethesda, MD: National Institutes of Health; 2009. http://www.nhlbi.nih.gov/resources/docs/2009_ChartBook.pdf. Accessed on January 11, 2011.
  22. Centers for Disease Control and Prevention. National diabetes fact sheet, 2007. Atlanta, GA: Centers for Disease Control and Prevention; 2008. http://www.cdc.gov/diabetes/pubs/factsheet07.htm. Accessed on January 11, 2011.
  23. Stewart WF, Ricci JA, Chee E, et al. Lost productive time and cost due to common pain conditions in the US workforce. JAMA. 2003 Nov;290(18):2443-54. PMID 14612481.
  24. Birnbaum H, Pike C, Kaufman R, et al. Societal cost of rheumatoid arthritis patients in the US. Curr Med Res Opin. 2010 Jan;26(1):77-90. PMID 19908947.
  25. Kotlarz H, Gunnarsson CL, Fang H, et al. Osteoarthritis and absenteeism costs: evidence from US National Survey Data. J Occup Environ Med. 2010 Mar;52(3):263-8. PMID 20190656.
  26. Wolf NJ, Hopko DR. Psychosocial and pharmacological interventions for depressed adults in primary care: a critical review. Clin Psychol Rev. 2008 Jan;28(1):131-61. PMID 17555857.
  27. Butler M, Kane RL, McAlpine D, et al. Integration of Mental Health/Substance Abuse and Primary Care. Evidence Report/Technology Assessment No. 173 (Prepared by the Minnesota Evidence-based Practice Center under Contract No. 290-02-0009). Rockville, MD: Agency for Healthcare Research and Quality; October, 2008. AHRQ Publication No. 09-E003.
  28. Roy-Byrne P, Craske MG, Sullivan G, et al. Delivery of evidence-based treatment for multiple anxiety disorders in primary care: a randomized controlled trial. JAMA. 2010 May 19;303(19):1921-8. PMID 20483968.
  29. Kroenke K, Bair MJ, Damush TM, et al. Optimized antidepressant therapy and pain self-management in primary care patients with depression and musculoskeletal pain: a randomized controlled trial. JAMA. 2009 May;301(20):2099-110. PMID 19470987.
  30. Agency for Healthcare Research and Quality Web site. List of Priority Conditions for Research under Medicare Modernization Act Released. Press Release, December 15, 2004. http://www.ahrq.gov/news/press/pr2004/mmapr.htm. Accessed on September 16, 2011.
  31. Adams K, Corrigan JM, eds. Priority areas for national action: transforming health care quality. Washington, DC: National Academies Press; 2003.
  32. Norris S, Atkins D, Bruening W, et al. Selecting observational studies for comparing medical interventions. Methods Guide for Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; Posted June 2010. Available at /sites/default/files/pdf/methods-guidance-observational-studies_methods.pdf.
  33. Jonas D, Viswanathan M, Crotty K. Selecting Evidence for Comparative Effectiveness Reviews: When To Use Observational Studies. Presentation prepared for the Agency for Healthcare Research and Quality; 2009. www.ahrq.gov/about/annualconf09/viswanathan2/viswanathan2.ppt. Accessed on September 15, 2011.
  34. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville: MD: Agency for Healthcare Research and Quality; March 2011. AHRQ Publication No. 10(11)-EHC063-EF. Chapters available at: http://www.effectivehealthcare.ahrq.gov. Accessed on September 15, 2011.
  35. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions--Agency for Healthcare Research and Quality and the Effective Health Care Program. J Clin Epidemiol. 2010 May;63(5):513-23. PMID 19595577.

Definition of Terms

The term “practice-based intervention” is used many different ways in the health literature, but our working definition for the purposes of this review is “any intervention that targets the care process within a system of care.” Practice-based interventions can also include person-level components.†


†An example of a “person-level” component is a particular psychotherapy, such as problem-solving therapy — that while part of a broader practice-based intervention, (e.g., coordinated care) would be delivered directly to the patient by a trained practitioner in the primary care setting; another example is antidepressant medication, as part of the larger intervention, but delivered at the patient level. While we are selecting for practice-based interventions that target an overall process of care, we will not be excluding components of such interventions because they occur at the level of the patient.

Summary of Protocol Amendments

In the event of protocol amendments, the date of each amendment will be accompanied by a description of the change and the rationale for it.

Review of Key Questions

For all EPC reviews, Key Questions are reviewed and refined as needed by the EPC with input from Key Informants and the Technical Expert Panel to assure that the questions are specific and explicit about what information is being reviewed. In addition, for comparative effectiveness reviews, the key questions are posted for public comment and finalized by the EPC after review of the comments.

Key Informants

Key Informants are the end users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the role of Key Informants is to provide input into identifying the Key Questions for research that will inform health care decisions. The EPC solicits input from Key Informants when developing questions for systematic review and when identifying high-priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.

Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Technical Experts

Technical Experts comprise a multidisciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, and outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore, study questions, design, and/or methodological approaches do not necessarily represent the views of individual Technical Experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not conduct analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.

Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts, and those who present potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.

Peer Reviewers

Peer Reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. Peer Reviewer comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer Reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual Peer Reviewers. The dispositions of the Peer Reviewer comments are documented and will, for comparative effectiveness reviews and Technical Briefs, be published 3 months after the publication of the Evidence Report.

Potential Peer Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer Reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.