1. What is the decision or change (e.g. clinical topic, practice guideline, system design, delivery of care) you are facing or struggling with where a summary of the evidence would be helpful?

This proposal is to conduct an integrative meta-analysis of the change processes research in individual psychotherapy treatment for adults with depression to be used as part of updating the American Psychological Association’s (APA, 2019) Clinical practice guideline for the treatment of depression across three age cohorts. This type of review (sometimes given other names such as mixed methods systematic reviews or critical interpretive syntheses; Dixon-Woods et al., 2001; Stern et al., 2021) is an innovative approach to conducting systematic reviews that synthesizes the qualitative literature and the quantitative literature on change processes. The qualitative change process literature is centered upon the analysis of patients’ reports on their therapy experiences and provides a description of how change unfolds in sessions – considering both their therapists’ and their own contributions. Quantitative literature provides evidence on whether these change processes appear reliably effective.

The inclusion of an integrative meta-analysis in clinical practice guidelines (CPGs) would increase patient centeredness of the CPG for the treatment of depression by providing guidance to clinicians on the client’s personal and cultural preferences, clinical utility, and implementation considerations across diverse populations. While experimental methods can identify whether treatments produce change effectively, philosophers of science and methodologists have identified qualitative research methods as adept at identifying and articulating causal processes as they occur in clients’ subjective experiences and their interpersonal and cultural contexts (e.g., Cartwright & Hardie, 2012; Maxwell & Levitt, 2023). Specifically, within the literature on individual psychotherapy for adults with depression, the review would examine the three research questions listed below. The production of integrated evidence in response to these three questions will be used to guide therapists in both their selection and application of clinical treatments of depression. They will be disseminated with the APA clinical practice guideline for depression to maximize their impact in improving mental and behavioral health. This will be a landmark shift in the way that clinical practice guidelines are developed, providing clinicians with information on how to best engage with clients with depression, based upon their own descriptions of their needs, preferences, and potential for contributing to the change process.

Specific Research Questions:

1. What are the change processes that unfold within the treatment of depression and how do they influence micro-level outcomes (i.e., in-treatment session-level change) and macro-level outcomes (i.e., post-treatment change including follow-up)?
2. What are the patient preferences/values/experiences that effect their engagement in the treatment of depression and how do they influence micro- and macro-outcomes?
3. What are clinical decisions that therapists report making in sessions to personalize the treatment of depression to clients and how do they influence micro- and macro- outcomes? All three questions will be examined in relation to the varied personal characteristics (e.g., socioeconomic status) and cultural contexts (e.g., gender, race) of clients and therapists.

Methodological Summary:

We propose an integrative meta-analysis of change processes in individual psychotherapy treatment for adults with depression (inclusive of quantitative and qualitative research). We propose that the qualitative aspect of the meta-analytic method (sometimes called meta-synthesis) be structured as follows, based upon a critical-constructivist grounded theory method which examines both meaning-making and cultural processes related to a phenomenon (Glaser & Strauss, 1967; Charmaz, 2009; Levitt, 2021):

1. The identification of qualitative and quantitative meta-analyses and primary research articles will follow PRISMA 2020 and PRISMA-S guidelines (Page et al., 2021a; Page et al., 2021b; Rethlefsen et al., 2021). Articles selected may include mixed methods studies as well.
2. A review of findings from the meta-analytic and primary studies will be conducted in which a unit will be generated to reflect each of the primary study findings. Each unit will be assigned a succinct label that reflects its content in response to one of the three questions previously indicated.
3. These units will be sorted then into the three domains as indicated by the three specific research questions listed above (i.e., change processes, patient preferences/values/experiences, therapists’ clinical decisions). Units can be assigned to multiple domains as relevant.
4. Within each domain, units will be organized into initial categories that reflect patterns of commonalities and difference within them. Then, these initial categories will be organized into higher-order categories to identify the causal processes that unfold within the three domains related to the treatment of depression.
5. A report on these categories, including illustrative quotations from the literature, will be developed. The reporting of the findings will be organized according to the identification of change processes in treating depression. For each change process (Specific Research Question 1), relevant patient preferences/values/experiences (Specific Research Question 2), and therapist decision-making guidance (Specific Research Question 3) will be identified. The report will generate principles for practice (see Levitt et al., 2006 for a description of how to develop principles for practice from qualitative research) that inform both treatment selection and application and that are sensitive to the cultural and personal characteristics of patients and in-session dynamics.

This project will advance the methodologies used in clinical practice guideline development, using innovative methods that increase the scope of guidance. It will produce information on the treatment of adult depression that will inform clinicians across theoretical orientations – increasing the relevance of clinical practice guidelines in the field. Also, it will provide guidance that other international and professional guideline developers can use in shaping guidelines that are informed by the change process literature on patient experiences, preferences, values, and cultures as well as the personalization of treatment to patients. It will amplify the voices of clients about their treatment experiences and clarify personalization processes that therapists use.

The integrative meta-analysis that we are nominating will allow for the integration of the qualitative and quantitative findings on change processes, which can inform the treatment efficacy and comparative effectiveness guidance that will be developed by APA’s clinical practice guideline update panel on depression. For instance, if we found that certain change processes were experienced by clients and therapists as beneficial, we would consider how they are reflected in treatments that are found to be efficacious or in promising treatments that use those change processes but have yet to be evaluated using RCT studies, possibly recommending future research in that area. The integration of the findings from this review will allow for an innovative update of APA’s (2019) clinical practice guideline on depression that identifies not only which therapies have evidence of effectiveness or efficacy but also how therapies lead to change and how they should be applied and personalized in relation to clients, providing new forms of guidance to bolster responsive clinical practice.

Population of interest:

While the original APA depression guideline included children through older adults, we propose that this integrative meta-analysis focuses on adults and older adults with depression. We are proposing the focus on adults/older adults and not children/adolescents for three reasons. First, this is the first time in APA’s clinical practice guideline initiative that an integrative meta-analysis inclusive of qualitative literature would be used and focusing on this smaller though still substantial portion of the population would allow us to start smaller and build from there. Second, there is a larger body of qualitative body of literature for this population than for children/adolescents, which would provide greater confidence in the resultant findings. Third, the Advisory Steering Committee for development of clinical practice guidelines is considering focusing the overall guideline update on adults/older adults and separating out children/adolescents where findings seem to have been more stable.

As part of this project, we would recruit a group of experts composed of stakeholders in therapeutic treatment of depression (e.g., clients with a history of depression, therapists, researchers) to provide any feedback to the systematic review team as needed throughout the project. This group could be consulted at various points in the research and could engage with the team in developing ideas about the interpretations of the findings, on dissemination efforts, and on the reporting of findings. Engaging a participatory approach in this process will help to ensure that findings are developed, disseminated, and written in ways that are effective and accessible for patients who experience depression and their therapists.

Interventions/comparators:

We propose that this review focus on individual psychotherapy for adults with depression and specifically on change processes. Change processes are defined as: (1) change principles, which refer to conditions or characteristics (e.g., participant, relationship, and treatment components) that have been shown to predict treat¬ment outcome (see American Psychological Association Presidential Task Force on Evidence-Based Practice, 2006; Castonguay et al., 2019; see also Norcross & Lambert, 2019); (2) change mechanisms, which refer to factors that lead to therapeutic change as demonstrated by mediational or related analyses (Kazdin, 2007; Laurenceau et al., 2007; Lorenzo-Luaces et al., 2015); and (3) change events, which refer to in-session interactions between patients and therapists that are linked to overall treatment outcomes (Greenberg, 1986, 2007; Pascual-Leone et al., 2009).

In the APA 2019 CPG for depression, an unsystematic review of the change process literature (provided by a panel of experts) was conducted and implicated the following processes.

They are presented here as an illustration for our proposal:

Change Principles

See Beutler et al. (2006) for the following examples:

Participant. Patient expectations; readiness to change; and attachment, coping, and personality styles (Beutler et al., 2006; see also Bernecker, 2012).

Technical. Cognitive reappraisals, trying new behaviors, altering behavioral reinforcements, improving interpersonal and social functioning, promoting emotion processing and regulation, and conducting a structured therapy session (Follette & Greenberg, 2006; see also Auszra et al., 2013; Missirlian et al., 2005; Pos et al., 2003; Whelton, 2004).

Relationship. Patient-therapist alliance (Cuijpers et al., 2012), as well as empathy, consensus, positive regard, rupture repair and management of the therapist emotional reactions (Norcross & Wampold, 2019).

Change Mechanisms

Here are examples resulting from mediational analyses:

Lemmens and colleagues (2016) conducted a system¬atic review of 35 studies aimed at identifying mediators in various forms of psychotherapy for depression. They found that change in dysfunctional attitudes, negative “automatic” thoughts, mindfulness and worry skills, and rumination mediated change in outcomes (e.g., depression symptoms) in most studies, and they found mixed evidence supporting mediational roles for attributional style, behavioral components, and therapeutic alliance.

A review by Lorenzo-Luaces et al. (2015) reported some evidence that cognitive change (including both ability to use skills to modify cognitions and changes in cognitions, such as automatic thoughts and beliefs) mediates symptom change in cognitive therapy and mindfulness-based cognitive therapy for depression, as predicted by the theoretical models underpinning those therapies. A review by van der Velden et al. (2015) reported that changes in mindfulness, rumination, worry, compassion, or meta-awareness mediated effects of mindfulness-based cognitive therapy for depression. In addition, Cohen and colleagues (2014) reported that among women who received brief couple therapy for depression, changes in depressive symptoms were mediated by changes in their own illness-related cognitions and behaviors, and in their perceptions of increased positivity and support from their husbands. There is also some evidence that skills learning and practice during homework assignments contribute to better outcomes for cognitive, behavioral, and cognitive-behavioral therapies for depression (Kazantzis et al., 2010; Terides et al., 2018).

Research has provided varying sup¬port for the following variables facilitating better outcomes in the dynamic treatment of depression: greater insight, change in defensive style (more mature), less relationship rigidity and maladaptive representations, higher object relations, greater reflective functioning, and better therapeutic alliances (see Barber et al., 2013, for a review; see also Minges et al., 2017; Zilcha-Mano, Chui, et al., 2016a; & Zilcha-Mano, Muran, et al., 2016b). Research on change mechanisms in humanistic-experiential therapies has shown that emotional processing facilitates better outcomes (Auszra et al., 2013; Pos et al., 2009; Pos et al., 2003). Finally, changes produced by antidepressant treatments are extremely complex and our understanding of those changes is evolving (Dell’Osso et al., 2011).

Change Events

Here are examples of research on in-session change events involving mixed methods (quantitative and qualitative).

These examples include several intensive (small-scale, mixed method) studies in which patient and therapist interactions have been reliably observed, and these interaction features are found to be related to treatment out¬comes for depression (Greenberg, 1986; Greenberg & Newman, 1996). Examples include resolving problematic reactions (i.e., the reexperiencing of a problematic situation to better symbolize it in awareness and understand its personal significance; Watson, 1996), unfinished business (i.e., exploring lingering negative feelings toward a significant other; (Greenberg & Malcolm, 2002), and alliance rup¬tures (i.e., understanding and overcoming difficulties in the patient–therapist relationship; Safran & Muran, 1996). This is an area of research that requires more investment as it can provide clinicians with more detailed information on how to interact with their patients.

Outcomes:

Evidence generated from this topic will help end users to better understand how and why a treatment works allowing providers to better tailor treatment to individual patients’ diverse characteristics. This integrated review will examine both micro-outcomes, which are in-session outcomes that facilitate change in depressogenic thinking, feeling, relating, and behaving and micro-outcomes that examine post-therapy changes. Some of these outcomes will be identified in the process of identifying articles and conducting the analysis (because qualitative analyses engage inductive analysis). From our review, however, micro-outcomes will include processes such as the development of new insight, the differentiation of emotional experience, the evolution of relational connection, and clients’ identification and resolution of problematic patterns. Macro-outcomes will include shifts in the therapeutic alliance, symptom reduction, interpersonal problems, and emotional regulation).

2. Why are you struggling with this issue?

Typically, clinical practice guidelines provide information to patients and clinicians on whether a treatment works (efficacy) and as available, which treatment works compared to other treatments (comparative effectiveness). However, they are often lacking in information that is important to patients and clinicians, such as what works, for whom, and under what circumstances and what is known about change processes or principles underlying the effects of efficacious treatments (Castonguay et al., 2019; Muran & Lipner, 2023; Norcross & Lambert, 2019).

Striving to provide this information would greatly enhance the clinical utility of a clinical practice guideline for the end user. Understanding how treatments work is critical to improve care, particularly when expected treatment gains are not made or in cases with significant comorbidity. Yet, quantitative literature (e.g., RCTs included in typical systematic reviews) on causal processes within treatment application is lacking (Castonguay et al., 2019; Muran & Lipner, 2023; Norcross & Lambert, 2019). Turning to the qualitative psychotherapy research literature allows us to explore the how and why a treatment is working or not and why changes may or may not be occurring from therapists’ as well as clients’ perspectives. Using qualitative methods allows for exploration of a wide swath of possible change processes, often revealing those unfamiliar to therapists and researchers, because these methods are driven by inductive inferences grounded in patient experiences rather than deductive inferences based upon researchers’ a priori hypotheses.

Though quantitative meta-analyses have a long history in psychology (Smith & Glass, 1977) qualitative meta-analyses are newer and their use has grown alongside the growth of qualitative methods (Kovács et al., 2019; Levitt et al., 2017). This growth has been reflected in the establishment of multiple national qualitative societies as well as in the most frequently downloaded APA journal article in 2018, the inaugural APA Journal Article Reporting Standards for qualitative primary research, mixed methods research, and qualitative meta-analysis (Levitt et al., 2018). The significant growth in qualitative literature now permits the synthesis of the findings of this body of literature and the ability to reconcile these findings together with quantitative meta-analytic findings. Integrative meta-analyses are cutting-edge methods in psychology that combine mixed methods research and qualitative meta-analytic methods. An integrative meta-analysis including qualitative literature allows for the aggregation and synthesis of information from qualitative and quantitative primary studies or meta-analytic data (Levitt & Morrill, 2023). Supporting the introduction of these methods in psychology will allow for stronger clinical guidance, that reflects not only treatment efficacy and comparative effectiveness, but change processes, clients’ preferences, values and cultural characteristics, and therapists’ clinical decision-making. The use of this approach within the development of guidelines for adult depression will serve as a pilot evaluation to determine if this process can be used going forward with other sets of clinical guidelines.

Internationally, there has been growing interest in incorporating qualitative research in guideline development processes from both stakeholders and guideline development bodies. The pre-proof article by Levitt and colleagues (2024; for internal use only--please refer to the email following this nomination to view the article) is the product of an international collaboration of 19 qualitative psychotherapy researchers from 10 countries who articulated the benefits of including qualitative findings within the guideline development process and generated recommendations for guideline developers. These recommendations provide methodological advice to be used by guideline developers when selecting to incorporate qualitative research in the implementation of an expanded guideline development process and can be considered and used to inform our process. The innovative approach can also inform the revised evidence-based medicine framework proposed by Aldous and colleagues (2024) to further represent the totality of evidence when it comes to medical decision-making.

In summary, adding an integrative meta-analysis that includes qualitative literature to the update of APA’s depression guideline will allow end-users of the guideline to (1) better individualize therapy to patients and their individual and cultural characteristics (e.g., socioeconomic status, gender, race), (2) broaden the implementation potential of the guideline, (3) better examine both the outcome as well as the causal processes that unfold in-session for depression treatment, (4) providing guidance for clinical decision-making regarding when, why, and how (particularly important in cases of small/moderate effect sizes), and (5) increase the comprehensiveness and general usefulness of the guideline. This would also serve to produce an individualizable Patient-Centered PCORI Evidence Update that could be disseminated together with APA’s depression guideline. It would create a model for clinical guideline producers to follow, both internationally and within the US.

3. What do you want to see changed? How will you know that your issue is improving or has been addressed?

We would like to see and produce guidance on depression treatment that is more user-friendly for providers and patients and better individualized to patients and their and cultural characteristics (e.g., socioeconomic status, gender, race). Overall, we would see increased clinical utility of the depression clinical practice guideline because it will address a much broader range of micro-outcomes and macro-outcomes and see increased dissemination and implementation of the guideline across diverse populations, as advised by the advisory group of stakeholders tied to the psychotherapeutic treatment of adult depression.

Further, we hope that this innovative methodology will spur future research and innovative methods in guideline development and systematic reviews to better individualize treatment considering what works and for whom going forward.

We have some flexibility with the timing of the report and can coordinate forming a panel to coincide with report completion. Ideally, we’d hope to have the evidence report within approximately one to one and a half years.

4. When do you need the evidence report?

Wed, 10/01/2025

5. What will you do with the evidence report?

The U.S. has been seeing a mental health crisis unlike any it has seen before. Contributing in part to this crisis, Americans have been facing significant stressors such as the recent COVID-19 pandemic, racial injustice, a divisive political environment, and more. APA’s Communications team noted increased traffic to the PTSD guideline website during the COVID-19 crisis (https://www.apa.org/ptsd-guideline). Between April 1, 2020, and May 3, 2022, there were 3,958,871 pageviews, by 2,750,593 users, with 86% of these users being “new users” to the website. Since going live in July of 2017, the PTSD guideline website had 5,561,352 pageviews (as of May 3, 2022). Between May 2022 and May 2023, the PTSD CPG website had 115,499 pageviews and of those pageviews 85% of them were new users. A 2022 survey by APA found that 76% of adults reported health impacts from significant stress in the past month, including 33% reporting feeling depressed or sad (American Psychological Association, 2022). Further, the World Health Organization reported a 25% increase in prevalence for depression and anxiety globally in the first year of the COVID-19 pandemic (World Health Organization, 2022). These stats are particularly dire for diverse populations with rates especially pronounced in populations that are marginalized as well as from lower socio-economic status backgrounds (Weinberger et al., 2018). Clearly, information to help providers tailor treatments for a diverse range of individuals is critical.

We plan to update APA’s (2019) Clinical Practice Guideline for the Treatment of Depression Across Three Age Cohorts. However, while existing systematic reviews and meta-analyses are available with quantitative information on efficacy and comparative effectiveness of depression treatments, the integration of qualitative meta-analysis data and quantitative data in the study of change processes is lacking. Given the growing diversity of the U.S. population and beyond, it is imperative for end users to understand how and why treatments work as they implement the guideline with diverse populations. This information would greatly enhance the clinical utility of the guideline and subsequently its ability to help a larger portion of the population. Given the growing rates of depression across diverse populations, particular with the recent pandemic, it is critical to ensure this guideline is useful for a diverse range of individuals.

An integrative meta-analysis/systematic review by AHRQ on change processes would be instrumental to updating this guideline and providing this information to the end-users, and it would be one of the first reviews of its kind in this area. This updated guideline will of course continue to provide efficacy and comparative effectiveness recommendations for treating depression based on existing systematic reviews and meta-analyses of the empirical literature. It will also include recommendations for future research in this area and implementation considerations for the recommendations such as equity, diversity, and inclusion and anti-racism issues. It would help to address and improve mental health care across a range of diverse individuals.

If funded, the integration of quantitative and qualitative literature would serve complementary functions for the guideline. For example, both types give researchers the opportunity to detect the key variables across the group of studies (Elliott et al., 2018), summarize which research questions are being addressed or not in the field (Tao et al., 2015), and provide a critical review of the field’s methodologies (Kovács et al., 2019;). Strengths of quantitative meta-analyses include identification of whether a change process is helpful, (Zhang et al., 2022), identification of outcomes and processes (Pascual-Leone & Yeryomenko, 2017), and identification of mediation and moderation change factors (Del Re et al., 2021; Spielmans & Flückiger, 2018). Strengths of qualitative methods include the development of treatment application guidance (Marren et al., 2022), facilitating better understanding of the decision-making process used by therapists (Hill et al., 2018), and facilitating better understanding of the outcomes and processes of patients in the session (Lavik et al., 2018).

In summary, for the update of APA’s depression guideline, we are nominating an integrative meta-analysis on change process that would combine these strengths. It would be focused within an updated review of the meta-analytic literature on depression. Integrating these two types of literature will allow for a deeper understanding of depression research literature and provide clinical guidance for both the selection, application, and personalization of treatments (Levitt, 2018). Results from this report will be disseminated to our organization’s membership which includes nearly 157,000 researchers, educators, clinicians, consultants, and students. Once the clinical practice guideline on depression is updated, it will be placed on APA’s website as well as submitted to the ECRI Guidelines Trust and Guidelines International Network’s guideline repositories.

Supporting Documentation

References AHRQ Topic Submission Depression (Word, 46 KB)

Optional Information About You

What is your role or perspective? Professional society

If you are you making a suggestion on behalf of an organization, please state the name of the organization American Psychological Association

May we contact you if we have questions about your nomination? Yes