On January 10, 2012, several amendments were made to this protocol. To view these amendments, please see the section titled “Summary of Protocol Amendments.”
Background and Objectives for the Systematic Review
The topic of this review, as originally nominated, was focused on the effectiveness of parent- or caregiver-mediated interventions with foster or adoptive parents in improving child mental health, reducing child problem behaviors (such as delinquency and other types of maladaptive behavior), and preventing negative life outcomes. The nominator’s particular interest was in interventions that support the empathic attunementof caregivers to the emotional needs of the children in their care. A preliminary search of the peer-reviewed literature germane to this topic (i.e., intervention studies focused on improving caregiver sensitivity, nurturing behaviors, and other dimensions of a healthy caregiver-child relationship with foster or adoptive parents) yielded numerous studies representing marked clinical heterogeneity. Through the topic development and refinement process, the decision was made to focus broadly on psychosocial interventions for children exposedto trauma in the form of maltreatment or family (i.e., domestic) violence. The decision was also made not to limit this review to only caregiver-mediated interventions or interventions with children in foster care or who had been adopted, to ensure that the full range of interventions and caregiving contexts relevant to the population would be included in this review.
Another important factor affected the scope of this comparative effectiveness review (CER) such that both psychosocial interventions and pharmacotherapy will be included in it. During the period of topic refinement, the Agency for Healthcare Research and Quality (AHRQ) received a new, related topic nomination on the effectiveness of interventions for children with post-traumatic stress disorder (PTSD). In consideration of the limitations of the child PTSD diagnosis and the overlap between the topic of child PTSD and the first review, the decision was made to coordinate the two CERs as a two-part series focused on the comparative effectiveness of interventions for children exposed to trauma. The first review in the series will focus on the comparative effectiveness of interventions that address child exposure to familial trauma in the form of maltreatment or family violence (including PTSD as an outcome of interest). The second review will focus on the comparative effectiveness of interventions that address child exposure to traumatic experiences other than maltreatment or family violence, including terrorism, war, refugee status, natural disasters, fire, motor vehicle and other accidents, medical trauma, community or school violence, parent separation or divorce, dating violence, and death of a loved one.
An extensive body of research in the behavioral, social science, and neurobiological sciences has demonstrated the association between child experiences of maltreatment or family violence and negative, often severe, mental and behavioral problems across the developmental continuum.1-5 The term child maltreatment is defined variously in the scientific literature and across the many health and human services sectors that address the issue.6 For the purpose of this review, we based our definition of child maltreatment on two core resources. The first is a recent report produced by the Centers for Disease Control and Prevention (CDC) that proposes uniform definitions for improved public health surveillance of child maltreatment.6 The CDC report defines maltreatment as any act or series of acts of commission or omission (specified further below) by a parent (custodial and noncustodial parents) or other caregiver that results in harm, potential for harm, or threat of harm to a child:
- Child abuse (act of commission)
Defined as words or overt actions that cause harm, potential harm, or threat of harm to a child. Acts of commission are deliberate and intentional; however, harm to a child may or may not be the intended consequence. This includes physical abuse, sexual abuse, and psychological abuse.
- Child neglect (act of omission)
Defined as failure to provide for a child’s basic physical, emotional, or educational needs or to protect a child from harm or potential harm. This includes physical neglect, emotional neglect, medical/dental neglect, educational neglect, inadequate supervision, and exposure to violent environments.
The term family violence refers to violence in the home, including intimate partner violence (i.e., violence between the child’s caregivers) and violence between the caregiver and another child in the family. In the CDC resource upon which we are basing our definition of maltreatment, family violence is differentiated from maltreatment (i.e., in the form of failure to protect the child from exposure to violent environments) because the child’s primary caregiver is being victimized and may not be aware of or have alternatives to protect his or her children.
To ensure the comprehensiveness of this review, we had to expand the definition of sexual abuse by using the broader language provided in the Federal Child Abuse Prevention and Treatment Act (CAPTA) Reauthorization Act of 2010 (Public Law 111-320).7 The CAPTA definition more accurately aligns with the scientific literature in that intervention studies a) do not restrict inclusion by type of perpetrator and b) are commonly directed at the nonperpetrating primary caregiver to address issues such as causal attributions regarding the abuse, negative perceptions of the child, appropriate emotional support for the child, and management of sexually inappropriate child behavior.8
- Sexual Abuse
The employment, use, persuasion, inducement, enticement, or coercion of any child to engage in, or assist any other person to engage in, any sexually explicit conduct or simulation of such conduct for the purpose of producing a visual depiction of such conduct; or the rape, and in cases of caretaker or interfamilial relationships, statutory rape, molestation, prostitution, or other form of sexual exploitation of children, or incest with children.
Children’s exposure to maltreatment or family violence is associated with long-term mental health and behavioral outcomes, including severe emotional and behavioral disturbance, substance abuse, high-risk sexual behaviors, aggression and violent crime, and dysfunctional parenting.9-13 A growing body of psychobiological research in this area suggests that chronic exposure to stressful and arousing events is associated with dysregulation of the hypothalamic-pituitary-adrenal (HPA) axis. Activation of the HPA axis releases a cascade of steroid hormones, including the primary stress hormone cortisol, which stimulate immune, metabolic, circulatory, and other bodily system responses. Dysregulation of the HPA axis also affects key regions of the brain, including the areas responsible for executive functioning (prefrontal cortex), emotional responses (amygdala), and short-term memory (hippocampus).14-20
Child maltreatment and family violence are seldom single-incident events. Instead, these forms of familial trauma tend to occur repeatedly over long periods of time and typically take multiple forms. When a maltreated child or child exposed to family violence is removed from the home, the psychological harm of the toxic exposure is further aggravated by the separation from and loss of the child’s attachment relationship with his or her primary caregiver. In turn, the initial traumatizing experience of maltreatment is compounded by the exposure and removal from the home, which sets off a trajectory of cumulative harm when children in foster care experience multiple placement changes and repeated attachment disruption.21 This phenomenon of children’s exposure to multiple or prolonged traumatic events within the primary caregiving system is referred to as complex trauma22, 23 and is recognized as having a profoundly negative impact on fundamental developmental processes throughout childhood:
- Complex trauma
Complex trauma is a subset of the full range of psychological trauma that has as its unique trademark a compromise of the individual’s self-development. The timing of its occurrence—in critical windows of development during childhood, when self-definition and self-regulation are being formed and consolidated—and its very nature—the disruption or distortion of fundamental attachment security due to betrayal of the developing child’s security and trust in core relationships—distinguish complex trauma from all other forms of psychological trauma.24
It is important to note that young children, in the context of rapid development and maturation, seldom meet the usual criteria for PTSD or other mental health or behavioral disorders. Rather, they often exhibit symptom clusters that can be characterized in terms of alternate diagnostic systems (e.g., 0–3 Diagnostic Classification25) that address developmentally specific clinical presentations and are precursors to later poor outcomes throughout childhood and adolescence and into adulthood. Recently, a Developmental Trauma Disorders Task Force of the National Child Traumatic Stress Network has begun to conceptualize a new diagnosis, developmental trauma disorder. This diagnosis may address with greater precision the developmental, psychological, biological, and social factors that serve as both causes and outcomes of child maltreatment and family violence, with the expected result that children with complex trauma histories will receive more accurate diagnostic assessment and effective treatment.26
The clinical complexity and heterogeneity of maltreatment and family violence is reflected in a multifarious field of intervention approaches that span the continuum of primary, secondary, and tertiary prevention. Using a framework provided by the Administration for Children and Families Child Welfare Information Gateway,27 primary prevention approaches are defined as having a universal focus to prevent the occurrence of child abuse, neglect, or family violence in the general population. Secondary prevention targets families with risk factors associated with maltreatment or family violence to prevent their occurrence. Tertiary prevention targets children and families in which maltreatment or family violence has already occurred to promote the child’s well-being and prevent recurrence.
Although it is useful to conceptualize the studies included in this review as tertiary in nature, there is no single well-recognized structural framework that currently exists or readily emerges for organizing a comparative review of tertiary prevention for children exposed to maltreatment or family violence. With the substantive input of nationally recognized experts on the clinical needs of this population, the Evidence-based Practice Center (EPC) team worked through a number of iterations to reach a conceptual framework for the CER that represents the array of interventions and treatments being used in the field and resonates with the human service systems that serve this population.
This framework categorizes interventions used to address the negative impact of child maltreatment and family violence on children’s well-being as either clinical-level or system-level in approach.
The clinical-level interventions we will review in this CER are most commonly used with a heterogeneous population in that children are targeted based on exposure (either to a specific form of maltreatment or family violence or are based on their involvement with the child welfare system) and not necessarily the presence or degree of symptomatology. A small subset of studies in this arena restrict the study sample to children with clinical-level symptomatology associated with the exposure, and even within these studies the population will include children with a wide range of symptoms or symptom severity. For these studies, general and specific types of pharmacotherapy are relevant treatment approaches (e.g., selective serotonin-reuptake inhibitors (SSRIs), tricyclic antidepressants (TCAs), benzodiazepines, beta-blockers, alpha-blockers, mood stabilizers, antipsychotics).
System-level interventions comprise service-delivery approaches or strategies to improve the system of care for children and their caregivers and family. Examples include wraparound services and other service integration approaches, service delivery strategies such as solution-based casework, and broad ecological models such as milieu therapy that provide treatment within and through the different settings in the child’s daily life. System-level interventions incorporate clinical components, but the unit of analysis for comparing effectiveness is the collective set of activities comprising the service-delivery approach or strategy.
In addition to these two broad categories of interventional approaches, two major clinically relevant factors will also be addressed in this review. First, the review will take into account the primary caregiving environment in which the maltreated child or child exposed to family violence lives, as the caregiving context represents clinically distinct scenarios for intervention: a) children living with and for whom the primary caregiver is the biological parent, b) children in foster or kin (relative) care or who were adopted from foster care, and c) children in residential treatment or group home settings in which the child likely has multiple primary caregivers. Second, the review will also examine findings as they apply to age-related developmental periods that reflect children’s particular emotional needs and emerging developmental capabilities.
A number of government and nonprofit organizations have developed highly regarded and widely used evidence-based registries and informational resources to guide clinical and other practitioners, funders, and policymakers in selecting and supporting effective interventions to mitigate risk and to address the mental and behavioral health needs of children exposed to maltreatment or family violence. The purpose of this review is to further help clinicians and other decisionmakers in the field of child trauma by providing a comprehensive, systematic review of the comparative benefits and harms of evidence-based interventions with children exposed to maltreatment or family violence.
The Key Questions
We revised and finalized the PICOTS (population, intervention, comparators, outcomes, timing and setting) and the Key Questions (KQs) after we received input from the public and our Technical Expert Panel (TEP) to assure a complementary approach across the CER series on interventions for children exposed to trauma. Based on the TEP’s input, we revised the title of the review to reference child maltreatment and family violence separately in the review. In the PICOTS, we revised the definition of the population to include children under the age of 18 (i.e., excluding those who had reached the legal age of adulthood).We provide an additional exclusion criterion related to children’s placement in out-of-home care; specifically, to maintain the focus on exposure to maltreatment, we will exclude children placed in out-of-home care because of psychiatric disorders, chronic delinquency, and/or other serious behavioral problems. With the decision to define the population as children exposed to maltreatment or family violence, we expanded the scope of the review to include all relevant interventions (i.e., not limited to caregiver-mediated interventions, as was the focus of the topic as initially nominated). Additionally, as described above, we modified the conceptual framework for the psychosocial interventions, shifting to a clinical-level or system-level categorization. To create a parallel approach across the two CERs focused on children exposed to trauma, pharmacologic interventions have been added to the included interventions for this review. Additional changes to the PICOTS were made to ensure a consistent and parallel approach to the PICOTS across the CER series (e.g., terminology used in the sections on outcomes and setting; specific approaches named in the section on interventions).
The wording of the KQs has been changed to align with the definition of the population and focus on children, rather than on their foster and adoptive parents, as was the focus of the topic as initially nominated. To make clear that caregiver-level outcomes are included in KQ 1 under “healthy caregiver-child relationship,” we provide several examples in the parenthesis next to this outcome (both in the KQs and the PICOTS). To avoid overlap with “mental and behavioral health” outcomes in KQ 1, we removed the reference to the “emotional and social” aspects of healthy development and added “language,” given its salience as a developmental outcome in the literature we have reviewed to date. Additionally, the outcome “school success” has been replaced with the term “school-based functioning” in line with the terminology commonly used in the literature. In KQ 3, the terms used to describe types of intervention characteristics have also been modified to reflect the terminology used in the literature. Several minor changes were made to KQ 4. The term “primary caregiving context” is used rather than “caregiver type,” reflecting guidance from the TEP to view the caregiving environment more broadly; accordingly, we also added group settings as additional examples of the caregiving context. We also refined the terminology used to describe the types of caregivers. In KQ 4, we also removed the reference to institutional care, and complex stress disorders and serious emotional disturbance were added as examples of mental or behavioral health problems. In KQ 5, with the expansion to include all interventions for children exposed to maltreatment or family violence, children are included as well as caregivers for the analyses of treatment adherence and withdrawal.
Revised Key Questions
- What is the comparative effectiveness of interventions with children exposed to maltreatment or family violence for promoting child well-being? Specifically:
- Mental and behavioral health
- Healthy caregiver-child relationship (e.g., increased caregiver responsivity and sensitivity; positive attitudes toward childrearing; positive perceptions of the child and causal attributions about the child’s behavior; decreased negative parent-child interactions; increased family functioning; secure attachment)
- Healthy development (e.g., cognitive, language, physical)
- School-based functioning
- What is the comparative effectiveness of interventions with children exposed to maltreatment or family violence for promoting child welfare outcomes? Specifically:
- Safety (i.e., prevention of maltreatment recurrence)
- Placement stability
- Among the interventions under review, how do interventions with particular characteristics compare in improving child outcomes. Intervention characteristics may include:
- Modality (e.g., individual, dyadic, group, family-based, mixed)
- Theoretical orientation (e.g., cognitive behavioral, psychodynamic, eclectic)
- Type of setting (i.e., specialty or nonspecialty service-delivery settings)
- How do interventions compare for improving child outcomes within population subgroups? Population subgroups comprise the following:
- Child subgroups
- Age groups: 0 to 2 years (infant/toddler), 3 to 5 years (preschool), 6 to 12 years (middle childhood), and 13 to 17 years (adolescence) and other sociodemographic subgroups (e.g., race, ethnicity, sex)
- Type of traumatic exposure (e.g., neglect, physical abuse, sexual abuse, intimate partner violence)
- Severity of traumatic exposure
- Presence of mental or behavioral health problems (e.g., complex traumatic stress disorders, serious emotional disturbance) or other special needs (e.g., failure to thrive, prenatal substance exposure)
- Caregiver subgroups
- Primary caregiving context: biological parent; foster, kin (relative), or adoptive caregivers; residential program or group home)
- Presence of mental health problems, substance abuse, or domestic violence
- Sociodemographic groups (e.g., age, race, ethnicity, sex)
- Child subgroups
- What is the comparative effectiveness of interventions with children exposed to maltreatment or family violence for engaging children and/or caregivers in treatment (e.g., treatment adherence, treatment withdrawal)?
- What adverse events are associated with interventions for children exposed to maltreatment or family violence (e.g., retraumatization)?
The PICOTS further define the scope of the CER (Table 1).
|Abbreviations: PTSD = post-traumatic stress disorder; SSRIs = selective serotonin-reuptake inhibitors; TCAs = tricyclic antidepressants|
The population may include the child’s primary caregiver(s) when the intervention targets the caregiving context. The primary caregiver is defined as the biological parent; foster, kin (relative), or adoptive caregiver; or caregivers in a residential program or group home.
Interventions that aim to prevent, ameliorate, or improve mental health symptoms, behavior problems, or psychopathology; optimize child development and functioning; and/or improve child welfare outcomes. Interventions of interest are categorized as:
Pre-service foster parent training programs will not be included in this review.
The comparison condition as defined in the respective studies, including active controls (such as usual care) and inactive controls (such as wait-list groups).
The populations included in this review are children and adolescents who have been exposed to maltreatment or family violence and their caregivers when applicable (Figure 1). KQ 1 will assess the effectiveness of the interventions in improving child outcomes, specifically mental and behavioral health, a healthy caregiver-child relationship, healthy development, and school-based functioning. KQ 2 will assess child welfare outcomes, specifically safety, placement stability, and permanency. The effectiveness of interventions in population subgroups will be compared in KQ 4, while differences in efficacy by intervention characteristics will be reviewed in KQ 3. KQ 5 will review the evidence on treatment adherence, and KQ 6 will assess adverse events during treatment.
Figure 1. Analytic framework
1Population may include the child’s primary caregiver(s) when the intervention targets the caregiving context.
A. Criteria for Inclusion/Exclusion of Studies in the Review
Criteria for inclusion and exclusion of studies are based on the PICOTS model outlined in Section II, as well as the study-specific inclusion criteria listed in Table 2.
|Category||Criteria for inclusion|
|* Search to be updated when report is under peer review.|
|Study design||Systematic reviews, randomized controlled trials, nonrandomized controlled trials, prospective and retrospective cohort studies, and case-control studies|
|Sample size||N ≥ 10|
|Geography||United States and international|
|Time of publication||1990 to present*|
|Language of publication||English|
B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies To Answer the Key Questions
We will systematically search, review, and analyze the scientific evidence for each KQ. To identify articles for this review, we will conduct focused searches of PubMed®, the Social Science Citation Index, PsycINFO®, and the Cochrane Library. An experienced research librarian will use a predefined list of search terms and medical subject headings (MeSH®), when applicable. Search terms and limits are listed in Table 3. We will limit the search to studies published in English based on limited resources; this may bias the report toward including more studies from English-speaking countries.
|#1||Search "Child Abuse"[Mesh] OR "Child Welfare"[Mesh] OR "Infant Welfare"[Mesh] OR "Domestic Violence"[Mesh] OR "Foster Home Care"[Mesh]|
|#2||Search "child abuse"[tiab] OR "child maltreatment"[tiab] OR "neglect"[tiab] OR "domestic violence"[tiab] OR "child welfare"[tiab] OR "foster care"[tiab] OR "kinship care"[tiab] OR "out of home care"[tiab] OR "out of home placement"[tiab] OR "looked after child"[tiab] OR "looked after young"[tiab] OR child protective service* OR physical abuse*|
|#3||Search #1 OR #2|
|#4||Search "Adolescent"[Mesh] OR "Child"[Mesh] OR "Infant"[Mesh]|
|#5||Search #3 AND #4|
|#6||Search #5 Limits: Humans, English|
|#7||Search ((#6) AND "1990/01/01"[Publication Date] : "3000"[Publication Date]) AND "0"[Publication Date] : "3000"[Publication Date]|
|#16||Search Citalopram OR Escitalopram OR Fluoxetine OR Fluvoxamine OR Paroxetine OR Sertraline OR Desvenlafaxine OR Duloxetine OR Venlafaxine OR Buproprion OR Mirtazapine OR Nefazodone OR Trazodone OR Clonidine OR Guanfacine OR Propranolol OR Phenelzine OR tranylcypromine OR Clomipramine OR Imipramine OR Topiramate OR Tiagabine OR Lamotrigine OR Lithium OR Carbamazepine OR “Divalproex sodium” OR Oxcabazepine OR Aripiprazole OR Olanzapine OR Risperidone OR Quetiapine OR Clonazepam OR Lorazepam OR Alprazolam OR Buspirone OR Propranolol OR Estazolam OR Flurazepam OR Temazepam OR Triazolam OR Chlordiazepoxide OR Clorazepate OR Diazepam OR Oxazepam OR Prazepam OR Quazepam|
|#17||Search #7 AND #16|
|#18||Search "Psychotropic Drugs"[Mesh]|
|#19||Search #7 AND #18|
|#23||Search "Intervention Studies"[Mesh]|
|#24||Search #7 AND #23|
|#26||Search #7 AND #25|
|#27||Search "Drug Therapy"[Mesh]|
|#28||Search #7 AND #27|
|#29||Search "Complementary Therapies"[Mesh]|
|#30||Search #7 AND #29|
|#31||Search #17 OR #19 OR #24 OR #26 OR #28 OR #30|
|#32||Search "Randomized Controlled Trial"[Publication Type] OR "Randomized Controlled Trials as Topic"[Mesh] OR "Single-Blind Method"[Mesh] OR "Double-Blind Method"[Mesh] OR "Random Allocation"[Mesh]|
|#33||Search #31 AND #32|
|#34||Search "meta-analysis"[Publication Type] OR "meta-analysis as topic"[MeSH Terms] OR "meta-analysis"[All Fields]|
|#35||Search #31 AND #34|
|#36||Search "Comparative Study"[Publication Type] OR "comparative study"|
|#37||Search #31 AND #36|
|#38||Search ("review"[Publication Type] AND "systematic"[tiab]) OR "systematic review"[All Fields] OR ("review literature as topic"[MeSH AND "systematic"[tiab])|
|#39||Search #31 AND #38|
|#40||Search "Cohort Studies"[Mesh]|
|#41||Search #31 AND #40|
|#43||Search #31 AND #42|
|#44||Search "Case-Control Studies"[Mesh]|
|#45||Search #31 AND #44|
|#47||Search #31 AND #46|
|#48||Search #33 OR #35 OR #37 OR #39 OR #41 OR #43 OR #45 OR #47|
|#49||Search "intervention"[tiab] OR "interventions"[tiab] OR "treatment"[tiab] OR "treatments"[tiab] OR "therapy"[tiab] OR "therapies"[tiab] OR "therapeutic"[tiab] OR "training"[tiab] OR "psychoeducation"[tiab] OR "program"[tiab] OR "programs"[tiab] OR "pharmacotherapy"[tiab]|
|#50||Search #7 AND #49 AND (#32 OR #34 OR #36 OR #38 OR #40 OR #42 OR #44 OR #46)|
|#51||Search #48 OR #50|
To build on the work of the existing evidence-based registries and databases on interventions for children, we will search the following registries for relevant peer-reviewed articles that may have been missed in the systematic literature search.
- National Child Traumatic Stress Network’s Empirically Supported Treatments and Promising Practices28
- California Evidence-based Clearinghouse for Child Welfare29
- National Registry of Evidence-based Programs and Practices30
- Office of Juvenile Justice and Delinquency Prevention Model Programs Guide31
- Center for the Study and Prevention of Violence Program Database32
We will also complete targeted searches for unpublished or grey literature relevant to the review. Methods for identifying grey literature will include a review of trial registries, specifically ClinicalTrials.gov, Health Services Research Projects in Progress (http://www.nlm.nih.gov/hsrproj/) and the European Union Clinical Trials Register (https://www.clinicaltrialsregister.eu/). Further, AHRQ will also request Scientific Information Packets from the developers or distributors of the interventions identified in the literature review. Scientific Information Packets allow an opportunity for the developers and distributors of the interventions to provide the EPC with both published and unpublished data that they believe should be considered for the review. The EPC will review the information provided in the Scientific Information Packets and grey literature. We will include studies that meet all inclusion criteria and contain enough information on the research methods used for the risk of bias assessment.
We will also conduct an updated literature search (of the same databases searched initially) concurrent with the peer review process. Any literature suggested by peer reviewers or public comment respondents will be investigated and, if appropriate, incorporated into the final review. Reference lists of systematic reviews that are pertinent but do not meet our inclusion criteria will be scanned for studies that should be considered for this review. Appropriateness will be determined by the same inclusion and exclusion criteria described in the previous section.
C. Data Abstraction and Data Management
All titles and abstracts identified through searches will be independently reviewed for eligibility against our inclusion/exclusion criteria by two trained members of the research team. Studies marked for possible inclusion by either reviewer will undergo a full-text review. For studies without adequate information to determine inclusion or exclusion, we will retrieve the full text and then make the determination. All results will be tracked in an EndNote® database.
We will retrieve and review the full text of all articles included during the title/abstract review phase. Each full-text article will be independently reviewed by two trained members of the research team for inclusion or exclusion on the basis of the eligibility criteria described earlier. If both reviewers agree that a study does not meet the eligibility criteria, the study will be excluded. If the reviewers disagree, conflicts will be resolved by discussion and consensus or by consulting a third member of the review team. All results will be tracked in an EndNote database. We will record the reason why each excluded full-text publication did not satisfy the eligibility criteria so that we can later compile a comprehensive list of such studies.
For studies that meet our inclusion criteria, we will abstract relevant information into evidence tables. We will design data abstraction forms to gather pertinent information from each article, including characteristics of study populations, settings, interventions, comparators, study designs, methods, and results. Trained reviewers will extract the relevant data from each included article into the evidence tables. All data abstractions will be reviewed for completeness and accuracy by a second member of the team.
D. Assessing the Risk of Bias of Individual Studies
To assess the risk of bias of studies, we will use criteria described in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews.33 We will assess the potential for selection bias, performance bias, attrition bias, detection bias, and reporting bias. Results of this assessment will be summarized in a rating of low, medium, or high risk of bias. In general, a study with a low risk of bias has a strong design, measures outcomes appropriately, uses appropriate statistical and analytical methods, reports low attrition, and reports methods and outcomes clearly and precisely. Studies with a medium risk of bias are those that do not meet all criteria required for low risk of bias but do not have flaws that are likely to cause major bias. Missing information often leads to ratings of medium as opposed to low. Studies with a high risk of bias are those with at least one major flaw that is likely to cause significant bias and thus might invalidate the results. Examples of such major flaws include errors in conduct or analysis. Studies with a high risk of bias will be considered in this review only if we are unable to answer the KQs with the available studies with low or medium risk of bias.
Two independent reviewers will assess the risk of bias for each study. Disagreements between the two reviewers will be resolved by discussion and consensus or by consulting a third member of the team.
E. Data Synthesis
If we find three or more similar studies for a comparison of interest, we will consider quantitative analysis (i.e., meta-analysis) of the data from those studies. To determine whether quantitative analyses are appropriate, we will assess the clinical heterogeneity using the PICOTS framework and following established guidance.34 We will consider similarities and differences by sociodemographic factors (e.g., age), type and severity of trauma exposure, and caregiving context. Because many studies in this literature do not stratify by these factors, we will first describe all of the included studies and in KQs 1 and 2. Planned population subgroup analyses for outcomes listed in KQs 1 and 2 are specified in KQ 4. We will evaluate the statistical heterogeneity of pooled analysis using both the chi-squared statistic and the I2 statistic (the proportion of variation in the study estimates due to heterogeneity).
When quantitative analyses are not appropriate (e.g., because of heterogeneity, insufficient numbers of similar studies, or insufficiency or variation in reporting), we will synthesize the data qualitatively.
F. Grading the Evidence for Each Key Question
We will grade the strength of evidence on the basis of guidance established for the EPC Program.35 Developed to grade the overall strength of a body of evidence, this approach incorporates four key domains: risk of bias (including study design and aggregate quality), consistency, directness, and precision of the evidence. The grades of evidence that can be assigned are described in Table 4. Grades reflect the strength of the body of evidence to answer the KQs on the comparative effectiveness, efficacy, and harms of the interventions in this review. Two reviewers will assess each domain and the overall grade for each key outcome listed in the framework, and conflicts will be resolved by consensus.
|Source: Owens DK, Lohr KN, Atkins D, et al. J Clin Epidemiol 2010 May;63(5):513-23.35|
|High||High confidence that the evidence reflects the true effect: Further research is very unlikely to change our confidence in the estimate of effect.|
|Moderate||Moderate confidence that the evidence reflects the true effect: Further research may change our confidence in the estimate of the effect and may change the estimate.|
|Low||Low confidence that the evidence reflects the true effect: Further research is likely to change our confidence in the estimate of the effect and is likely to change the estimate.|
|Insufficient||Evidence either is unavailable or does not permit estimation of an effect.|
G. Assessing Applicability
We will assess the applicability34, 36 both of individual studies and of the body of evidence. For individual studies, we will examine conditions that may limit applicability based on the PICOTS structure. Such conditions may be associated with heterogeneity of treatment effect and the ability to generalize the effectiveness of an intervention to use in everyday practice.
To assess the applicability of a body of evidence, we will consider the consistency of results across studies that represent an array of different populations. We will abstract and report key characteristics that may limit applicability into evidence tables. Abstractors will assess how each of the following characteristics may limit the applicability of the study results:
- Narrow eligibility criteria or exclusion of patients with comorbidities
- Large differences between demographics of the study population and community patients
- Intensity and delivery of interventions that may not be feasible for routine use
- Highly selected intervention team or level of training/proficiency not widely available
- Comparison group does not represent an available alternative treatment
- Perry BD. Incubated in terror: neurodevelopmental factors in the "cycle of violence." In: Osofsky J, ed. Children in a violent society. New York: Guilford Press; 1997. p. 124-49.
- Shonkoff JP, Phillips D, eds. From neurons to neighborhoods: the science of early childhood development. Washington, DC: National Academy Press; 2000.
- National Scientific Council on the Developing Child. Excessive stress disrupts the architecture of the developing brain: working paper 3. Updated and Reprinted June 2009. Available at: http://developingchild.harvard.edu/index.php/resources/reports_and_working_papers/. Accessed November 11, 2011.
- Felitti VJ, Anda RF, Nordenberg D, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The Adverse Childhood Experiences (ACE) Study. Am J Prev Med 1998 May;14(4):245-58. PMID: 9635069.
- Turner HA, Finkelhor D, Ormrod R. The effect of lifetime victimization on the mental health of children and adolescents. Soc Sci Med 2006 Jan;62(1):13-27. PMID: 16002198.
- Leeb RT, Paulozzi L, Melanson C, et al. Child maltreatment surveillance: uniform definitions for public health and recommended data elements, version 1.0. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; 2008. Available at: http://www.cdc.gov/violenceprevention/pdf/CM_Surveillance-a.pdf. Accessed November 15, 2010.
- The Child Abuse Prevention and Treatment Act As Amended by P.L. 111-320, the CAPTA Reauthorization Act of 2010. Washington, DC: U.S. Department of Health and Human Services; 2011. Available at : http://www.acf.hhs.gov/programs/cb/laws_policies/cblaws/capta/capta2010.pdf. Accessed August 15, 2011.
- Cohen JA, Mannarino AP. A treatment outcome study for sexually abused preschool children: initial findings. J Am Acad Child Adolesc Psychiatry 1996 Jan;35(1):42-50. PMID: 8567611.
- Cicchetti DV, Lynch M. Failures in the expectable environment and their impact on individual development: the case of child maltreatment. In: Cicchetti DV, Cohen DJ, eds. Developmental psychopathology: Vol. 2. Risk, disorder, and adaptation. New York: Wiley; 1995. p. 32-71.
- Egeland B, Yates T, Appleyard K, et al. The long-term consequences of maltreatment in the early years: A developmental pathway model to antisocial behavior. Child Serv Soc Policy Res Pract 2002;5(4):249-60.
- MacMillan HL, Fleming JE, Streiner DL, et al. Childhood abuse and lifetime psychopathology in a community sample. Am J Psychiatry 2001 Nov;158(11):1878-83. PMID: 11691695.
- Widom CS, DuMont K, Czaja SJ. A prospective investigation of major depressive disorder and comorbidity in abused and neglected children grown up. Arch Gen Psychiatry 2007 Jan;64(1):49-56. PMID: 17199054.
- Widom CS, Marmorstein NR, White HR. Childhood victimization and illicit drug use in middle adulthood. Psychol Addict Behav 2006 Dec;20(4):394-403. PMID: 17176174.
- Child Welfare Information Gateway. Issue brief: understanding the effects of maltreatment on early brain development. Washington, DC: U.S. Department of Health and Human Services; 2009. Available at: http://www.childwelfare.gov/pubs/issue_briefs/brain_development/brain_development.pdf. Accessed September 29, 2010.
- Gunnar MR, Fisher PA, for the Early Experience Stress and Prevention Network. Bringing basic research on early experience and stress neurobiology to bear on preventive interventions for neglected and maltreated children. Dev Psychopathol 2006 Summer;18(3):651-77. PMID: 17152395.
- Pears K, Fisher PA. Developmental, cognitive, and neuropsychological functioning in preschool-aged foster children: associations with poor maltreatment and placement history. J Dev Behav Pediatr 2005 Apr;26(2):112-22. PMID: 15827462.
- De Bellis MD. The psychobiology of neglect. Child Maltreat 2005 May;10(2):150-72. PMID: 15798010.
- Graham YP, Heim C, Goodman SH, et al. The effects of neonatal stress on brain development: implications for psychopathology. Dev Psychopathol 1999 Summer;11(3):545-65. PMID: 10532624.
- Tottenham N, Hare TA, Quinn BT, et al. Prolonged institutional rearing is associated with atypically large amygdala volume and difficulties in emotion regulation. Dev Sci 2010 Jan 1;13(1):46-61. PMID: 20121862.
- Nelson CA. A neurobiological perspective on early human deprivation. Child Dev Perspect 2007 Jul;1(1):13-8.
- Cicchetti D, Rogosch FA, Toth SL. Fostering secure attachment in infants in maltreating families through preventive interventions. Dev Psychopathol 2006 Summer;18(3):623-49. PMID: 17152394.
- National Child Traumatic Stress Network Complex Trauma Task Force. Complex trauma in children and adolescents: white paper from the National Child Traumatic Stress Network Complex Trauma Task Force. Available at: http://www.nctsn.org/trauma-types/complex-trauma. Accessed November 11, 2011.
- National Child Traumatic Stress Network Zero to Six Collaborative Group. Early childhood trauma. Available at: http://nctsn.org/trauma-types/early-childhood-trauma. Accessed November 11, 2011.
- Courtois CA, Ford JD, eds. Treating complex traumatic stress disorders: an evidence-based guide. New York: The Guilford Press; 2009.
- Zero to Three. Diagnostic classification of mental health and developmental disorders of infancy and early childhood, revised ed (CD:0–3R). Washington: Zero to Three Press; 2005.
- van der Kolk BA, Pynoos RS, Cicchetti D, et al. Proposal to include a developmental trauma disorder diagnosis for children and adolescents in DSM-V. Available at: http://www.traumacenter.org/announcements/DTD_papers_Oct_09.pdf. Accessed August 15, 2011.
- Child Welfare Information Gateway. Framework for Prevention of Child Maltreatment.Available at: http://www.childwelfare.gov/preventing/overview/framework.cfm. Accessed November 11, 2011.
- National Child Traumatic Stress Network. Empirically Supported Treatments and Promising Practices. Available at: http://www.nctsn.org/resources/topics/treatments-that-work/promising-practices. Accessed November 11, 2011.
- California Evidence-Based Clearinghouse for Child Welfare. Information and Resources for Child Welfare Professionals. Available at: http://www.cebc4cw.org/ Accessed November 10, 2010.
- U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration. NREPP: SAMHSA's National Registry of Evidence-based Programs and Practices. Available at: http://www.nrepp.samhsa.gov/. Accessed November 11, 2011.
- Office of Juvenile Justice and Delinquency Prevention.OJJDP Model Programs Guide. Available at: http://www.ojjdp.gov/mpg/. Accessed November 10, 2010.
- University of Colorado Center for the Study and Prevention of Violence. CSPV Information Clearinghouse. Available at: http://www.colorado.edu/cspv/infohouse/index.html Accessed November 12, 2010.
- Agency for Healthcare Research and Quality. Assessing the risk of bias of individual studies when comparing medical interventions. In: Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality; June 2011. AHRQ Publication No. 10(11)-EHC063-EF. Draft chapter available at: www.effectivehealthcare.ahrq.gov.
- Fu R, Gartlehner G, Grant M, et al. Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol 2011 Nov;64(111187-97. PMID: 21477993.
- Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the Effective Health-Care Program. J Clin Epidemiol 2010 May;63(5):513-23. PMID: 19595577.
- Atkins D, Chang S, Gartlehner G, et al. Assessing applicability when comparing medical interventions: AHRQ and the Effective Health Care Program. J Clin Epidemiol 2011 Nov;64(111198-207. PMID: 21463926.
- Lieberman AF, Van Horn P, Ippen CG. Toward evidence-based treatment: child-parent psychotherapy with preschoolders exposed to marital violence. J Am Acad Child Adolesc Psychiatry 2005 Dec;44(12):1241-8. PMID: 16292115.
- Lieberman AF, Van Horn P. Psychotherapy with infants and young children: Repairing the effects of stress and trauma on early attachment. New York: The Guilford Press; 2008.
Definition of Terms
- Child maltreatment—the definition provided by the Centers for Disease Control and Prevention includes both child abuse (words or overt actions that cause harm, potential harm, or threat of harm to a child) and child neglect (failure to provide for a child’s basic physical, emotional, or educational needs or to protect a child from harm or potential harm; privation).6
- Family Violence—the definition of child maltreatment provided by the Centers for Disease Control and Prevention identifies a caregiver’s failure to take available measures to protect the child from pervasive violence in the home as a form of neglect. For example, a caregiver who sells illicit drugs out of the child’s home exposes the child to a violent environment. However, the definition also provides the following exception: “Exposure of a child to violence between caregivers in the home may also qualify as exposure to a violent environment, particularly if the caregivers do not take available measures to protect the child from exposure. But, in instances where the caregiver is being victimized by a partner, and alternatives to protect the child are not available, or the caregiver is unaware of alternatives, the caregiver is not maltreating the child. 6
- Caregiver—the definition provided by the Centers for Disease Control is a person in a permanent or temporary custodial role. In a custodial role, the person is responsible for care and control of the child and for the child’s overall health and welfare. Primary caregivers must live with the child at least part of the time and can include, but are not limited to, a relative or biological, adoptive, step-, or foster parent(s); a legal guardian(s); or their intimate partner.6
- Sexual abuse—the definition provided by the Child Abuse Prevention and Treatment Act is the employment, use, persuasion, inducement, enticement, or coercion of any child to engage in, or assist any other person to engage in, any sexually explicit conduct or simulation of such conduct for the purpose of producing a visual depiction of such conduct; or the rape, and in cases of caretaker or interfamilial relationships, statutory rape, molestation, prostitution, or other form of sexual exploitation of children, or incest with children.7
Summary of Protocol Amendments
Review of Key Questions
For all EPC reviews, key questions were reviewed and refined as needed by the EPC with input from Key Informants and the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed. In addition, for Comparative Effectiveness reviews, the key questions were posted for public comment and finalized by the EPC after review of the comments.
Key Informants are the end-users of research, including patients and caregivers, practicing clinicians, relevant professional and consumer organizations, purchasers of health care, and others with experience in making health care decisions. Within the EPC program, the Key Informant role is to provide input into identifying the Key Questions for research that will inform health care decisions. The EPC solicits input from Key Informants when developing questions for systematic review or when identifying high priority research gaps and needed new research. Key Informants are not involved in analyzing the evidence or writing the report and have not reviewed the report, except as given the opportunity to do so through the peer or public review mechanism.
Key Informants must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their role as end-users, individuals are invited to serve as Key Informants and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
Technical Experts comprise a multi-disciplinary group of clinical, content, and methodological experts who provide input in defining populations, interventions, comparisons, or outcomes as well as identifying particular studies or databases to search. They are selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. Technical Experts provide information to the EPC to identify literature search strategies and recommend approaches to specific issues as requested by the EPC. Technical Experts do not do analysis of any kind nor contribute to the writing of the report and have not reviewed the report, except as given the opportunity to do so through the public review mechanism.
Technical Experts must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Because of their unique clinical or content expertise, individuals are invited to serve as Technical Experts and those who present with potential conflicts may be retained. The TOO and the EPC work to balance, manage, or mitigate any potential conflicts of interest identified.
Peer reviewers are invited to provide written comments on the draft report based on their clinical, content, or methodological expertise. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. Peer reviewers do not participate in writing or editing of the final report or other products. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published three months after the publication of the Evidence report.
Potential Reviewers must disclose any financial conflicts of interest greater than $10,000 and any other relevant business or professional conflicts of interest. Invited Peer Reviewers may not have any financial conflict of interest greater than $10,000. Peer reviewers who disclose potential business or professional conflicts of interest may submit comments on draft reports through the public comment mechanism.
EPC Team Disclosures
With the exception of the following, the team has no interests to disclose:
- The Lead Investigator discloses the following business and professional interests: As an implementation evaluator/researcher, she is currently collaborating on a translational research study with colleagues in the field who have developed a court improvement model to improve child well-being outcomes for adjudicated young children and their caregivers. The study is funded by the National Center for Injury Prevention and the CDC. The court improvement model comprises two sets of core components: 1) systems integration across the judiciary, child welfare, and child mental health, and 2) effective implementation of evidence-based therapeutic services adapted for the court setting. The evidence-based therapy used in the originating intervention site is Child-Parent Psychotherapy.37,38 However, because the model is intended to have broad applicability for communities, it does not specify a single intervention approach. Rather, the model incorporates any evidence-based therapeutic treatment designed to repair the relationship between the young maltreated child and his or her primary caregiver and thereby improve child well-being.
- Scientific Advisor A discloses the following business and professional interests: She is a coinvestigator on the translational research study described above.
- Coinvestigator A discloses the following financial, business and professional interests. He is the Executive Director of Child and Family Support Services, Inc. (501c3) and faculty at the University of North Carolina at Chapel Hill and Duke University.
- Research Associate A discloses the following business and professional interests. She is a member of the Scientific Review Panel for the National Campaign to Prevent Teen and Unintended Pregnancy and a Grant Review Panelist for the Office of Population Affairs, Office of Adolescent Pregnancy Programs. Research Associate A is also an analyst on a CDC-funded project to develop Web materials for disseminating a universal parenting program for mothers of children aged 2 to 4 years.
Role of the Funder
This project was funded under Contract No. 290-2007-10056-I from the Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. The Task Order Officer reviewed contract deliverables for adherence to contract requirements, including the objectivity and independence of the research process and the methodological quality of the report. The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services.