Draft Key Questions
Disposition of Comments Report
Background and Objectives for the Systematic Review
This review addresses two aspects of ADHD (Attention Deficit Hyperactivity Disorder) treatment that were nominated independently, ADHD treatment in preschoolers and long-term effects of treatment. Discussions during our partner calls and the Technical Expert Panel calls resulted in the current protocol for combining the two reviews.
In addition, there was an interest in summarizing information about the variation in prevalence, diagnosis and treatment of ADHD by factors such as geography, age, time period and socio-demographic characteristics. This question was added to the review.
a. Treatment of preschoolers
Diagnosis of preschoolers may often not adhere to strict DSM (Diagnostic and Statistical Manual of Psychiatric Disorders) criteria, so we have widened the included diagnoses for this age group to include studies that diagnose these children using the broader DSM category of Disruptive Behavior Disorders, which includes ADHD, CD (Conduct Disorder) and ODD (Oppositional Defiant Disorder).
ADHD usually begins before children enter school. In the preschool age group ADHD is characterized not only by impairment in attention span, excessive impulsivity and over-activity, but also is frequently accompanied by temper tantrums, accident proneness, uncooperative behavior and aggressiveness that can interfere with attendance at daycare or preschool, and high family burden of care and distress. Accurate identification of ADHD requires evaluation of symptom impairment in multiple settings and at multiple times, with input from more than one informant. Such diagnostic evaluations can be complicated and often result in identification of co-occurring anxiety and disruptive behavior disorders, as well as developmental delays. Preschoolers with ADHD may continue to have difficulties with attention, learning and behavior in grade school. With the increasing awareness of the burden of impairment that school-age children with ADHD experience, early identification and intervention has become a priority. This is especially so as the preschool age is an important neurodevelopmental stage when children consolidate social, emotional and cognitive skills that affect long-term academic and psychosocial outcomes. However, the disruptive behavior often brings preschoolers into treatment, rather than academic concerns.
High quality evidence regarding interventions for preschoolers with psychiatric disorders is sparse, reflecting the lack of clear diagnostic guidelines available until recently. In the past 10 years clinicians have been applying evidence from studies of school-age children with ADHD, identifying psychostimulants as an effective and safe first line treatment choice. Medicaid surveys note increasing amounts of off-label prescriptions for psychostimulants and other psychiatric medications for preschoolers in the past 10 years.1 Recent reviews emphasize that non-pharmacotherapeutic interventions, especially parent training, should be tried first to ameliorate the disruptive behavior, before any interventions such as medications that could have unknown long-term neurobiologic impact.2
b. Long-term effects
Since ADHD is a chronic disorder, many children, teens and adults stay on medications for years at a time. Given the possibility of cumulative effects over time, a review of evidence regarding benefits and risks of prolonged medication use for ADHD is indicated.
When studies evaluating treatment for ADHD were previously reviewed,3 the majority of published studies were of short duration and examined psychostimulant use for core ADHD symptoms. Since that time more agents are in use, and systematic open label follow-up studies are available evaluating treatment effectiveness over longer time periods.
c. Variability of diagnosis and treatment
ADHD symptoms exist on a continuum in the general population, identified to a greater or lesser degree depending on methods of diagnosis, including who provides the information (e.g. parent or teacher) and the threshold chosen for defining a “case”. A recent meta-analysis estimates prevalence for childhood ADHD world-wide to be 5.29% ( 95% CI 5.01 – 5.56).4 This study argues that apparent variability in estimated prevalence can be explained by differences in the methodology, primary source of information, and diagnostic criteria used in different studies. Although overall prevalence may vary from place to place, in all areas boys are classified with ADHD twice as frequently as girls and school age children twice as frequently as adolescents. Fewer attempts have been made to estimate prevalence among adults. A recent meta-analysis produced a pooled prevalence world wide of 2.5 % (95% CI 2.1-3.1).5
Over the past 10 years studies have also documented increasing rates of identification and treatment for people with ADHD, many using health administrative databases. In some cases increases in prescriptions have been linked to specific physicians, suggesting that increases in identification may be associated with changes in practice patterns rather than an increase in the underlying prevalence of the disorder.6,7 In fact the underlying prevalence of the disorder in children appears to have been relatively stable since the 1980s, to the extent that it has been measured using identical methods.8 Increases in identification and treatment using medication have occurred primarily among girls and older children consistent with changes in clinical guidelines.9,10 Furthermore, increases in off label prescription of psychotropic medications for very young children (preschoolers) to treat ADHD or disruptive behavior have also been noted.1
The Key Questions
Question 1. Among children < 6 years of age with ADHD or Disruptive Behavior Disorder, what are the effectiveness and adverse event outcomes following treatment?
Question 2. Among people 6 years of age or older with ADHD, what are the effectiveness and adverse event outcomes following 12 months or more of any combination of follow-up or treatment, including, but not limited to, 12 months or more of continuous treatment?
Question 3. How do a) underlying prevalence of ADHD, and b) rates of diagnosis (clinical identification) and treatment for ADHD vary by geography, time period, provider type, and socio-demographic characteristics?
|PICO||Question 1||Question 2||Question 3|
Figure 1. ADHD in preschoolers and long-term effects of ADHD pharmacotherapy
A. Criteria for Inclusion/Exclusion of Studies in the Review
For key question one, the population will include children less than 6 years of age with a diagnosis of ADHD or Disruptive Behavior Disorder (including ODD and CD) by DSM or ICD criteria.
For key question two, the population will include subjects of any age who have been treated for ADHD or are a control group of ADHD subjects, diagnosed with ADHD by DSM or ICD criteria.
For key question three, the population will include subjects of any age who have been diagnosed with ADHD or treated for ADHD. Because much of this data will come from cross-sectional, survey and medical databases, using drug treatments and survey symptom checklists to identify ADHD subjects, subjects do not require a DSM or ICD diagnosis for inclusion.
There are no restrictions for study sample size.
Study Design, and Publication types
Full-text reports of clinical trials and comparative observational studies will be included for questions one and two. For question three, we will also include cross-sectional reports.
Letters, editorials, commentaries, reviews, meta-analysis, abstracts, proceedings, case reports, case series, qualitative studies, and theses will be excluded.
Language of Publication
Review of non-English publications will be excluded for this review.
Further search methods
Study authors will be contacted via email for missing outcome or design data.
Reference lists of included papers will be screened for possibly included papers that have not already been screened.
Grey literature will not be searched for this topic.
B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions.
There is no limit to publication date for studies to be included for key question one. Studies will be limited for key question two to any publication from 1997 to 2010 because long-term treatment of ADHD has already been reviewed for earlier dates in a previous systematic review.3 For key question 3, publication dates back to 1980 will be included. EMBASE starts at this date and prevalence analysis will include data from earlier years in the later studies.
The following databases will be searched for key questions one and two: MEDLINE; the Cochrane Library including CENTRAL, EMBASE; PsycInfo, Eric (Education Resources Information Center). For key question three, the Cochrane Library and Eric Database will not be searched because we are not targeting clinical trials. Strategies will use combinations of controlled vocabulary (medical subject headings, keywords) and text words.
Review of reference lists of eligible studies at full text screening will be undertaken. Any potentially relevant citations will be cross-checked within our citation database. Any references not found within the database will be retrieved and screened at full text.
Updating of the search
At the time of submission of the draft peer review report, an updating of our search in all specified databases (see above) will be undertaken.
Incorporation of Public and Peer Review suggestions for literature
Any publications suggested by peer reviewers or from public comment will be documented and verified within our citation database. Any references not included within our citation database will be retrieved and screened at full text.
C. Data Abstraction and Data Management
Relevant fields of information will be extracted from individual studies by trained data extractors using standardized forms and a reference guide. Prior to performing the data extraction, a calibration exercise will be conducted using a random sample of 10 included studies. Key study elements will be reviewed by a second person (study investigator) with respect to study outcomes, seminal population characteristics and characteristics of the intervention. Disagreements will be resolved by consensus.
Abstracted data will include study characteristics (e.g., first author, country of research origin, study design, sample size, clinical indications; and study duration or length of follow-up). Details of the patient population will include but not be limited to age, gender, racial composition, socio-economic status (income, education), co-morbidities (psychiatric and medical histories), Details of the study intervention will include but will not be limited to type of intervention (pharmacological and non-pharmacological) and the comparators, dosage of intervention, duration of follow-up (from immediately post treatment to long term), and characteristics of treatment providers. Characteristics of the outcomes will include the type of instrument or scale, primary or secondary outcome status, type of effect measure (endpoint or change score, measure of variance (standard deviation, standard error, etc), and definition of treatment response.
D. Assessment of Methodological Quality of Individual Studies
We interpret methodological quality to include primarily elements of risk of bias, (systematic error) related to the design and conduct of the study. In addition, we will evaluate the presence of additional biases, such as the funding bias.
We will use a standardized tool, the Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies.11 We will minimize inconsistency amongst raters by providing adequate training for raters and specifying clear decision rules within the standardized instructions.
This tool provides a framework for rating reports on selection bias, study design, confounders, blinding, data collection method and withdrawals and dropouts. We will also evaluate potential biases related to funding sources or conflict of interest.
E. Data Synthesis
For each trial, information on population characteristics (including history of treatment(s), age of first diagnosis, etc), study outcomes (both of benefit and of harm), sample size, settings, funding sources, treatments (type, dose, duration, and provider), methodological limitations, statistical analyses, and any important confounders will be summarized in text and summary tables. We will stratify results based on the diagnosed disorder (ADHD, ODD, CD) and by age (preschool, child, adolescent, adult).
The decision to pool individual study results will be based on clinical judgment with regards to comparability of study populations, treatments, and outcome measures. Specifically, methodological quality (e.g., high-risk of bias vs. low- risk of bias) and clinical diversity (e.g., study population gender, disease severity), treatment (type of intervention) and outcome characteristics (e.g., long-term follow-up vs. short-term follow-up; different measuring scales, different definitions of dichotomous outcomes) of individual studies will be considered. The extent of heterogeneity will be explored through sub-group and sensitivity analyses.
Subgroup and Sensitivity Analysis
There are key patient-specific or intervention-specific factors that may affect the treatment effect and should be explored. Clinical heterogeneity will be assessed by considering any potential differences in participants amongst the trials (e.g., age gender, diagnoses, disease severity, definition of response). Methodological heterogeneity will be explored by evaluating where studies failed criteria.
To maximize the similarities amongst studies that could potentially be combined for meta-analyses, we will further stratify where possible studies based on the 1) behavior disorder (ADHD, ODD, CD), and 2) age categories (preschool, child, adolescent, adult). There are several patient characteristics that we may further explore with sensitivity analyses (if meta-analyses can be undertaken) and these can include the following: 1) disease severity (within ADHD only), 2) gender, 3) co-morbidities related to other psychological disorders. Additionally, if there are sufficient studies we will explore trial specific factors such as 1) duration or dose of intervention, 2) type of treatment provider, and 3) method of defining response. Finally, we will attempt to explore the impact of key methodological study limitations, in particular 1) percent of withdrawals, 2) sample size, and 3) high versus low overall quality.
F. Grading the Evidence for Each Key Question
We will assess the overall strength of the body of the evidence using the GRADE approach.12 There are several factors that may decrease the overall strength of the evidence and these include the following:
- Study limitations (predominately risk of bias criteria)
- Type of study design (experimental versus observational)
- Consistency of results (degree to which study results for an outcome are similar; that is that variability is easily explained, range of results is narrow)
- Directness of the evidence (assesses whether interventions can be linked directly to the health outcomes)
- Precision (degree of certainty surrounding an effect estimate for a specific outcome)
There are factors recommended by the GRADE working group (e.g., burden of therapy, importance of the outcome being evaluated) that may be taken into consideration when assigning a GRADE category.
- Zito JM, Safer DJ, Valluri S, et al. Psychotherapeutic medication prevalence in Medicaid-insured preschoolers. J CHILD ADOLESC PSYCHOPHARMACOL 2007 Apr;17(2):195-203.
- Carlson SR, Iacono WG. Deviant P300 amplitude development in males is associated with paternal externalizing psychopathology. J Abnorm Psychol 2008 Nov;117(4):910-23.
- Jadad AR, Boyle M, Cunningham C et al. Treatment of attention-deficit/hyperactivity disorder. Evidence Report/Technology Assessment No.11. AHRQ Publication No. 00-E005. Rockville, MD: Agency for Healthcare Research and Quality; 1999 Nov.
- Polanczyk G, de Lima MS, Horta BL, et al. The worldwide prevalence of ADHD: a systematic review and metaregression analysis. Am J Psychiatry 2007 Jun;164(6):942-8.
- Simon V, Czobor P, Balint S, et al. Prevalence and correlates of adult attention-deficit hyperactivity disorder: meta-analysis. Br J Psychiatry 2009 Mar;194(3):204-11.
- Brownell MD, Yogendran MS. Attention-deficit hyperactivity disorder in Manitoba children: medical diagnosis and psychostimulant treatment rates. Canadian Journal of Psychiatry - Revue Canadienne de Psychiatrie 2001 Apr;46(3):264-72.
- Rappley MD, Gardiner JC, Jetton JR, et al. The use of methylphenidate in Michigan. Arch Pediatr Adolesc Med 1995 Jun;149(6):675-9.
- Achenbach TM, Dumenci L, Rescorla LA. Are American children's problems still getting worse? A 23-year comparison. J Abnorm Child Psychol 2003 Feb;31(1):1-11.
- Charach A, Cao H, Schachar R, et al. Correlates of methylphenidate use in Canadian children: a cross-sectional study. Canadian Journal of Psychiatry - Revue Canadienne de Psychiatrie 2006 Jan;51(1):17-26.
- Robison LM, Sclar DA, Skaer TL, et al. National trends in the prevalence of attention-deficit/hyperactivity disorder and the prescribing of methylphenidate among school-age children: 1990-1995. Clinical Pediatrics 38(4)()(pp 209-217), 1999 Date of Publication: 1999 1999;(4):209-17.
- Armstrong R, Waters E, Doyle J. Reviews in health promotion and public health. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions: The Cochrane Collaboration, 2008. 21
- Grade Working Group. Grading the quality of evidence and the strength of recommendations. www.gradeworkinggroup.org.
Definition of Terms
ADHD, ODD and CD will be as defined by the version of DSM or ICD current at the time of the study or of the publication.
Summary of Protocol Amendments
To be determined.
NOTE: The following protocol elements are standard procedures for all protocols.
- Review of Key Questions
For Comparative Effectiveness reviews (CERs) the key questions were posted for public comment and finalized after review of the comments. For other systematic reviews, key questions submitted by partners are reviewed and refined as needed by the EPC and the Technical Expert Panel (TEP) to assure that the questions are specific and explicit about what information is being reviewed.
- Technical Expert Panel (TEP)
A TEP panel is selected to provide broad expertise and perspectives specific to the topic under development. Divergent and conflicted opinions are common and perceived as health scientific discourse that results in a thoughtful, relevant systematic review. Therefore study questions, design and/or methodological approaches do not necessarily represent the views of individual technical and content experts. The TEP provides information to the EPC to identify literature search strategies, review the draft report and recommend approaches to specific issues as requested by the EPC. The TEP does not do analysis of any kind nor contribute to the writing of the report.
- Peer Review
Approximately five experts in the field will be asked to peer review the draft report and provide comments. The peer reviewer may represent stakeholder groups such as professional or advocacy organizations with knowledge of the topic. On some specific reports such as reports requested by the Office of Medical Applications of Research, National Institutes of Health there may be other rules that apply regarding participation in the peer review process. Peer review comments on the preliminary draft of the report are considered by the EPC in preparation of the final draft of the report. The synthesis of the scientific literature presented in the final report does not necessarily represent the views of individual reviewers. The dispositions of the peer review comments are documented and will, for CERs and Technical briefs, be published three months after the publication of the Evidence report.
It is our policy not to release the names of the Peer reviewers or TEP panel members until the report is published so that they can maintain their objectivity during the review process.