Medicaid and the Children’s Health Insurance Program (CHIP) provide healthcare coverage, access to comprehensive benefits, and medically necessary services to over 38 million low-income children across the United States.¹ To improve health care quality and outcomes for children, the Pediatric Quality Measures Program (PQMP) was established under the 2009 Children’s Health Insurance Program Reauthorization Act (CHIPRA), in partnership with the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare & Medicaid Services (CMS).² In the first phase of the program, PQMP 1.0 developed a portfolio of evidence-based pediatric quality measures, which were endorsed nationally.^2,3 In the second phase, PQMP 2.0 focused on pediatric measure usability, feasibility, and implementation.4 The PQMP 1.0 and 2.0 initiatives were successful in developing and implementing pediatric quality measures, while also identifying challenges with usability and feasibility in real-world settings.^3-5

For PQMP 3.0, federal and state agencies, health plans, and clinical groups have identified vision, hearing, and developmental screening and followup as areas for pediatric measure development to improve the care of children enrolled in Medicaid and CHIP. The critical decisional dilemma involves identification and development of a parsimonious set of valid and feasible quality measures that can be used to support strategies for improving screening and followup for vision, hearing, and developmental problems in children. The Quality Measures Development Framework outlines the importance of reviewing evidence and identifying gaps in the lifecycle of developing measures.⁶ This technical brief will support this crucial step of mapping existing evidence as part of the lifecycle of pediatric measure development for vision, hearing, and developmental screening and followup.

Children in the United States only receive 40% of needed preventive care.⁷ Early screening for vision, hearing, and developmental delay leads to early identification of problems and timely interventions which can prevent long-term health problems if there is appropriate followup with linkage to treatment.^6,8-10 Despite multiple quality improvement efforts, there are challenges to developing and implementing measures for vision, hearing, and developmental screening in children. These challenges include the age-specific time frames for screening and the gaps in evidence specific to measurement of preventive care delivery in children. When comparing children to adults, the differences in evaluating and implementing quality measures stem from the unique pediatric considerations of developmental change, dependency, differential epidemiology, demographics, and limited financing for child health services.^6,11,12

The goal of this technical brief is to create a map of the evidence on measures for pediatric vision, hearing, and developmental screening and followup that can help to guide the PQMP, with an emphasis on the applicability to children enrolled in Medicaid and CHIP. Given the gaps identified by CMS and Medicaid state agencies in these areas, it is critical to understand: 1) the current state of evidence on existing screening and followup measures for vision, hearing, and developmental problems in children, 2) barriers to development and implementation of these screening and followup measures, and 3) research and evidence gaps for these measures. This evidence review will identify gaps and future research considerations to inform AHRQ, CMS, and Medicaid agencies of the need for refinement, harmonization, or development of measures for pediatric vision, hearing, and developmental screening and followup.

The technical brief will answer the following Guiding Questions:

1. What quality measures have been developed in the following areas for children? If no quality measures have been developed for children, what equivalent quality measures have been developed for adults that could be adapted for use in children?
   • Vision screening
   • Vision screening followup
   • Hearing screening
   • Hearing screening followup
   • Developmental screening
   • Developmental screening followup
     1. What is known about the reliability, validity, usability, and feasibility of screening and followup measures for vision, hearing, and development in children?
2. What studies have assessed whether specific clinical tools or implementation practices for such quality measures of screening in children are associated with differences in the targeted quality metrics or related patient outcomes (i.e., related to vision, hearing, or development)?
   1. Have any of these studies shown improvements in quality of care, improvements in health outcomes, or decreased disparities (by race/ethnicity, sex, insurance type, or socioeconomic status) in quality of care or health outcomes?
   2. How has the use of screening and followup measures for vision, hearing, and development in children differed by states or at other levels (e.g., targeted population, type of payor/health plan, type of institution/system/hospital, or type of clinician)
   3. If no such studies of children are available, what studies have assessed whether specific equivalent clinical tools or implementation practices for quality measures of screening in adults are associated with differences in the targeted quality metrics or related patient outcomes?
   4. If such research evidence is not available for children or adults, are there guidelines or frameworks to suggest whether specific clinical tools or implementation practices for quality measures of screening are associated with differences in the targeted quality metrics or related patient outcomes?
3. What is known about barriers to development and implementation of screening and followup measures for vision, hearing, and development in children?
   1. Are there broader or more generalizable barriers related to the development and implementation of screening measures or followup measures?
4. What are the evidence gaps and future research needs related to screening and followup for problems in vision, hearing, and development in children?

1. Data Collection

A. Discussion with Key Informants

We will identify Key Informants with experience and expertise across the spectrum of domains associated with pediatric quality measures for: vision screening and followup; hearing screening and followup; and developmental screening and followup. We will include Key Informants representing experts in the development and evaluation of quality measures for screening in children, experts in application and use of such quality measures, state Medicaid program directors, societies of healthcare professionals, and other stakeholders such as health systems, governmental agencies, nongovernmental organizations interested in pediatric quality of care, as well as patient advocates.

We will send information about the project to the Key Informants to facilitate efficient discussion. We will conduct virtual meetings with the Key Informants to solicit their input on our approach to addressing the Guiding Questions about quality measures for vision, hearing, and developmental screening in children as well as related tools and implementation practices. Specifically, we will cover the questions listed below during the Key Informant meetings:

1. Do you have any questions or concerns about the clarity of the Guiding Questions?
2. What are the most commonly used types of screening tests or interventions for which we should look for quality measures?
3. What are the most prominent guidelines or frameworks that influence pediatric screening for vision, hearing, and developmental problems, aside from guidelines from the American Academy of Pediatrics (AAP)^13-16 and the United States Preventive Services Task Force (USPSTF)?^10,15,16
4. What are the best sources for finding evidence about how measures of the quality of pediatric screening for vision, hearing, and developmental problems differ across states or at other levels, such as by targeted population, type of payor/health plan, type of institution/system/hospital or type of clinician?

Additionally, Key Informants will be engaged individually after the meetings, as appropriate.

We will prepare a summary of the Key Informants’ comments to share with the representatives from AHRQ and CMS and with the authors of the technical brief. The feedback on the clarity of the Guiding Questions will help to guide how we approach and present the evidence to ensure that the questions are appropriately addressed in our final report. The feedback will also inform the choice of sources to be included in scanning the gray literature, for example, if Medicaid directors are aware of white papers related to this topic. Also, we will use the feedback to help identify guidelines or frameworks that suggest an association between specific screening tools or implementation practices and the targeted quality metrics or related outcomes in children.

B. Published Literature Search

We will search PubMed, Embase, CINAHL, Cochrane Central Register of Controlled Trials, and the American Psychological Association’s (APA) PsychINFO from 2009 to present, based on the year that the PQMP program was established. We will also hand search the included studies of previous relevant reviews and consult with our Key Informants and internal advisors to ensure that we capture a comprehensive set of studies. A preliminary search strategy is presented in Appendix A.

C. Grey Literature Search

Targeted gray literature searching will be conducted using search terms similar to those employed for the published literature search using LexisNexis and other websites to identify reports from governmental and non-governmental sources including: CMS, AHRQ, State Health Departments, Centers for Disease Control and Prevention (CDC) - Early Hearing Detection and Intervention Group, National Committee for Quality Assurance (NCQA), National Quality Forum (NQF), Battelle Partnership for Quality Measurement, Accreditation Association for Ambulatory Health Care, Kaiser Permanente, Academy Health, AAP, American Academy of Family Physicians (AAFP), National Institute for Children’s Health Quality, Child and Adolescent Health Measurement Initiative’s (CAHMI) Data Resource Center for Child and Adolescent Health, Maternal Child Health Bureau of the Health Resources and Services Administration, Utilization Review Accreditation Commission, Joint Committee on Infant Hearing, American Academy of Audiology, American Association for Pediatric Ophthalmology and Strabismus, National Improvement Partnership Network, and Mathematica. We will also search for relevant on-going research by using clinicaltrials.gov and by querying advisors as well as the Key Informants.

2. Data Organization and Presentation:

A. Information Management

We anticipate finding a large amount of published literature on this topic. We will first screen systematic reviews identified by our search strategies. We will define eligibility criteria, using refined Population, Intervention, Comparisons, Outcomes, Timing, and Setting criteria (see Table 1) individualized to the Guiding Questions.

Defining Eligibility Criteria (PICOTS)

Population: We will include children and adolescents less than or equal to 18 years of age, without signs or symptoms of delay or disorders. Our pediatric population will be defined by those age groups in the AAP/Bright Futures recommendations for preventive pediatric healthcare which specify ages for vision, hearing, and developmental screening (see Appendix B). Vision screening is recommended at specific ages in early childhood (at 3 and 4 years), middle childhood (at 5, 6, 8 and 10 years), and adolescence (at 12 and 15 years). Hearing screening is recommended at the newborn period and at specific ages in early childhood (at 4 years), middle childhood (at 7 and 9 years), and adolescence (once during each age range of 11-14 years, 15-17 years, and 18-21 years). Developmental screening is recommended in infancy (at 9 months) and early childhood (at 18 and 30 months), with screening for autism spectrum disorder in early childhood (at 18 and 24 months).

Interventions: We plan to include studies with measurement of screening and followup interventions for vision, hearing, or developmental problems. We are defining developmental screening as screening for developmental disabilities. We did not include screening for behavioral, emotional, and social needs in our definition of developmental screening because this would exponentially increase the scope of this review and not be feasible within the time limitations and resource constraints of this task order. We plan to exclude studies of screening tools that do not cover vision, hearing, or developmental problems. For Guiding Question 2, we will include studies evaluating whether specific clinical tools or implementation practices for quality measures of screening in children are associated with differences in the targeted quality metrics or related patient outcomes (i.e., related to vision, hearing, or development)?

Comparisons: We plan to include studies that have comparisons of different interventions or measures, or pre-post comparisons of a single intervention or measure implemented over time. For Guiding Question 1, we will also seek to identify studies that report on relevant quality measures without having a comparison group.

Outcomes: We will look for studies that report on the following outcomes of interest: quality measures and the reliability, validity, usability, or feasibility of the measures (for Guiding Question 1); health care process measures and clinical outcome assessments that can be used as quality indicators for measures such as identification and treatment of vision, hearing, or developmental problems, or clinical measures of vision, hearing, or development, or developmental and educational outcomes (for Guiding Question 2); disparities in quality of care or health outcomes by race, ethnicity, sex, insurance type, or socioeconomic status (for Guiding Question 2a); differences in outcomes by state or level such as targeted population, type of payor/health plan, type of institution or system or hospital, or type of clinician (for Guiding Question 2b); guidelines and frameworks that suggest associations between specific screening tools or implementation practices and the targeted quality metrics or related outcomes in children (for Guiding Question 2d); reported barriers to development and implementation of the measures of interest (for Guiding Question 3); and reported evidence gaps and future research needs (for Guiding Question 4).

Timing: We will include studies published in 2009 or later, based on the year that the PQMP program was established. 

Setting: We will exclude studies conducted outside of the United States, because they would be less relevant to the charge of the PQMP.

Study design: We are interested in including evidence from original studies with any design that helps to address a Guiding Question, but we will start by searching for relevant systematic reviews because the volume of studies on this topic is so large. We will exclude editorials, letters, commentaries, non-systematic reviews, and studies not written in English.

We plan to summarize recent good quality systematic reviews that address the Guiding Questions. For relevant systematic reviews, we will assess their quality and extract information that applies to the Guiding Questions. We will use the criteria developed by the USPSTF Methods Workgroup¹⁷ for assessing the quality of systematic reviews:

• Good - Recent relevant review with comprehensive sources and search strategies; explicit and relevant selection criteria; standard appraisal of included studies; and valid conclusions.
• Fair - Recent relevant review that is not clearly biased but lacks comprehensive sources and search strategies.
• Poor - Outdated, irrelevant, or biased review without systematic search for studies, explicit selection criteria, or standard appraisal of studies.

We will conduct a primary literature search to capture studies published after the included systematic review search dates, and capture studies addressing populations, interventions, or outcomes not included in the relevant recent systematic reviews.

For primary studies not covered in the systematic reviews, we will assess the risk of study bias in the following manner. When assessing randomized controlled trials (RCTs), we will use the seven items in the Cochrane Collaboration’s tool that cover the domains of selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias.¹⁸ When assessing non-randomized studies, we will use specific items in the ROBINS-I tool (Risk Of Bias In Non-randomised Studies – of Interventions) that assess bias due to confounding, bias in selection of participants into the study, bias in classification of interventions, bias due to deviations from intended interventions, bias due to missing data, bias in measurement of outcomes, and bias in selection of the reported results.¹⁹ The risk of bias assessments will focus on the main outcome of interest in each study.

We will use DistillerSR for title/abstract screening and full text screening. We will create customized data extraction forms within DistillerSR to capture information from the included systematic reviews and primary studies. We will upload all extracted data from the included systematic reviews and primary articles to the Systematic Review Data Repository (SRDR+).

B. Data Presentation

We will create a table of data for each of the three screening topics (vision, hearing, and developmental screening and followup). The tables will identify: study characteristics (e.g., study inclusion years, topic area [vision, hearing, or developmental screening], and eligibility criteria); the quality measures developed; whether the quality measures are associated with change-related metrics or related patient outcomes or decreased disparities by race/ethnicity, insurance type, or socioeconomic status; metrics for reliability, validity, usability, and feasibility; and barriers to development or implementation.

We will use the extracted information to identify differences in screening and followup measures by state and other levels such as target population, type of payor, type of institution (e.g., daycare, pre-school, and school settings), or type of clinician. We will also use this information to identify the gaps in screening and followup measures in children and will supplement these gaps with what is known about equivalent issues in an adult population. If there is no information for pediatric or adult populations, we will look for guidelines or frameworks that should be considered when developing or implementing a quality measure in this area. Frameworks suggesting an association of implementation practices with quality metric outcomes will be identified and described.

We will work with a data visualization expert to create graphical displays of the collected data. For example, for Guiding Question 2, various vision screening tools and implementation practices for vision screening could be compared with respect to their reported level of benefit or impact on different types of outcomes. Bubbles of different sizes could be used to relay the number of studies reflecting certain outcomes, or different shading could reflect the type of population (children or adults) or setting. The visual mapping of the evidence will enhance understanding of  trends, tendencies, or patterns in data.

1. December 2023 Medicaid & CHIP Enrollment Data Highlights. Medicaid.gov; 2023. Accessed on April 24 2024.
2. Mangione-Smith R, Schiff J, Dougherty D. Identifying children’s health care quality measures for Medicaid and CHIP: an evidence-informed, publicly transparent expert process. Academic pediatrics. 2011;11(3):S11-S21.
3. Mistry KB, Chesley F, LLanos K, et al. Advancing children's health care and outcomes through the pediatric quality measures program. Academic pediatrics. 2014;14(5):S19-S26.
4. Mistry KB, Sagatov RD, Schur C, et al. Design and implementation of the pediatric quality measures program 2.0. Academic pediatrics. 2022;22(3):S59-S64.
5. Schur C, Johnson M, Doherty J, et al. Real-World Considerations for Implementing Pediatric Quality Measures: Insights From Key Stakeholders. Academic pediatrics. 2022 Apr;22(3s):S76-s80. doi: 10.1016/j.acap.2021.04.007. PMID: 35339247.
6. Adirim T, Meade K, Mistry K. A New Era in Quality Measurement: The Development and Application of Quality Measures. Pediatrics. 2017 Jan;139(1). doi: 10.1542/peds.2016-3442. PMID: 28025242.
7. Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. The New England journal of medicine. 2007 Oct 11;357(15):1515-23. doi: 10.1056/NEJMsa064637. PMID: 17928599.
8. Lipkin PH, Macias MM. Promoting Optimal Development: Identifying Infants and Young Children With Developmental Disorders Through Developmental Surveillance and Screening. Pediatrics. 2020 Jan;145(1). doi: 10.1542/peds.2019-3449. PMID: 31843861.
9. Year 2007 position statement: Principles and guidelines for early hearing detection and intervention programs. Pediatrics. 2007 Oct;120(4):898-921. doi: 10.1542/peds.2007-2333. PMID: 17908777.
10. Grossman DC, Curry SJ, Owens DK, et al. Vision screening in children aged 6 months to 5 years: US preventive services task force recommendation statement. Jama. 2017;318(9):836-44.
11. Stille C, Turchi RM, Antonelli R, et al. The family-centered medical home: specific considerations for child health research and policy. Academic pediatrics. 2010 Jul-Aug;10(4):211-7. doi: 10.1016/j.acap.2010.05.002. PMID: 20605546.
12. Raphael J, Sadof M, Stille C, et al. Not Just Little Adults: Considerations for Quality Measures of Child Health Care. Agency for Healthcare Research and Quality, US Department of Health & Human Services Updated April. 2014;7.
13. Hagan JF, Shaw JS, Duncan PM. Bright futures: Guidelines for health supervision of infants, children, and adolescents: Pocket guide. (No Title). 2017.
14. Donahue SP, Nixon CN. Visual System Assessment in Infants, Children, and Young Adults by Pediatricians. Pediatrics. 2016 Jan;137(1):28-30. doi: 10.1542/peds.2015-3596. PMID: 29756730.
15. Bower C, Reilly BK, Richerson J, et al. Hearing assessment in infants, children, and adolescents: recommendations beyond neonatal screening. Pediatrics. 2023;152(3).
16. Lipkin PH, Macias MM, Norwood KW, et al. Promoting optimal development: identifying infants and young children with developmental disorders through developmental surveillance and screening. Pediatrics. 2020;145(1).
17. Force UPST. US Preventive Services Task Force Procedure Manual: appendix VI: criteria for assessing internal validity of individual studies. Published 2017.
18. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. Bmj. 2011;343.
19. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. bmj. 2016;355.

Table 1: Populations, Interventions, Comparison, Outcomes, Timing, Setting, and Study Design

*We will include quality measures as defined by the authors of the included studies  for vision screening and followup, hearing screening and follow up, and developmental screening and followup.

Appendix B. American Academy of Pediatrics Bright Futures Recommendations for Preventive Pediatric Healthcare