Comparison of PEDS Tools and ASQ Tools

Common Measurement Options in Early Detection of Developmental-Behavioral (including Social-Emotional and Mental Health) Problems in Young Children

State Trends, Policy and Commonly Selected Measures

Trends Across States

Most States, via advocacy from professional organizations such as the National Association of State Health Policy (in association with ABCD,) and societies such as Chapters of the American Academy of Pediatrics and the National Association of Pediatric Nurse Practitioners, created a list of accurate developmental-behavioral screening tools, approved by the Centers for Medicaid and Medicare and (eventually) endorsed by private payers. Professionals deploying approved tools usually qualify for reimbursement under the 96110 billing code or enhanced reimbursement if in Federally Qualified Health Centers.

Over a decade ago, a few States approved only a single tool for use by providers. This proved unpopular because professionals often have measurement preferences. Some prefer parents’ report of children’s skills, others prefer capitalizing on parents’ concerns, while some prefer hands-on measures in which professionals elicit skills directly from children. Wisely, these States eventually expanded their lists to ensure that providers in various disciplines had a range of choices and thus measurement options.

Rationale and American Academy of Pediatrics’ Policy Highlights

State’s recommendations for use of accurate measures are rooted in the evidence provided by quality instruments, and the evidence that informal approaches (e.g., ad hoc milestones, informal questions to parents, etc.) fail to detect the majority of children with problems-- abrogating young children’s opportunities for beneficial early intervention. State policy is also informed by the many early detection policies of the AAP, i.e., statements on early detection of developmental problems, social-emotional/behavioral and mental health problems, autism spectrum disorder, and most recently motor disorder. These policies coalesce into the following:

• Detecting and addressing parents’ concerns (every encounter)
• Measuring milestones/skills across all domains of development, including language, motor, (pre-) academics, self-help, behavior, social emotional/mental health (every well-visit)
• Using accurate screens at targeted well-visits up to 30 months, with strong recommendations for evidence-based screening at subsequent, annual well-visits
• Screening for autism spectrum disorders (ASD) (at 18 months and again at 24 months)
• Maintaining a current family/child medical history (all visits) and psychosocial status (at periodic well-child visits)
• Providing parent/patient education (every visit in which concerns arise)
• Timely referral to needed services plus ongoing monitoring of outcomes and referral uptake

Commonly Selected Measures: Descriptions, Research Highlights and Typical Settings

Because clinicians, whether medical or non-medical professionals, need several tools to accomplish the above tasks, and because most States’ lists overlap, an analysis of the commonly used screening batteries are presented: PEDS Tools (meaning PEDS+PEDS:DM), and ASQ Tools (meaning ASQ-3+ASQ:SE). Both sets of measures call for use of an ASD-specific measure such as the Modified Checklist of Autism in Toddlers (M-CHAT).

The tables below provide descriptions, purpose, age range, types of scores, and costs. The details within were vetted by numerous screening test researchers, approved by the authors of the tools described and published in various texts and periodicals. Research findings were drawn from research literature databases, test manuals, and publishers’ websites. After each table is an expanded description of each battery including the benefits, challenges, other research highlights, and common settings.

Costs

Prices for measures cover the purchase of an English version. Prices are comparable (although sometimes less) for a Spanish version or for other languages. Material cost per use is shown in the far right column and includes purchased protocols or photocopying (when permissible). Administration, scoring and interview time were assigned a value of $60 per hour (because professionals as well as staff are often involved). The time and expense of interpreting results to families is not assigned a value because it is consistent across all measures.

In general, material costs whether protocols are purchased or photocopy-able are much lower than the practice expense for staff/professional time. The longer the measure, the greater is the practice expense.

©2013, adapted from “Identifying and Addressing Developmental-Behavioral Problems: A Practical Guide for Medical and Non-Medical Professionals, Trainees, Researchers and Advocates.” www.pedstest.org. Permission is granted to reproduce the information in this document as long as this copyright notice is shown.

©2013, adapted from “Identifying and Addressing Developmental-Behavioral Problems: A Practical Guide for Medical and Non-Medical Professionals, Trainees, Researchers and Advocates.” www.pedstest.org. Permission is granted to reproduce the information in this document as long as this copyright notice is shown.

Other Research Highlights and Settings for PEDS Tools:

Benefits:

PEDS:DM and PEDS (with its emphasis on parents’ verbatim concerns), are known to:

• Facilitate parents’ willingness to follow through with recommendations;
• Reduce disruptive “oh by the way” concerns enabling appointment schedules to remain within expected time frames;
• Improve parent satisfaction with care;
• Engender parent-professional communication and shared decision-making;
• Alert parents that developmental-behavioral topics are part of care;
• Assist in discerning disorder (e.g., a children with an age-appropriate 3-word length of utterance but who simply repeats “Wheel of Fortune” over and over);
• Be suitable for interview administrations (e.g., over the phone or when parents’ have literacy challenges);
• Predict future problems (e.g., children at 12 months who receive a diagnosis at 36 months of ASD; kindergarten children found to have academic and other deficits by 2^nd grade);
• Measure developmental as well as behavioral/social-emotional/mental health problems;
• Facilitate delivery of difficult news (via affirmation of parents’ concerns);
• Help parents view development as professionals do—as a range of domains;
• Identify when to refer to special education services, versus Head Start/Early Head Start/quality day care, versus provide parent education with monitoring of effectiveness and developmental-behavioral status;
• Provide (in the case of PEDS) an essential literacy check to make sure families understand the questions asked;
• Include guidance on parent training, parent information handouts and links to parent training programs;
• Capture parents’ concerns about a range of health issues;
• Offer, when the M-CHAT is added, basic compliance with AAP policies, and stronger compliance when supplementary measures included in the PEDS:DM (e.g., measures of psychosocial risk and resilience) are included.

Challenges:

• As with a few ASQ studies, some research on PEDS suffered from problematic administration. In the case of PEDS, the common error is the failure of researchers to re-administer PEDS by interview if little or nothing is written on the Response Form;
• Ad hoc translations (e.g., those used in the NSECH studies of Survey PEDS), rather than those tested by PEDS researchers using guidelines from the International Test Commission suffer from problematic response rates;
• PEDS scoring (by hand) requires professionals to understand the various domains of development so that types of concerns can be categorized, i.e., PEDS should not be scored by non-professionals. Nevertheless, PEDS Online provides automated scoring enabling naïve staff to use PEDS effectively;
• PEDS calls for a second screen for two of its decision-paths, meaning that the PEDS:DM is needed in about 30% to 40% of cases. Nevertheless, use of both tools involves only 16 – 18 items;
• As with the ASQ Tools, use of the M-CHAT for optimal detection of possible ASD is encouraged;
• PEDS Tools perform better when used as an interview (which is the most common administration method in health care settings).

Other research Highlights

• PEDS and the PEDS:DM were standardized on a total of 49,150 families. Both measures were normed in English and Spanish. Families in 38 US States (and Canada) participated;
• Both PEDS Tools were standardized on nationally representative population reflective of current Census Bureau population parameters including ethnicity, parents’ level of education (e.g., 16% had not completed HS), language spoken at home, gender, poverty levels, etc.;
• Reliability studies for PEDS and PEDS:DM embraced 747 children and revealed test-retest agreement of 96%, inter-rater agreement between professionals of 94%, and (Kappa = .81) for inter-rater agreement on the PEDS:DM. Although PEDS prompts professionals to add their own concerns, PEDS is not normed for professionals’ concerns without including those of parents;
• Validity including accuracy studies were conducted on 6,092 children using a range of diagnostic and assessment-level measures. Summary figures for both measures are sensitivity of 85% and specificity of 77%. Note: More than 70% of over-referrals on PEDS Tools are children performing below average, who have multiple psychosocial risk factors, but who are not eligible for IDEA services, i.e., other kinds of help are still needed.

Typical Settings

• Although PEDS Tools were normed in many settings including health care, Head Start, public schools and day care/preschools, the measures are used most often in primary care, crisis call centers, and survey studies;
• The PEDS:DM Assessment Level is more commonly used in NICU/subspecialty follow-up clinics and for Early Intervention intake because it provides age-equivalent scores useful for eligibility determination and progress monitoring.

©2013, adapted from “Identifying and Addressing Developmental-Behavioral Problems: A Practical Guide for Medical and Non-Medical Professionals, Trainees, Researchers and Advocates.” www.pedstest.org. Permission is granted to reproduce the information in this document as long as this copyright notice is shown.

Other Research Highlights and Settings for ASQ Tools:

Benefits

ASQ Tools:

• Offer a range of cutoffs tied to 2.0, 1.5, and 1.0 standard deviations below the mean for which children performing at the latter two cutoffs are identified as in need of monitoring and provided with follow-up activities;
• Have an optional ASQ activities kit to designed to help parents promote development (and help educators with initial intervention planning);
• Because there are multiple items per domain, experienced professionals can potentially discern strengths and weaknesses within skill areas;
• Provide a helpful video about using the ASQ on a home visit;
• Help with immediate instructional planning;
• As with PEDS Tools, ASQ Tools are known to improve parents’ understanding of child development;
• Offer, when the M-CHAT is added, basic compliance with AAP policies;
• Elicit parents’ concerns about vision, hearing and health and overall appraisals of developmental-behavioral status;
• The Spanish language translation is much improved as compared to the ASQ-2
• As with PEDS Tools, the ASQ Tools provide templates for referral letters, consent forms, and take-home parent summary reports (in English and Spanish);

Challenges

• Because the ASQ-3 does not measure behavioral/social-emotional/mental health skills, the ASQ:SE must be administered to capture these domains or whenever ASQ-3 results are problematic;
• The length of each measure, 30 – 35 per screen, and the absence of a complete interview administration option, make ASQ Tools less than suitable for interview administration (often needed in health care and telephone services), in survey studies, etc. Many health care providers find ASQ Tools too lengthy for routine use in primary care;
• As with some PEDS Tools studies, some research on ASQ suffered from problematic administration. In the case of the ASQ, the common error is the failure to have parents self-administer items to children (but instead to rely on professional judgment about skills);
• Does not identify what other services could be helpful to those who score in the monitoring zone;
• Does not include measures of psychosocial risk and resilience;
• Although overall readability (when response options are included) is low, many individual items require reading skills at the 10^th grade level and higher) as evaluated by http://www.readability-score.com Nevertheless, experienced users of ASQ Tools learn which items are particularly challenging and work through these with parents;
• Does not provide psychosocial risk and resilience measures for full compliance with AAP screening and surveillance policies;
• When used in a mail-out program, return rates are less than 35% for high risk families;
• Requires a materials kit (unless, in an at-home self-administration, parents can be counted on to have needed stimuli);
• Does not offer an evidence-based interpretation of parents’ concerns;
• Problems found on the ASQ:SE were more associated with parental distress than with children’s performance via clinical observation.

Other research Highlights on ASQ Tools

• The ASQ Tools were normed on 18,152 children in 50 US States and territories. Both measures were standardized in English and Spanish.
• Although ethnicity matched US Census population estimates, only 6% of families had not completed HS.
• Validation/accuracy studies were conducted on 579 children who were administered both the ASQ and the Battelle Developmental Inventory (to which was applied eligibility criteria for IDEA enrollment).
• Test-retest reliability ranges from 75% - 94%. Inter-rater agreement was not tested for the ASQ:SE but reaches 93% for the ASQ-3;
• The ASQ-3 enjoys predictive validity studies showing that problematic performance years earlier is associated with subsequent problematic performance. To date, there are not predictive validity studies for the ASQ:SE.

Typical Settings

The ASQ Tools are most commonly used by non-medical professionals, i.e., in Head Start, public schools, day care/preschools. In some States ASQ Tools are used for IDEA intake. The presence of educational activities makes the ASQ Tools particularly useful in settings where instruction or instructional planning will occur. Although some health care providers use ASQ Tools, these are typically deployed at a few targeted visits and not routinely, due to length and need for materials.

Summary

PEDS Tools enjoy far more psychometric support than ASQ Tools and are less expensive to use, but both sets of measures are shown to be equally accurate and both enjoy substantive outside research (although not all studies were conducted with fidelity to scoring guidelines and thus some research has questionable results). Nevertheless, a comparison of PEDS Tools and ASQ Tools is, to some extent, “apples to oranges”. The measures were developed for different purposes. Although both serve well in early identification, there are decided preferences across professional disciplines about which tools to use. PEDS Tools are designed not only to screen but also to focus on the routine aspects of well-visit encounters—eliciting and addressing parents’ concerns across domains, replacing informal milestones checklists found on age-specific encounter forms with evidence, and providing evidence-based triage for the many decisions health care must make—and all in a manner that leaves time for one of the most important tasks of health care—advising parents and making referrals. PEDS Tools are far more suitable for interview and telephone administrations, enabling clinicians and researchers to circumvent literacy and language barriers (e.g., when using PEDS Tools in health care, with telephone foreign language interpretation service, in survey studies). PEDS Tools also have the advantage of screening for problems with behavior and social-emotional/mental health within each measure.

The PEDS:DM Assessment levels is sometimes used in IDEA intake because it provides scores across all domains necessary for determining percentage of delay. Nevertheless, the PEDS:DM Assessment Level is more commonly used in medical subspecialty follow-up clinics (e.g., Neonatal Intensive Care, child abuse, rare diseases outcomes studies, etc.). ASQ Tools, in contrast with PEDS Tools (in their screening version but not with Assessment version), offer greater detail about children’s performance and the potential for experienced professionals to view strengths and weaknesses within domains. The availability of supplementary materials for ASQ Tools (e.g., the video, “ASQ on a Home Visit”, “ASQ-3 Learning Activities”) offer substantial support for educators wishing to use ASQ Tools to aid in instructional planning.

References

Test Manuals and Publishers’ Websites

Glascoe FP. Collaborating with Parents: Using Parents’ Evaluation of Developmental Status (PEDS) to Detect and Address Developmental and Behavioral Problems. Nolensville, TN: PEDStest.com, LLC, 2013.
www.pedstest.com

Glascoe FP, Robertshaw NS. PEDS Developmental Milestones: A Tool for Surveillance and Screening (Professionals’ Manual). Nolensville, TN: PEDStest.com, LLC, 2008.
www.pedstest.com

Squires J, Bricker D, Twombly E. Ages & Stages Questionnaires: Social-Emotional (ASQ-SE).
Baltimore: Paul H. Brookes Publishing Company, 2002.
www.pbrookes.com

Squires, J., & Bricker, D. Ages & Stages Questionnaires, Third Edition (ASQ-3). Baltimore, MD: Brookes Publishing, 2009.
www.agesandstages.com