PEDS:DM^® Validation

PEDS^®: Developmental Milestones (PEDS:DM^®) Psychometrics

In 2006, the American Academy of Pediatrics revised its recommendations for early detection of developmental problems to embrace not only validated screening but also developmental/behavioral surveillance, defined as the careful accretion of “big-picture” information about multiple dimensions of child and family health, well-being and functioning, and intervening promptly with both existing and potential problems. Repeated screening with accurate measures embracing parents’ concerns (Parents’ Evaluation of Developmental Status – PEDS^®) and children’s evolving milestones (PEDS:DM^®) is the foundation of surveillance.

To summarize the issues of screening and surveillance:

1. Repeated screening using accurate tools is fundamental to surveillance and early detection;
2. Additional separate surveillance measures (e.g., of parents’ mental health, psychosocial risk and resilience) should also be evidence-based; and
3. Screening and surveillance should render decision-support to guide clinicians toward appropriate next steps, i.e., action plans that indicate whether to refer, screen further, advise parents, and monitor outcomes.

To address the above, the PEDS:DM^® Screening Version was created. The goals of the PEDS:DM^® are to:

1. Support AAP policy on early detection;
2. Provide an accurate measure of milestones with definitive scoring criteria (i.e., replace informal checklists);
3. Ensure that other aspects of screening/surveillance are included in the PEDS:DM^® longitudinal reporting forms, i.e., PEDS^® with its longitudinal view of parents’ concerns and how to address them with evidence;
4. Indicate when other surveillance measures are needed (e.g., assessment of psychosocial risk and resilience factors);
5. Offer clear guidance to providers on how best to respond to results of screening/surveillance measures.

PEDS:DM^®–Assessment Level (PEDS:DM^®–AL) is designed for:

1. In-depth triage within developmental-behavioral subspecialty clinics
2. Outcomes research and longitudinal monitoring
3. NICU and other subspecialty follow-up clinics
4. Early Intervention Child Find, i.e., determining eligibility for services under the Individuals with Disabilities Education Act (IDEA).

The PEDS:DM^®–AL measures the following domains: Fine Motor, Gross Motor, Receptive Language, Expressive Language, Self-Help, Social-Emotional, Academic/Pre-academic/school readiness; and Cognitive skills. The PEDS:DM^®–AL provides age-equivalent scores for each domain. From these scores, the percentage of delay (or more positively) the percentage of skills mastered, can be readily calculated.

The PEDS:DM^®–AL can also be used to determine whether children’s development is advanced, thus enabling a view of both strengths and weaknesses.

The PEDS:DM^®–AL can be administered hands-on by parents or by professionals. Many items, especially those for younger children, can be administered by either parent-interview or by observation. The PEDS:DM^®–AL is also useful in training emerging professionals such as residents, fellows, nurse-practitioners, educators, speech-language pathologists and psychologists.

Research Supporting the Addition of a Cognitive Score for the PEDS:DM^® – Assessment Level (PEDS:DM^®–AL)

Discriminant validity studies in the 2010s lead to the creation of a Cognitive Score for the PEDS:DM^®–AL. This is valuable because most IDEA Part C programs determine eligibility for services based on deficits in multiple domains including cognitive performance (e.g., 25% – 40% delay). Via a discriminant function analysis grouping children with intellectual/developmental quotients < 93 (< 25 percentile) versus 93 (26th percentile) or higher, the relationship between cognitive status and a range of reference standard measures was significant [c 2(6) = 33.221, p < .0001]. Table 1 shows the PEDS:DM^® domains contributing to detection of cognitive performance and that performance on fine motor, expressive language, receptive language and self-help skills differentiated the two groups. Gross Motor and Social-emotional skills were non-contributing.

Comment on the PEDS:DM^®–AL Cognitive Score. As a consequence of the above findings, the combination of scores in Fine Motor, Expressive Language, Receptive Language and Self-help skills are used to generate an overall Cognitive Score. The PEDS:DM^®–Assessment Level test booklets, scoring guidelines within the revised PEDS:DM^® Manual, and the automated scoring for PEDS Online^® provide methods for generating the Cognitive Score.

Documentation, Coding and Billing for PEDS^® and PEDS:DM^® in Print

Modify the appropriate preventative service visit or evaluation and management code by –25 (or -59, checking first with payers on which to use) and add 96110 to indicate that screening was performed in addition to usual services. If giving both PEDS^® and the PEDS:DM^®, code 96110  x 2 to show that two screens were administered. See Chapter 1 for additional details on billing and coding.

If PEDS^® together with the PEDS:DM^® leads to patient education, you can also use:

Below are several ICD-10 diagnosis codes often used for problematic screening results. These were selected so as not to interfere with subsequent assessments and diagnostic findings:

Billing and Coding for the PEDS:DM^®—AL in Print

Because the PEDS:DM^®-Assessment Level is more than a screening tool, use 96111[X (times)]. If screens within the Family Book are administered, also use 96110 [X (times)] the number of screens administered). Note: Reimbursement for 96110 may not be possible if billing 96111 on the same day, but screens (e.g., MCHAT-R) could be administered during pre-appointment telephone intake or after the visit.

Because the Assessment Level PEDS:DM^® is neither a diagnostic instrument screen nor a screen, only provisional ICD-10 codes should be used for diagnoses (ones that will not interfere with more refined diagnostic work ups, such as speech-language evaluations, developmental psychology testing, neurological exams, etc.). The following codes are both sufficiently precise but still general:

PEDS Online^® – Administration, Billing and Coding

The web-based service includes both Screening and Assessment Level versions in English and Spanish at www.pedstest.com/online (for the Screening Version) and www.pedstest.com/assessment (for the Assessment Level Version).

• Online version of the PEDS:DM^®, together with an optional PEDS^® and Modified Checklist of Autism in Toddlers – Revised (M-CHAT-R). The site provides automatic scoring, instantly generated referral letters, parent take-home summary reports, ICD-10 and procedure codes for documentation and billing, and produces a database for progress tracking.
• PEDS Online^® can be administered in multiple ways: having families complete items at home before a visit, on paper with responses keyed in by professionals by live interview, or on waiting room computers.
• See www.pedstest.com/online to trial the web-based service and for information on integration with Electronic Records

PEDS:DM^® Psychometrics: Summary of Findings

Summary of Standardization Research

• The PEDS:DM^® is normed on a large sample (N = 19,607) of children throughout the US.
• Children ranged from birth to 8 years of age, with > 200 children per each cutoff score.
• Sites included health care clinics as well as day care, i.e., settings where screening tests are most likely to be used and where the percentage of children with likely or known disabilities is not disproportionately high [as compared to Centers for Disease Control  (CDC) prevalence indicators].
• The sample’s demographic characteristics are generally representative of US Census Bureau population parameters although levels of education are lower and ethnic/racial diversity higher than current prevalence figures.
• Nevertheless, the sample is characteristic of US Census Bureau projections for 2025 – 2035.
• Families who spoke only Spanish or other non-English languages were included in the sample (17%).
• Performance on the PEDS:DM^® is responsive to psychosocial risk factors correlated with developmental-social-emotional delays, i.e., parents with limited education, families living in poverty, or who have difficulty speaking English.
• Increased age is a known risk factor — the older the child the more likely the presence of delays per escalations in incidence rates shown by Individuals with Disabilities Act (IDEA) services (www.ideadata.org) and also disability prevalence per the CDC, www.cdc.gov). Performance on the PEDS:DM^® shows increasing rates of delays with age.

Summary of Reliability/Stability Research

• The PEDS:DM^® has strong test-retest reliability (> 90%) across repeated screens whether administered by the same examiner or by others within a 2 to 4 week interval.
• Inter-method reliability is high (92%) meaning that administration by parent self-report, interview, versus hands-on are interchangeable and produce nearly identical results.
• Stability performance over time (an average interval of 4 months) is stable (> 90%) despite differences in PEDS:DM^® Forms used and in the examiners who administered them. This means that children with unmet milestones in the past tend to continue on the same troubling trajectory while children who met milestones in the past, tend to progress in the same manner.
• Internal consistency is high: The various domains of the PEDS:DM^® are relatively unassociated with each other, meaning that items capture unique dimensions of child development, including social-emotional and academic status. Related domains (e.g., expressive and receptive language) enjoy predictable associations but are not so substantially correlated that items can be considered duplicative.

Summary of Validity and Accuracy of the PEDS:DM^®

• Enjoys validation studies with favorable comparisons to a range of diagnostic measures including tests of intelligence, autism, neuromotor functioning, psychological risk, and clinical assessments of vision and hearing;
• Is responsive to known psychosocial risk factors for developmental problems;
• The original PEDS:DM^® studies (described at the beginning of this chapter) remain strong evidence for the sensitivity and specificity of the PEDS:DM^® Screening Version, given that more than 1600 children were included in both standardization, validity and accuracy studies via comparisons to diagnostic measures. Sensitivity and specificity for each domain and age-level averaged 83% and 84% respectively;
• Has substantive discriminant validity, i.e., there are unique performance patterns for each of the common disabilities;
• PEDS:DM^® (in combination with PEDS^®) was 100% and specificity of 82% to diagnostic measures of communication deficits;
• PEDS:DM^® also detects with 74% – 84% sensitivity and 82% – 83% specificity children diagnosed with cerebral palsy and other motor coordination problems, those with intellectual disabilities and delays, and those with social-emotional deficits;
• Now produces a cognitive score (for the Assessment Level) that is helpful for IDEA eligibility determinations;
• Is an accurate and inexpensive approach to follow-up with graduates of neonatal intensive care units and other subspecialty services.

Concurrent and Discriminant Validity

The PEDS:DM^® enjoys statistically significant correlations with a wide range of diagnostic measures including tests of cognition/intelligence, autism, language development, adaptive behavior, neuromotor functioning, psychosocial risk, psychosocial resilience such as positive parent-child interactions, and clinical assessments of motor tone, vision and hearing;

• Because it taps all domains of development including social-emotional and academic, the PEDS:DM^®’s highest correlations (> .70), are shared with similarly broad diagnostic measures, especially adaptive behavior tests that tap practical aspects of social, motor, communication and school skills;
• Is responsive to the effects of known psychosocial risk factors for developmental and mental health problems; Children tend to perform less well on the PEDS:DM^® when families are poor, do not speak English well, have limited education, etc. Increases in risk factors are inversely correlated with performance on the PEDS:DM^®;
• Is responsive to resilience factors, defined as positive parent-child interactions, i.e., children tend to score higher on the PEDS:DM^®, the greater the number of resilience factors;
• Has substantive discriminant validity, i.e., there are unique performance patterns for each of the common disabilities as expressed by statistically significant odds ratios per condition including autism spectrum disorder, intellectual disabilities (and slow learning), cerebral palsy and other types of motor deficits, language impairment and delay, etc.

Accuracy (Criterion-Related Validity) and Discriminant Sensitivity

• The original PEDS:DM^® studies (described at the beginning of this chapter) remain strong evidence for the sensitivity and specificity of the PEDS:DM^® Screening Version, given that more than 1600 children were included and accuracy tested in comparison with diagnostic measures. Sensitivity and specificity for each domain and age-level averaged 83% and 84% respectively.
• A recent study of 201 children found sensitivity of 100% and specificity of 82%.
• PEDS:DM^® (in combination with PEDS^® was 100% and specificity of 82% to diagnostic measures of communication deficits;
• PEDS:DM^® also detects with 74% – 84% sensitivity and 72% – 83% specificity children diagnosed with cerebral palsy and other motor coordination problems, those with intellectual disabilities and delays, and those with social-emotional deficits.

Note: Federally Qualified Health Centers and clinicians within Accountable Care Plans with value-based payments, tend to receive enhanced reimbursement for quality care – not fees for specific services such as the number of screens administered. In such settings, quality of care and efficient care are inextricable. Although PEDS Online^® costs more than PEDS Tools^® in print, accuracy is greater and time-related costs are lessened. Please see www.pedstest.com/reimbursement for additional guidance.

Summary of Utility and Implementation Research

Utility

Costs and Time-Related Expenses for Screening and Training

• The PEDS:DM^® in combination with PEDS^® can be completed by parent-report in 5 minutes.
• Professional time to score (if using print) takes ~ 2 minutes.
• With a reading level of 2nd – 4th grade, ~ 95% of parents can answer the questions on their own.
• When interview administrations are needed (for ~ 5% of families), these require only 6 minutes of professionals’ time for both PEDS:DM^® and PEDS^®
• Availability of ~ 45 carefully vetted translations (as of 2016) ensure that providers, parents or interpretation services can elicit accurate responses from parents.
• The combination of PEDS Tools^® along with a periodic M-CHAT-R are the most comprehensive of screens – tapping all developmental domains including social-emotional and behavior (plus math and reading skills for children 3 ½ years and older).
• The PEDS:DM^® Screening Version together with PEDS^® can be billed separately via the 96110 procedure code (96110 X 2) and if the M-CHAT-R is added, then 96110 X 3. Given ~ $8.00 reimbursement per screen, PEDS Tools^® should be a profit center for settings able to receive reimbursement for screening, but if not:
• The PEDS:DM^® in combination with PEDS^® is least the expensive approach to screening due to brevity, reading ease, and minimal material costs. Reusable materials include laminated materials (for the PEDS:DM^®) and longitudinal growth charts that make material costs less than those of freely available tools that require photocopying.
• The publisher of PEDS Tools^® provides freely available training materials via its website, www.pedstest.com. These include videos, slide shows, case examples, frequently asked questions, billing/coding guidance, etc. This saves $2,000 – $4,000 in the training fees often charged for use of some measures.

Costs and Time-Related Expenses for Early Intervention and Subspecialty Clinic Assessment

• The PEDS:DM^®–AL can be administered by parent self-report (~10 minutes) by interview (~10 minutes) or via hands-on elicitation (~10 – 15 minutes) thus offering flexible administration approaches responsive to professionals’ demands and families’ needs.
• 95% of parents can answer PEDS:DM^®–AL items independently.
• Because the PEDS:DM^®–AL provides age-equivalent scores (in expressive language, receptive language, fine motor, gross motor, self-help, social-emotional, cognitive, and academic skills) it is useful in determining percentage of delay scores necessary for IDEA services. The PEDS:DM^®–AL can eliminate the need for lengthy and expensive administrations of measures demanding specialized expertise (e.g., the Bayley Scales, Battelle Developmental Inventory, etc. that generally require 1½ – 2 hours of professional time).
• The PEDS:DM^®–AL’s Longitudinal Booklet further reduces costs because it can be re-used over time (e.g., if given to the same child 6 times the ~$3.00 cost is reduced to ~ $0.50 per administration).
• PEDS Online^® saves the time-related expenses of hand-scoring, generating referral letters and parent summary reports, chart documentation and looking up billing/procedure codes ~ 31 minutes of professional time. Such time can be assigned a dollar amount by summing overhead expenses (meaning salaries, rent, supplies, utilities, etc.), calculating per minute costs and multiplying by 31 minutes.

Decision-Support, Incidence and Referral Criteria

• PEDS Tools^® offer guidance to providers about when to: Refer (and where to refer); Screen further; Offer guidance; Monitor progress; or, Assure families that developmental-behavioral progress is continuing.
• IDEA programs are provided information on varying incidence rates given the combination of results from the combination of the PEDS:DM^® plus PEDS^®. These results are useful in establishing additional referral guidance that best matches the varying criteria for enrollment that exists across US States (as well as prevalence figures from the Centers for Disease Control).

Implementation

• Implementation studies reveal the various ways screens are deployed in busy settings. Different approaches are used according to families’ literacy skills and languages spoken, but ~ 75% of settings use paper-pencil to gather information by parent self-report even if using PEDS^®  Online for scoring and report-writing.
• That 91% of providers administered both the PEDS:DM^® plus PEDS^®, illustrates compliance with professional policies on early detection, i.e., a combination of evidence-based milestones and eliciting/addressing parents’ concerns is workable even in busy pediatric clinics.
• 64% of providers were able to use PEDS:DM^® (with or without PEDS^®) plus the M-CHAT-R, i.e., the majority were able to embrace all three tenets of early detection policies. Note that many clinics have the M-CHAT/M-CHAT-R built into electronic records and so the 64% rate for use of both essential broad-band tools plus an autism-specific screen may be an underestimate of uptake and thus utility.