Summary of PEDS® Research from Frances Page Glascoe, PhD Collaborating with Parents. 2nd Edition. Nolensville, Tennessee: PEDStest.com, LLC, 2013.
PEDS® has been re-standardized and re-validated. Its technical manual, Collaborating With Parents was published in 2013. Below is a summary of psychometric findings for standardization, reliability, validity, and accuracy:
- PEDS® was restandardized in 2012 on a nationally representative sample of 47,531 families in 27 US states and Canada, representing the major geographic regions of the US.
- Parents’ levels of education were similar to US Census Bureau 2010 indicators (e.g., 16% had not completed high school while 28% had completed college).
- Sites included rural and urban/suburban areas in proportion to prevalence.
- Ethnicities were represented at percentages between US Census Bureau indicators for 2010 and projections for 2020, and included white (not Hispanic), Black, American Indian, Asian, Hawaiian/Pacific Islander, Hispanic, etc.
- Six percent of families were administered PEDS® in Spanish (in keeping with Census Bureau indicators for those not speaking English well).
- Children’s gender and poverty rates also matched Census parameters.
- Children ranged in age from 0 – 11 months (20%, N = 13,523) through 7 years, 11 months years (2%, N = 913), i.e., a much younger sample than in PEDS® original norming studies.
- 91% of children and families participated in general pediatric clinics and the remainder participated from public schools (kindergarten intake), day care/preschool programs, developmental-behavioral pediatric clinics, and non-emergent crisis call centers.
- The frequency of parents’ concerns and risk on PEDS® rises, as expected, with children’s age, and with psychosocial risk factors. Hispanics and Blacks were more likely to have concerns as well as higher risk PEDS®’ paths.
- Educated families are more likely to receive Path C scores (i.e., to be the “worried well” in need of professional advice).
- Native Americans were more likely to have social-emotional and behavioral concerns as compared to other ethnicities, probably in keeping with the tribal custom of respect for elders.
- PEDS® has been translated into over 50 languages and standardized separately in many other nations and languages. In many international studies, different constellations of concerns and thus assignation of PEDS® Paths have been established. For example, self-help concerns are of greater concerns to parents and also more predictive of overall problems than in the US.
- Survey PEDS® (used in the National Survey of Child Health, First Five California, the Promoting Healthy Development Survey, The Child and Adolescent Health Measurement Initiative (CAHMI)) depends on 12 closed-ended questions and is scored only from “yes”, “no”, or “a little” responses. Although risk rates on Survey PEDS® are similar to clinical PEDS®, assignation to PEDS® Paths is strikingly different because 24% to 32% of families across studies are not answering the questions according to their intended content. For this reason, Survey PEDS® may not be used in clinical care with individual families–only for broad population-based needs assessment.
Summary of 2012 RELIABILITY Studies
- Test-retest reliability studies were conducted on 193 children over a 0 – 32 day time frame and revealed 94% agreement in PEDS® Paths and parents’ concerns.
- Inter-rater reliability (between expert coders and the PEDS® Online text-based scoring analyzer) was established on 355 children for both categorization of concerns (95% agreement) and for correct assignment to PEDS® Paths (97% agreement). Teachers and parents had lower levels of agreement (74%), but PEDS® was not normed for teachers’ appraisals (and, children often behave quite differently at home than at school).
- Stability of concerns for parents of older children (N = 402) was high over a two year time frame (for children approximately 5 – 7 years of age). Parents with predictive concerns when their children were five years of age were 5 times to have ongoing predictive concerns when their children were 7 years old (as compared to parents who did not have predictive concerns when their children were five years old.
- Stability of concerns in younger children (N = 325) from 6 months to 18 months was also high, with 48% percent of parents with concerns at 6 months, having ongoing concerns 12 months later.
- Internal consistency studies (on 45,310 children) showed a predictable factor structure: verbal versus non-verbal items clustered significantly and accounted for the majority of test variance. There were significant but only modest inter-correlations among concerns, suggesting that answers to each PEDS® question contributes uniquely to overall results. The coefficient of reliability/index of generalizability was moderately high, again suggesting PEDS® items are not redundant.
Summary of 2012 Validity Studies
- PEDS®’ content validity is evident in the finding that PEDS® facilitates a discussion of concerns across developmental (also meaning behavioral, social-emotional/mental health) domains in a way that other questions do not.
- PEDS®’ construct validity is shown in the finding that each type of parent concern clusters in predictable ways via associations among related domains (e.g., fine motor and gross motor; expressive and receptive language).
- The concurrent validity of PEDS® was studied on 1158 children in relationship to measures of development, including academics, intelligence, language, and motor skills. Although each type of parental concern enjoys significant associations with measures or subtests focused on the same domain, parents often have concerns in seemingly unrelated domains. For example, academic deficits are associated with social-emotional concerns—meaning parents may be commenting on self-esteem problems related to under-achievement or difficulties performing well in a group, i.e., parents’ concerns often reflect not just the apparent problem but also its impact on other aspects of development.
- Parents’ concerns on PEDS® and PEDS® Paths are also associated not just with eligibility for early intervention or special education but also with emerging delays and psychosocial risk factors.
- There are unique patterns of concerns for most diagnoses (e.g., cerebral palsy, ASD, language impairment) with correlations ranging from .84 – .99. The sensitivity of these patterns is discussed in the Accuracy Chapter on PEDS®.
- PEDS®’ predictive validity is established in studies showing that concerns raised early in a child’s life have a strong association with later deficits and diagnoses. For example, certain patterns of concerns along with specific content of concerns (e.g., repetitive behavior) when raised at 12 months of age is significantly predictive, 18 months later at 30 months of age, of a diagnosis of autism spectrum disorder, 83% sensitivity. Correlations range from .90 – .99.
Summary Of 2012 Accuracy Studies
- Overall accuracy indices were computed on 4473 children across two separate studies who were administered a variety of diagnostic measures to determine eligibility for special education services. Summary figures are: Sensitivity = 86% and specificity = 74%.
- Over-referrals on PEDS® continue to include children with psychosocial risk factors and delays, but not delays sufficient for special education eligibility. For this reason, positive predictive value (37% for special education eligibility) lacks meaning.
- Discriminant sensitivity (meaning research on how well PEDS® identifies specific conditions) were conducted on an additional 4,000 children across more than 20 different studies. These show unique patterns of concerns and often PEDS® Path differences when children have autism spectrum disorder, mental health problems, cerebral palsy/motor disorders, intellectual disabilities, language impairment, etc. In all cases sensitivity was greater than 80%.
- Two studies viewed the ability of parents without and with depression, anxiety, and other mental health difficulties to raise concerns on PEDS®. Both groups performed similarly suggesting that PEDS® can be used effectively with parents who have mental health difficulties (at least mild ones). PEDS® performed less well in identifying abusend and neglected children with developmental-behavioral problems when the informant was the suspected perpetrator.
- PEDS® in nations outside the US (including in translation) shows high sensitivity and specificity to problematic development.
There have been numerous studies of PEDS® in other countries. A side-by-side comparison of PEDS® in the US versus PEDS® when used in other countries is not possible due to wide variations in sampling (e.g., differences in parents’ levels of education, poverty rates, access to health care, let alone languages).
Many international studies began with a determination of feasibility (e.g., did parents or providers find PEDS® useful? Was PEDS® readable? How well could parents’ complete it on their own? Did PEDS® open channels of communication enable providers to engage parents in services?). Because answers were uniformly positive, international researchers continued with norming studies. All international studies equally sampled the age ranges for which PEDS® is normed and most compared results to the PEDS® 2002 norming studies. Details on studies by nation/region are described below followed by a summary of findings and recommendations.
Ongoing studies in Israel (with PEDS® administered in Hebrew) and another focused on Palestinian refugees in Lebanon (with PEDS® administered in Arabic), both showed dramatically fewer parental concerns about children’s behavior than is seen in the US (personal communication from Lobel, 2011; Tan, 2007). That phenomena makes sense given cultural differences, at least in certain parts of the Middle East, in child-rearing attitudes: Families in Israel and Lebanon seem quite tolerant of children behaving like children! And it isn’t that parents didn’t comment on behavioral issues such as attentional problems, hyperactivity, or temper tantrums. They did indeed, but were less likely than US families to note that they were concerned about such behavior.
But, the Middle East is hardly a uniform culture. Colleagues in Jordan, Saudi Arabia and elsewhere, provided us preliminary reports suggesting that behavioral issues were at least equal to those in the US. We continue to await additional information from different nations and will provide prompt updates on www.pedstest.com as we receive them.
In Africa, Tanzania specifically, parents had far more concerns than any other nation, i.e., 4 times the number of children receiving a high risk score on PEDS® as compared to the original US norming study. The context of this study is important to note: Conducted in the midst of a malaria outbreak, parents were uniformly worried about their children’s health and the potential impact of malaria on their development (Kosht-Fedyshin, 2006).
Both PEDS®, PEDS®:Developmental Milestones (Assessment Level) and the ASQ were administered in English to 133 graduates of the neonatal intensive care nursery when they were 6 months of age. Because this study did not report PEDS® results apart from the combination of PEDS® and PEDS:DM® and because it is more of a validation study than an standardization study, this research is discussed in more detail in Chapter 4 of Collaborating With Parents, 2nd edition. Nevertheless, PEDS® in combination with the PEDS:DM® were found to be more effective in identifying communication delays than was the ASQ although both sets of tools were equally likely to identify overall delays.
Asia, India, and the South Pacific
In Thailand, PEDS® was used with 216 0 – 72 month old children who were seen in outpatient hospital clinics. Frequencies of PEDS® Paths was not reported (only the abstract was written in English) but the authors commented, “….significant concerns of parents about their children’s development are the critical information for referral to have further management. In other words, parents concerns could have far more advantage than the [Denver-II].” (Theeranate & Chuengchitraks, 2005). This seems to mean that researchers found parents’ verbatim comments clinically useful; more so than simple a pass/fail score. Studies in Thailand are ongoing for both PEDS® and the PEDS:DM®.
Taiwan and Indonesia
In Taiwan and in Indonesia, PEDS® was validated against diagnostic measures (the findings are discussed further Chapter 4 of Collaborating With Parents, 2nd edition.Specific categories of concerns were not reported except within a comment that behavioral concerns were the most frequent (and least predictive) of concurrent test results (Chen, Lin, Wen & Wu, 2007; Gustawan, Soetjiningsih & Machfudz, 2010).
A study in Singapore conducted in 2002, provided translations for PEDS® in two different languages and then compared rates on PEDS® Paths when administered in Malay (N=569), English (N = 864), and Chinese (N = 383). Children of Malay-speaking parents were somewhat less likely to receive a Path A (high risk) score on PEDS® (18% versus 29%), and more likely to receive a Path B (moderate risk score (13% versus 8%) or a Path C score (11% versus 8%). Overall Malay speaking parents had fewer concerns (58% versus 54%). When Malay and English speaking parents were compared to Chinese-speaking parents, the latter had marked performance differences, i.e., 86% received Path A scores). This study is described in detail by King et al, 2012 who argue that “culture matters”. This is irrefutable, but quality “translations matter” too. So, by changing the wording of the Chinese translation to use a word more synonymous with “worries” than “concerns” (which can be interpreted as “care”), rates on PEDS® Paths became equivalent across language backgrounds.
Using translations into Visayan, researchers from the Phillipine Society for Developmental-Behavioral Pediatrics, trialed PEDS® on 421 children to determine detection rates (and the effectiveness of translations). Despite similarlty in Path A rates, children were somewhat younger in the US subsample (which was selected to reflect the same age range as the Phillipine sample: mean age 44 months in the US versus 50 months in the Phillipines). Proportional sampling would have been more optimal, because mean age differences may explain the differences especially in Path B (more common in younger versus older children). In contrast, Path C may reflect cultural differences in child-rearing practices, tolerance for children’s behavior, and differences in curricular demands once children enter school. In any case, the findings suggest that risk rates on PEDS® likely differ across nations and that separate standardization is needed. Even so, Phillipine studies found similar trajectories as US studies across age ranges: the older the child, the more frequent were parental concerns and as with US studies, behavioral issues rise dramatically at 2 years of age and older.
In India, a study of PEDS® was conducted with children (N = 79, age range 24 – 60 months) receiving outpatient care in a tertiary hospital’s clinics. Parental concerns were abundant, surely as a consequence of sampling an at-risk population and one that was generally older (and probably more likely to have health problems) than the 2012 US sample. In some ways, frequencies of of concerns found in India’s sample paralled percentages found in the US (2002 study), but with higher rates of each: Behavior concerns were the most common (40% in India versus 32% in the US), followed by social emotional concerns (22% versus 19%), and global/cognitive (6% versus 4%). In other ways, the two samples differed substantially: Parents in India had much higher rates of health concerns (17% India versus 5% US) while US parents were more likely to have expressive language language concerns (18% India versus 24% US). (Malhi & Singhi, 2002). Again, unique standardization seems needed.
There are numerous but still ongoing studies in various European nations (e.g., The Netherlands, Iceland, Germany, Portugal, etc.). Results will be reported on www.pedstest.com when studies are completed. Current studies and reports include:
Researchers in Galicia, Spain published a study of 6 – 24 month olds attending Galician pre-primary schools (N = 1089) whose parents completed PEDS®. Information about parents’ levels of education or poverty were not described in the paper making it difficult to compare results with the US sample (2002). Galician families received Path A scores less often than US parents (8% versus 11%), Path B (22% versus 23%), Path C (22% versus 20%), Path D (the path designated when hand-scoring PEDS® for parents who had difficulty completing forms on their own, had mental health problems, etc.) was 1% versus 3%, and Path E (46% versus 43%) (Campos, Squires, Ponte, 2010). PEDS® Paths in the Galicia study followed the general trajectory of US findings (using the more age-comparable 2002 sample) but at slightly lower rates.
Studies in Iceland are ongoing but positive enough to have included PEDS® in the national health care electronic record. Both PEDS® and the Brigance Screens were translated into Icelandic and adapted for the culture (e.g., on the Brigance, images of people, stoves, refrigerators and milk cartons all needed to be adapted to ensure familiarity to Icelandic children. And, because there are no snakes in Iceland, we included “worms” as a correct response for the snakes vocabularly question). Updates on the Icelandic experience will be added to the research pages on www.pedstest.com.
Many studies in the National Health Service of Great Britain are in progress. One completed study was conducted within England’s Sure Start (a program serving children at risk and much like Head Start in the US). Subjects were 1615 parents of children between 22 and 27 months of age. High rates of poverty probably explain some of differences in British sample as compared to the US (2002): Path A (7% versus 5%), Path B (17% versus 17%), Path C (18% versus 10%) and Path E (58% versus 68%). [Data from the National Evaluation of the Sure Start Programme]
The Milton-Keynes Public Health Trust in England has used PEDS® for almost a decade: Health visitors administer the measure by interview in families’ homes and return to administer an assessment level battery when screening suggests a problem. The pilot study found PEDS® beneficial in identifying developmental-behavioral problems, in promoting communication between parents and providers, parental and provider satisfaction with care, and in establishing a clear pathway of follow-up care. (Feeny & Davis, 2009). Preliminary results from an ongoing study of 76 two-year-old children at risk for developmental-behavioral problems found 26 (34%) to be high risk on PEDS® as well as on the Schedule of Growing Skills (SOGS) a lengthy assessment level measure. Interesting were the findings that 22 of the 26 had been tested with the SOGS at 1 year of age and found to be typically developing but PEDS® had identified many of these children as high or moderate risk. A predictive validity study on PEDS® is ongoing and results will be posted on www.pedstest.com.
Australia / Victoria
Australian researchers have conducted numerous studies on PEDS®. In a study based in day care centers (Coughlan, Kiing & Wake, 2003) 262 families participated whose children were between 18 months and 69 months of age. Results were compared to the 2002 PEDS® standardization study. Overall, Australian families were less likely to have any type of concern than were US families (48% versus 43%) and thus less likely to score on any of the elevated risk Paths than were Americans: Path A/High Risk (9% Australia, 11% US), Path B/moderate risk (19% Australia and 23% US); Path C/low risk, concerned with subsequent risk for mental health problems (24% Australia versus 20% US) and Path E/ low risk, no concerns (48% Australia versus 43% US). Types of concerns were lower for expressive language (21% versus 24%, receptive language (6% versus 8%), gross motor 5% versus 8%) school skills 9% versus 12%) and other/health (3% versus 5%). Rates were similar for global/cognitive and fine motor, Australian families had more concerns about behavior (34% versus 32%), social-emotional (21% versus 19%) and self-help (11% versus 19%).
The study also had day care teachers complete PEDS®. Teachers were less likely to have concerns in any area than were Australian parents. PEDS® has not been normed soley on teacher or provider report (but rather on a combination of the two) but it seems predicatible that children, even those with deficits, behave better and demonstrate a larger complement of skills when surrounded by appropriate role models.
A replication study in another Victorian city, Wondonga (N = 246) found similar performance patterns with fewer concerns than American parents. The researchers and also studied age differences in parents’ concerns, and although rates of concerns are lower than in the US, the patterns are similar: Concern regarding expressive language and behaviour increased with the age of the child. Expressive language concern increased with age and peaked between 18 months up to 3 years, with 32.7% of parents reporting concerns. Expressive language concerns then decreased slowly with the increase in age of the child with 23.2% of parents reporting concerns with children ≥ 4.5 years of age. Chi-Square Test for Independence showed there was a significant difference in parent reported concerns for expressive language between age groups (p= 0.003). Behavioural concerns also increased with the age of the child and peaked with 34.1% of parents reporting concern with children ≥ 4.5 years of age. Therewas a significant difference in parent reported concerns for behaviour between younger versus older age groups (p=0.023) (Armstrong & Goldfeld, 2008).
Western Australia and Queensland
Australia’s Aboriginal Health Service and its researchers, especially in Western Australia and Queensland) have used PEDS® (and other screens) in various communities, many of them rural. A summary of these studies is presented below: Cultural and language challenges abound: More than 100 different indigenous languages exist and unlike similar populations in North America, only about 3 are written down. Many indigenous families speak a combination of languages. The culture embraces the notion that “it takes a village” to raise a child and so aunts, uncles, and grandparents play a major role in child-rearing. At the same time, “kids rule”, meaning that concerns about children’s behavior is lower than in the US, and… if children don’t want to go to school, their refusal is acceptable. As a consequence, many children are ill-prepared for curricular demands (which surely increases absenteeism as children face of academic challenges). In turn, this leads to limited high school graduation rates and high rates of poverty. Health problems in adults are legion and seem to stem from intolerance of a western style diet (high in sugar and refined flour). Adults in indigenous communities still fear the history of the “Lost Generation”. Thus providers must gain the confidence of families of Aboriginal descent. When rapport is established, parents are more likely to express concerns. The word “concerns”, prominent in PEDS®, may be a challenge and studies are needed to determine whether “concerns” should be replaced with the more common Australian word, “worries” (as often heard in the phrase, “No worries”).
Of greater importance than just making sure a screening tool works well, is advocacy for early prevention services (e.g., an equivalent to the US Early Head Start and Head Start programs). Most young indigenous children fail screening tests such as the PEDS:DM® or Brigance Screens. Community-wide intervention is clearly needed– far more than screening– to ensure promotion of early development and increased likelihood of school success (Aboriginal Health Service (www.aboriginal.health.wa.gov.au).
Via Toronto’s Department of Public Health (Ontario), randomized telephone dialing was used to identify families with children in the appropriate age range and then administer PEDS® via interview (in English). Children (N = 221) ranged in age from birth to 6 years. Perhaps because families weren’t seeking services (or because health care services are more readily available) rates on the elevated risk PEDS® paths were lower than in the US: Path A (9% versus 11%), Path B (21% versus 23%), Path C (10% versus 20%) and Path E (61% versus 43%). (Ng et al, 2010).
In a separate study in Calgary (Alberta), parents were followed over time (when their children were 6, 12 and 18 months). In this study, parents were more likely to be educated (82% held college degrees and beyond versus 26% in the US). Parents in Calgary less likely to be poor (5% in Calgary versus 24% in the US), and were equally likely to be depressed (5% – 8% in Calgary versus 5.4% – 9% in the US (http://www.cdc.gov/prams/ppd.htm).
So what were PEDS® results in Calgary? Parents of 6 month olds (N = 372) were more likely than US parents of 6 month olds (N = 1837) to receive a Path A score (4% versus 1%), a Path B Score (15% verus 6%), a Path C score (16% versus 6%) and less likely to receive a Path E score (66% versus 87%). At 12 months, children in Calgary (N = 334), were still more likely than US parents of 12 month olds (N = 2159) to receive a Path A score (5% versus 1%), Path B score (15% versus 11%), Path C score (14% versus 8%) and less likely to receive a Path E Score (66% versus 80%). Calgarian parents of 18 month olds (N = 327), were marginally more likely than US parents of 18 month olds (N = 3376) to have children with Path A scores (4% versus 3%), but still more likely to have Path B scores (22% versus 17%), Path C (11% versus 8%) and continued to be less likely to have Path E scores (63% versus 72%). (Calgary data courtesy of Leew et al. The Cuddles Study, 2011. www.upstartchampions.ca). These comparisons illustrate that highly educated parents tend to have more concerns (as described in the section on US parents’ and levels of education).
Ongoing Studies In Other Nations
There are ongoing studies in Haiti, Fiji, Portugal, Brazil, Hungary, Turkey and many other nations. Updates will be found on www.pedstest.com. Many of these nations are also standardizing measures such as the PEDS:DM® and its inherent challenges with making tools developed in America work well in other nations.
Summary of International Studies and Recommendations for Clinicians and Researchers: It is tempting to think that in nations with universal health care, there might be lower rates of parental concerns–given better access to professionals who can provide guidance on child-rearing [apart from samples where families are in extreme poverty (e.g., Australia’s First Nation, England’s Sure Start) or almost exclusively college educated, i.e., the worried well (Calgary)]. That may be, but still remains an hypothesis requiring further research with samples matched across nations on critical variables such as parents’ level of education, poverty rates, and children’s ages. Nevertheless, it is clear that when parents face psychosocial risks including health challenges, their concerns about their children rise substantially, wherever in the world they reside.
Availability of early education programs varies substantially within and across nations. Rural and empoverished children often have limited access to preschool programs and so will perform less well on milestones-type screens. Because school services may be less available and unprepared children less likely to confront school failure, parents may well have fewer concerns about developmental domains associated with school success (e.g., language, fine motor, school performance) and may focus instead on how well children meet other life tasks (e.g., self-help, social-emotional development).
Overall, researchers should expect (once translations have been thoroughly tested) differences in the types and frequencies of concerns on PEDS® when used in other cultures. Cultural norms matter (e.g., as seen in the paucity of of behavioral concerns in Israel and Lebanon). Curricular demands and their universality matter too. For example, in Portugal, children are taught to read when they start 1st grade, whereas in the US, they are expected to have a substantial sight-word vocabulary and knowledge of all letter sounds before starting 1st grade. Thus parents in other nations might well be expected to have fewer concerns about school skills prior to age 6 than they might in the America. So, culture matters (and quality translations do too). The need for unique standardization in each nation is clear.
A comment on service availability across nations
Challenges in finding services even when available seems to be world-wide even in “developed nations”:The US is fortunate to have, under the Individuals with Disabilities Act (IDEA), something of a “one-stop shop”, i.e., a toll free number for each State for early intervention and follow-up evaluations. For older children, special education directors in the school of zone are identifiable and accessible. Nevertheless, IDEA across States does not always provide monitoring services for children at risk (e.g., false positives on screens but high likelihood of future academic problems due to elevated psychosocial risk factors) is not uniform across States. Most States’ IDEA programs are prohibited from referring non-qualifying children to private services (e.g., quality day care/preschool or private therapies). Head Start and Early Head Start are widely available but not for those above federal poverty levels—meaning that poor-ish children cannot attend. This leaves providers needing abundant knowledge of local service options.
Canada, in contrast, has many services (varying, as in the US, by province/State), but Canada lacks an umbrella, meaning that providers are much challenged when making referrals given the absence of a single toll-free number to identify services (a province-by-province directory of programs sounds much needed; US State by State as well).Australia has universal screening at four years of age and soon to begin at three years as well.. There are also family resource programs devoted to parent-training but there are challenges similar to Canada’s in terms of finding resources because they lack a universal early intervention program.
Great Britain has relatively easy access to services (via local and regional council websites) but its Sure Start program has faced budget cuts and loss of services (for which parents are currently suing their funding source, the Exchequer (the US’s IRS equivalent). But in contrast, the Exchequer directly funds Sure Start and has, until recent economic down-turns, ensured a stable source of funding in a way that the US does not do for Head Start).
In developing nations, several early detection initiatives foundered due to a dearth of early intervention/prevention services. Neverthless, application of screening tools in such nations can at least serve as needs assessment and a viable database for service advocacy.