Developmental Screening—Evaluation of an m-Health Version of the Parents Evaluation Developmental Status Tools
Boledi K. Maleka, BComm Path1, Jeannie Van Der Linde, PhD1, Frances Page Glascoe, PhD2, and De Wet Swanepoel, PhD1,3,4
Developmental screening is vital in establishing early detection and timely referral to early intervention services.1,2 Early interventions are beneficial in that stimulation could still have an effect on brain
growth and recovery.3 During the first 3 years of a child’s life, rapid brain development occurs, which is essential for future growth, development, and progress. Early intervention aims to ensure and enhance children’s personal development and resilience. Children with disabilities, who receive good care, as well as developmental opportunities during early childhood, are more likely to become healthy productive adults. This can potentially reduce the future costs of education, medical care, and social spending.4 Ultimately, future delays can be pre- vented by means of early intervention.5
Early intervention is especially important in developing countries, such as South Africa, where the prevalence of de- velopmental delays is high.6 To detect developmental delays, developmental screening measures can be used.7 Currently, there is no coordinated developmental monitoring and sur- veillance system in place within either the public or private sector in many countries like South Africa. Developmental screening is conducted by nurses in primary healthcare (PHC) that are often understaffed and under-resourced.8 A com- prehensive developmental screening approach is required for appropriate care and support, including early identifica- tion, assessment, and early intervention planning, provision of services, and monitoring and evaluation.4 The only im- plemented developmental screening tool in South Africa is integrated as part of the road to health booklet (RTHB). This tool is not standardized and has been shown to have low sensi- tivity (25%). There is a clear need for an efficient developmental screening tool to improve early detection of developmental delays at community levels.9
The Parents Evaluation Developmental Status (PEDS) tool, a standardized and validated measure, has recently been con- sidered for use in PHC contexts of South Africa.9,10 Ideally, a tool that is quick, reliable, and, which could be used by frontline health workers such as community health workers (CHWs), could ensure widespread access to early detection. The number of speech language pathologists (SLPs) and other
Department of Speech-Language Pathology and Audiology,
University of Pretoria, Pretoria, South Africa. 2Department of Pediatrics, Vanderbilt University, Nashville, 3Tennessee.
Ear Sciences Centre, School of Surgery, The University of Western
Australia, Nedlands, Australia.
4Australia Ear Science Institute Australia, Subiaco, Australia.
Background: Developmental delays are more prevalent in low-income countries and access to developmental screening is severely limited. Introduction: This study evaluated an m-Health version of a standard developmental screening tool, Parents Evaluation Developmental Status (PEDS) and PEDS: Developmental Milestones (PEDS:DM) for use by community health workers (CHWs) in terms of (1) correspondence with conventional paper-based testing by a speech language pa- thologist (SLP) and (2) inter-rater reliability compared to an SLP. Method: CHWs were trained in a primary healthcare (PHC) setting to administer the newly developed smartphone application version of the PEDS tools. One SLP and two CHWs recruited 207 caregivers who were attending a baby wellness clinic. Caregivers were tested by one CHW using the smartphone application of the PEDS tools; a qualified SLP simultaneously recorded and scored the PEDS tools on the same participants. Results: High positive (100%) and nega- tive correspondence (96%) was found between the paper- based PEDS tools and the smartphone application PEDS tools and between the SLP and CHW. Almost perfect (Cohen’s Kappa) inter-rater agreement between conditions was dem- onstrated (j = 0.873 to j = 0.961). Conclusion: Outcomes of the smartphone application, operated by a CHW, corre- sponded closely to the gold standard PEDS tools operated by a health professional. Trained CHWs can conduct accurate developmental screening using the smartphone version of the PEDS tools.
Key words: e-health, m-health, pediatrics, technology
DOI: 10.1089/tmj.2016.0007 a MARY ANN LIEBERT, INC. VOL. 22 NO. 12 DECEMBER 2016 TELEMEDICINE and e-HEALTH 1
MALEKA ET AL.
healthcare professionals are limited and overburdened with high caseloads in secondary and tertiary health centers.11 In South Africa, the use of smartphone applications is part of the community oriented primary care (COPC) initiative using CHW and m-Health initiatives to deliver continuous, com- prehensive, integrated, and informed healthcare services to underserved communities. A developmental screening like the PEDS tools in an App format could function as part of the COPC initiative whereby CHWs conduct the smartphone ap- plication PEDS tools remotely during home visits.12
Identification and assessment of children with disabilities in high-income countries often involve teams of highly trained professionals.4 The PEDS tools, for example, are usually ad- ministered by parents or trained developmental health pro- fessionals.13 However, in low- and middle-income countries such comprehensive expertise is often inaccessible and poor parental literacy skills may pose a challenge.4 In some coun- tries, CHWs are trained and supported by professionals to strengthen capacity and improve the quality of interven- tions.4,12,14 CHWs can extend care to underserved communi- ties, drawn from local communities. They speak the languages and identify with the local community to convey health messages more effectively.14 CHWs can, therefore, be un- iquely positioned for early detection of developmental con- cerns if the right tools that are simple and cost and time efficient are available.12 The PEDS tools could potentially be used in the form of a mobile phone-based assessment for developmental screening by CHWs. CHWs using m-Health tools have been proposed as an important way to improve access to healthcare services for early detection and subse- quent care for community members.12,15 This study, therefore, aimed to evaluate developmental screening in terms of (1) correspondence between conventional testing using paper- based methods by the SLP and testing using a smartphone application by the CHW and (2) inter-rater reliability between the SLP and CHW.
Data were collected at Stanza Bopape Clinic, a government PHC facility in Mamelodi, Gauteng Province, South Africa. Due to office space shortages at the clinic, a private mobile office was set up. CHWs were trained on administering the smartphone-based version of the PEDS tools as part of an outreach program. Three CHWs were asked upon completion of the training to volunteer to participate in the study. Two female CHWs with 5 years of CHW experience assisted with recruiting participants. One male CHW who had 6 years CHW experience in the PHC setting and no tertiary qualifications
was administering the smartphone PEDS tools. All the CHWs daily utilize smartphone applications in the PHC setting as part of their service delivery (i.e., health registrations and general risk surveys).12
A convenience and disproportionate stratified sampling method was utilized to select 207 caregivers who were waiting in a queue at a Baby Wellness Clinic at Stanza Bopape Clinic.16 Caregivers who were not proficient in English were excluded from the study. Caregivers attending the clinic with their child or children were asked to voluntarily participate in the study. Mothers were 88% (n = 182) of respondents, 7% (n = 15) other family members, and 4% (n = 9) fathers. Sepedi was 44% (n = 90) of the caregivers’ first language, 15% (n = 30) were Tsonga, 12% (n = 24) were Zulu speakers, and 29% (n = 63) had other lan- guages as their first language. Caregivers were divided into a stratified sample according to the age groups of their children. The age groups were 6–18 months 69% (n = 142) and 19–36 months 31% (n = 65). The mean was 1.937 and standard devi- ation was 1.3549; 99.9% (n = 206) of children screened were black and 0.5% (n = 1) were others.
MATERIAL AND EQUIPMENT
In this study, the PEDS and PEDS: Developmental Mile- stones (DM) combined, are referred to as the PEDS tools for reporting purposes. The PEDS tools are a developmental screening tool by parental report, which was used to collect data. The PEDS elicits parents’ concerns about children’s language, motor, self-help, early academic skills, behavior, and social- emotional/mental health. The PEDS consists of 10 questions, such as: Do you have any concerns about how your child understands what you say? Do you have any concerns about how your child behaves? The PEDS:DM are indicative of children’s skills across all developmental domains: expres- sive language, receptive language, fine motor, gross motor, social emotional, self-help, and academics. The PEDS:DM consist of six to eight questions, such as: Can your child walk without falling much? Does your baby drink (not suck) from a cup?10
An hour training session on administering the PEDS tools on a smartphone application was provided to the CHWs in person utilizing a training module of the PEDS tools and the PEDS tool guide to administration and scoring. The PEDS tools were developed into a smartphone application, using the same algorithm as the conventional paper-based PEDS tools. Two Samsung Neo Trend smartphones (Android OS 4.4.1) were used to install the PEDS tool application. The PEDS tool ap- plication was developed by the University of Pretoria, evalu- ated and piloted by two SLPs on eight caregivers. Screenshots of the PEDS tool application are presented in Figure 1.
2 TELEMEDICINE and e-HEALTH DECEMBER 2016
a MARY ANN LIEBERT, INC.
M-HEALTH DEVELOPMENTAL SCREENING
Fig. 1. Screenshot of the PEDS tool application. (a) Example question; (b) response options; (c) results screen; (d) results description. PEDS, Parents Evaluation Developmental Status.
CHWs were trained and thereafter, caregivers were inter- viewed by the CHW using the smartphone application of the PEDS tools; simultaneously, a qualified SLP was recording and scoring the PEDS tools on the same participants. The SLP completed the PEDS tools, based on caregiver responses, using either the smartphone application or conventional paper- based version in a counterbalanced manner. The CHW only administered the smartphone version. To eliminate bias, the SLP and the CHW did not communicate, make contact, or view each other’s records during testing. The PEDS tools were ad- ministered concurrently by the SLP and CHW to ensure that the context and caregiver responses, as elicited by the CHW, were similar.
Scores of the paper-based PEDS tools completed by the SLP were manually captured and uploaded to the same server as the smartphone application server. Caregivers whose children obtained referral results according to the findings of the SLP were issued with referral letters to the relevant healthcare professionals for follow-up.
The Statistic Package Social Sciences (SPSS) v22 (Chicago, Illinois) was used for statistical calculations and analysis.17 Frequency distributions, cross tabulations, and descriptive sta- tistics depicting the mean, standard deviation, and minimum and maximum values were used to analyze data. Pivot tables were used to calculate positive correspondence, which deter- mined the proportion of positive screen outcomes correctly identified, and negative correspondence, which measured the proportion of negative screen outcomes that were correctly identified. Positive and negative correspondence was calculated for paper-based and smartphone application PEDS tools, as well as for the results obtained by the CHW and the SLP.18 Cohen’s Kappa coefficient was used to establish the inter-rater agreement between the paper based and smartphone application PEDS tools, as well as between the SLP and CHW. Inter-rater agree- ment was classified according to the Landis and Koch-Kappa’s Benchmark Scale into poor (j=<0.0), slight (j=0.0–0.20),
a MARY ANN LIEBERT, INC. VOL. 22 NO. 12 DECEMBER 2016 TELEMEDICINE and e-HEALTH 3
MALEKA ET AL.
fair (j = 0.21–0.40),
A total of 207 children were assessed using the PEDS tools by the CHW and the SLP. Half of the children (51%) were assessed using the paper-based PEDS tools (odd-numbered participants) and the other half (49%) were assessed using the smartphone application (even-numbered participants) by the SLP. Referral rates were similar when the outcome of the CHW and the SLP was compared. Similar referral rates were also yielded when the paper-based outcomes were compared to the outcomes of the application (Table 1). The SLP and the CHW also found similar referral rates across age categories (Table 2).
High positive and negative correspondence was found be- tween the paper-based PEDS tools and the smartphone appli-
cation PEDS tools, as well as between the SLP and CHW (Table 3). Higher positive and nega- tive correspondence was noted in the younger, 6–18 month age group (Table 3). High inter- rater agreement between conditions varied from j = 0.873 to j = 0.961 (Kappa score; Table 4).
The smartphone-based version of the PEDS tools developed for this study operated by CHWs corresponded exactly with the paper-based version completed by an SLP in 99% of instances (n = 207). Agreement was therefore almost per- fect (j=0.960; Cohen’s Kappa) between test outcomes by a CHW using the application and results obtained by a health professional (SLP)
using conventional paper-based testing. Previous studies have reported that CHWs can provide high-quality care and bridged the gap between patients and healthcare providers.21 Further- more, CHWs were found to fulfill a crucial role in smartphone- based hearing screening and management of noncommunicable diseases.15,22 It has also been reported in a South African study that a trained lay telehealth clinic facilitator was effective to capture reliable images of the eardrum for accurate asynchro- nous diagnosis by an otolaryngologist.23 CHWs who are part of the COPC initiative are frontline health workers who are more accessible and cost effective than SLPs and other healthcare practitioners.12 The PEDS tool smartphone application, when used by users with different levels of training, was demonstrated to be reliable. CHWs who receive appropriate training will be able to effectively administer developmental screening using the smartphone-based PEDS tool application.
Outcome on the PEDS tool application corresponded with those found on the conventional paper-based PEDS tools (Table 3). The PEDS tool smartphone application was accurate and maintained the integrity of the conventional PEDS tools. Developmental screening by CHWs utilizing a smartphone- based version of the PEDS tools could be beneficial in un- derserved South African communities, where children are at an increased risk of developmental delays.6,9 The use of a developmental screening tool like the PEDS tools operated from a smartphone could ensure availability of developmental screening services and referrals to appropriate healthcare professionals for earlier intervention. The RTHB screening done by nurses in South Africa has limitations.9 The PEDS tools may offer an advantage and decentralize current screening initiatives from clinics to homes.
The use of smartphone applications in healthcare has been shown to improve access to PHC services.15 The use of the PEDS
(j = 0.41–0.60),
4 TELEMEDICINE and e-HEALTH DECEMBER 2016
a MARY ANN LIEBERT, INC.
tools as part of the COPC initiative would assist in early devel- opmental delay detection for the reasons that CHWs are already doing occasional home visits using smartphones. Caregivers would receive informational counseling on early development and early intervention could take place remotely. Furthermore, test results can be sent to a cloud-based service whereby the information would be accessible and safe. In addition, caregivers would be provided with referral appointments.
A high referral rate was obtained by both the SLP and CHW in the sample population. A possible reason for this may be due to the children being from a high risk population.6 Similar referral rates have been reported in a previous study con- ducted in other South African underserved communities.9 In contrast, a study conducted on a global scale, including data from 11 countries, depicted a lower referral rate of 34%.24
The high referral rate noted in the abovementioned South African contexts may be problematic as the already over- burdened healthcare system may not be able to provide ef- fective and accountable services to all, should developmental screening be implemented on a large scale. The PEDS tool referral algorithm may need to be adapted for the South Af- rican underserved population to ensure that moderate to se- vere developmental delays are detected, as well as referred, and that mild developmental delays are followed up by means of developmental surveillance. This could be implemented to obtain more reasonable referral rates. A validation of the application should be done evaluating the smartphone PEDS tools against PEDS tools as a diagnostic gold standard tool. Furthermore, it should be determined if caregivers will be able to effectively administer the PEDS tool smartphone applica- tion. Since the PEDS tools were administered by the SLP and CHW, the accuracy of the smartphone-based version was not determined on a group of participants with varying knowl- edge on child development. It is therefore recommended that the PEDS tool smartphone application should be evaluated when administered by parents themselves, as well as various health professionals, such as nurses, occupational therapists, pediatricians, and general practitioners.
Almost perfect agreement between conventional testing using the paper-based PEDS tools and the PEDS tools as a smartphone application was found. Furthermore, almost perfect inter-rater agreement between the SLP and CHW was reported. CHWs who have been trained successfully con- ducted developmental screening using the smartphone ver- sion of the PEDS tools. COPC initiatives may be a viable platform to render smartphone-based developmental screen- ing to high risk communities. CHWs can conduct develop-
mental screening in high risk communities easily with the smartphone application and results can be integrated into a telehealth framework to provide appointments, reminders, informational counseling, and even early teleintervention services. This makes early detection of developmental delays in underserved communities possible toward preventative measures and early initiation of necessary interventions.
The researchers acknowledge Stanza Bopape Clinic man- agement and staff for supporting this research project. The authors also thank all the caregivers for participating in the study. The researchers acknowledge F.P.G. for making the paper-based PEDS tools available for use in this study. Information regarding the PEDS tool application can be obtained from the authors.
No competing financial interests exist.
1. Conatser P, James E, Ledingham C. Early intervention developmental screening assessment. Palaestra 2013;23:37–41.
2. Fernell E, Wilson P, Hadjikhani N, Bourgeron T, et al. Screening, intervention and outcome in autism and other developmental disorders: The role of randomized controlled trials. J Autism Dev Disord 2014;44:2074–2076.
3. Sonnenschein B, Conover K, Shizgal P. Growth of brain stimulation reward as a function of duration and stimulation strength. Behav Neurosci 2003;117: 978–994.
4. World Health Organization (WHO). Early childhood development and disability: Discussion paper. 2012. Available at http://apps.who.int/iris/bitstream/10665/ 75355/1/9789241504065_eng.pdf (last accessed October 10, 2014).
5. Groark CJ, Eidelman SM, Maude S, et al. Early childhood intervention: Shaping the future for children with special needs and their families, 1st ed. California: Praeger, 2011:80.
6. Van der Linde J, Swanepoel DW, Glascoe FP, et al. Risks associated with communication delays in infants from underserved South African communities. Afr J Prim Health Care Fam Med 2015;7:841–847.
7. Lynn D, Newton HB, Rae-Grant AD. The 5-minute neurology consult, 2nd ed. Philadelphia: Lippincott Williams & Wilkins, 2012:164–165.
8. Samuels A, Slemming W, Balton S. Early childhood intervention in South Africa in relation to the developmental systems model. Infants Young Child 2012;25:334–345.
9. Van der Linde J, Swanepoel D, Glascoe FP, et al. Developmental screening in South Africa: Comparing the national developmental checklist to a standardized tool. Afr J Health Sci 2015;15:188–196.
10. Glascoe FP and Robertshaw NS. PEDS’ Developmental milestones a tool for surveillance and screening professionals manual, 2nd ed. USA: Ellsworth & Vandermeer Press, LLC. 2010:1–177.
11. Van der Linde J, Kritzinger A. Perceptions of rural primary healthcare personnel about expansion of early communication intervention. Afr J Prim Health Care Fam Med 2013;5:553–564.
12. Bam N, Marcus T, Hugo J, et al. Conceptualizing Community Oriented Primary Care (COPC)—The Tshwane, South Africa, health post model. Afr J of Prim Health Care Fam Med 2013;5:13–15.
M-HEALTH DEVELOPMENTAL SCREENING
a MARY ANN LIEBERT, INC. VOL. 22 NO. 12 DECEMBER 2016 TELEMEDICINE and e-HEALTH 5