Test Construction, Psychometrics, Translations, Quality Improvement and Other Research Issues in Developmental and Behavioral Screening
In Chapter 19, we discuss a wide array of issues helpful for researchers and those establishing clinical care initiatives. Topics include how to: Create new items; Translate items; Construct measures and establish psychometric support; and conduct Quality Improvement/Maintenance of Certification projects. We also provide Guidance for researchers on variable selection, research metrics, study designs, adapting existing measures for use in other nations, and issues in needs assessment and survey research. Finally we identify resources for conducting thorough literature reviews, bench-marking, and data sets available to researchers for analysis. Below are links to helpful resources.
Quality Improvement and CME Resources
For providers wishing to work on a clinic level quality improvement project, the American Board of Pediatrics (ABP) has well-defined performance improvement modules (PIMs) including ones on adolescent depression, vision screening, and early detection of developmental-behavioral (DB) problems using quality screening tools. The modules provide detailed procedures, requisite forms, demonstration videos, etc.
Many approved projects can be found at AAPs EQIPP/PediaLink site. CHADIS (an online screening service) has an AAP approved MOC initiative. Other electronic DB screening applications (e.g., ASQ Online and PEDS Online) generate databases useful for QI and MOC.
The Vermont Child Health Improvement Program (VCHIP) offers QI projects (e.g., improving rates of detection in autism spectrum disorder) but also technical assistance on QI (e.g. raising national literacy rates). VCHIP also sponsors the National Improvement Partnership Network (NIPN) and its many QI projects embracing over 15 states and regions.
The National Association of State Health Policy’s ABCD projects (all devoted to improving early detection, referral rates, and collaboration with IDEA) often provide MOC/QI credits for participating providers in specific sates States and regions working with Help Me Grow can receive QI support helpful for completing the American Board of Pediatrics’ developmental screening performance improvement module.
Reach Out and Read (ROR) offers a CME course focused on background information and tools to help promote children's developmental skills and later school success.
The National Institute for Children’s Health Care Quality (NICHQ) has a range of projects for which credits are offered including follow-up on infant hearing screening, autism spectrum disorders, ADHD, and medical home.
The Continuity Research Network of the Academic Pediatrics Association (CORNET) provides training, collaborative QI projects and support for faculty and residents in continuity care clinics. Among the many ongoing projects are initiatives focused on improving mental health and decreasing aggressive behavior in children via the “Play Nicely” program (described in Chapter 7).
Sources for Literature Reviews
The National Library of Medicine (MEDLINE) focuses on peer-review biomedical literature and includes full-text articles for selected journals and abstracts only for all others.
Health and Psychosocial Instruments (HaPi) provides information about behavioral measurement instruments abstracted from journals covering health and psychosocial sciences and research review databases such as the Buros Mental Measurement Yearbook. Records provide information on questionnaires, interview schedules, vignettes/scenarios, coding schemes, rating and other scales, checklists, indexes, tests, projective techniques, etc. The HaPi data base is often adopted by libraries and shown along side MEDLINE.
The PsycINFO database from the American Psychological Association includes more than 3 million records for peer-reviewed literature in the behavioral sciences and mental health and is often available via online university libraries.
ERIC is The Education Resources Information Center provided by the US Department of Education. ERIC provides free access to education literature to improve practice in learning, teaching, educational decision-making, and research, including long-term outcomes studies.
EMBASE includes all MEDLINE records but also millions of international journals. Carried by most university libraries (although often only with abstracts not full text of papers), EMBASE enables researchers interested in identifying studies in other nations to find comparable work. Exploration of this data should proceed with recognition of English spelling differences when searching, (e.g., “behaviour”, “immunisation”) and that terms for ethnic minorities vary substantially (e.g., in Great Britain, “Asian” usually refers to those from the subcontinent of India and Pakistan).
Collections of review papers in developmental-behavioral pediatrics are often available. Examples are the Cochrane Collaboration and its extensive summaries of research on a range of topics. The Journal of Developmental and Behavioral Pediatrics has an ongoing series on measurements issues in child development.
RAND is designed to assist the US government with scientific research, analysis, and development. RAND’s Child Policy staff have published over 1300 omnibus research reviews on such topics as homelessness, mental health, adjustment issues in armed services families, how to evaluate the quality of child care programs, etc. The organization also hosts post-graduate training and summer internships.
The Center for Mental Health in Schools recognizes that most mental health services for children are provided by the public schools. Initiated by the University of California, Los Angeles, the School Mental Health project houses a listserv, a quarterly newsletter, links to journals focused on child psychiatry and psychology, and creates research reviews such as “Mental Health in Schools: Engaging Learners, Preventing Problems, and Improving Schools.”
The University of California at Los Angeles’ Center for Healthier Children, Families & Communities provides publications and a listserv focused on building effective communities and services around families’ psychosocial needs.
National/International Demographics and Performance Indicators
Most nations have something comparable to the US Census Bureau for determining population demographics. If difficulty finding statistic bureaus for other countries, see the United Nations and World Health Organization websites. With population parameters in hand, refer to guidance above on sample sizes and other considerations for standardization. The Wikipedia has detailed articles about nations, languages spoken, and links to sites within countries for further demographic information.
Several agencies have compiled abundant rankings of US States embracing a wide range of variables relevant to developmental-behavioral pediatrics. Examples include:
The Commonwealth Fund hosts the Health Systems Data Center within which is an interactive comparison map on such variables as numbers of children receiving annual developmental-behavioral screening, mental health screening and treatment, etc.
The Agency for Healthcare Research and Quality provides a range of quality indicators in preventive health care including measures and software.
Child Trends focuses on improving developmental and health outcomes for children. Its database publishes effectiveness information on various initiatives (e.g., after school programs, book sharing, etc.).
Annie E. Casey Foundations’ Kids Count (described in detail in Chapter 17 on National Initiatives) publishes a regularly updated state and local-level data book, special reports, issue briefs and fact sheets focused on high school dropout rates and teen births, proportions of children born with low birth-weights, living in single parent families, and living in low-income families, etc.
The Offord Centre for Child Studies conducts population surveys on developmental-behavioral status and psychosocial risk. Developed in Canada and used in several nations, the Offord Centre provides public health mapping software showing needs in relation to services.
Existing Data Sets for Further Analysis
For researchers interested in working with existing data there are many options. For requests to analyze existing data (that must be HIPAA compliant and fully de-identified), approval from a university Internal Review Board is still needed. Among the many IRB protocols, the appropriate form is “Research Not Subject to FDA or Common Rule”—often (and somewhat oddly) referred to as Non-Human Subjects Research. Suggestions for locating data sets include:
The Data Resource Center for the Child and Adolescent Health Measurement Initiative (CAHMI). The site offers data from the National Survey of Children’s Health and the National Survey of Children with Special Health Care Needs together with national and state indicators established from prior studies.
Large multi-site/state collaborative investigations may have data sets available for research. See CORNET and the AAP’s Pediatric Research in Office Settings (PROS) projects that each have projects focused on developmental-behavioral issues (e.g., child behavior problems, child abuse recognition, and violence prevention).
Online screening services, see Chapter 4 for current options, may be able to share anonymized data sets. For example, data on the concerns of 47,000 parents PEDS Online is being analyzed by researchers interested in viewing the content and changes in parents’ concerns according to well-visit ages. Additional studies focus on specific parental concerns associated with child behavior/social-emotional problems, possible autism spectrum disorder and other developmental disabilities. CHADIS is able to provide large data sets and recently worked on new standardization for the Pediatric Symptom Checklist.
Miscellaneous Research Resources
If writing items, test the readability of questions with a variety of reading formulas—each has strengths and weaknesses. The website, www.readability-score.com explains how each formula works, what information they provide and enables input of text for analysis. Be sure to test questions separately from response options because some formulas use sentence length as part of the criteria and so using answers such as “yes, “sometimes”, etc. can falsely lower the results.
If translating items and particularly for international studies, see the recommendations provided in this chapter and also the International Test Commission’s Guidelines for adapting tests. The site also includes guidance on internet/computer applications and quality control in scoring and interpretation, etc.
Helpful resources for a basic understanding statistical methods include SW Huck’s Reading Statistics and Researchfor the essentials of univariate techniques including non-parametric tests (with web support at www.readingstats.com) Tabachnick & Fidell’s Using Multivariate Statistics provides a clear (and actually fun) discussion of multivariate methods. Both books are available through Amazon. Manuals for the various statistical packages are useful but at least for the Statistical Package in the Social Sciences (SPSS), earlier versions are more informative about metrics.