About this paper symposium

---

Panel information
Panel 12. Methods, History, Theory

Paper #1
Development and Norming of the NIH Baby Toolbox Cognitive Domain
Author information	Role
Philip David Zelazo, Ph.D., University of Minnesota, United States	Presenting author
Doug Clements, University of Denver, USA	Non-presenting author
Sandra Waxman, Northwestern University, USA	Non-presenting author
Sarah Pila-Leiderman, Northwestern University, USA	Non-presenting author
Miriam A. Novack, Northwestern University, USA	Non-presenting author
Rachel M. Flynn, San Francisco State University, USA	Non-presenting author
Y. Catherine Han, Northwestern University, USA	Non-presenting author
Elizabeth M. Dworak, Northwestern University, USA	Non-presenting author
Maxwell Mansolf, Northwestern University, USA	Non-presenting author
Aaron J. Kaat, Northwestern University, USA	Non-presenting author
Hubert Adam, Northwestern University, USA	Non-presenting author
Lihua Yao, Northwestern University, USA	Non-presenting author
Richard Gershon, Northwestern University, USA	Non-presenting author
Abstract
Cognition during early development encompasses a wide range of domains, including learning, memory, attention, language, and early numeracy. These domains are foundational for children’s development and are predictors of later outcomes like academic achievement. Existing standardized assessments of cognition tend to be limited in scope, can be burdensome to administer, or do not utilize modern technology. Here, we present the Cognition domain that offers a way to assess cognition of infants aged 1-42 months using an iPad for easy data collection and scoring. Within the Cognition domain, measures focus on the sub-domains of Executive Function and Cognition (EF-Cog), Numeracy/Early Math, and Language. Measures were selected by a team of domain experts and informed by both an expert survey (n = 567) and a scoping review of the literature. All tasks were adapted for developmental levels within the iPad and were tested in both English and Spanish for feasibility, validated, and normed. The domain includes gaze, touch, caregiver-report, and observational measures. Gaze measures, developed for NBT, use the iPad’s iOS ARKit software to collect gaze data in real time. NBT’s gaze technology had a high calibration rate (87%) and was highly reliable compared to the gold-standard hand coding. Table 1 lists all Cognition measures with age ranges, modality of administration, and descriptions of the measure. Table 2 provides the psychometric results from the norming study. For each NBT measure level score, we examined correlations between NBT Cognition scores and age (Spearman rho) as well as correlations between NBT Cognition scores and their closest convergent Bayley-4 domain score (e.g., Cognition, Receptive Language, Expressive Language, Gross Motor, Fine Motor). Findings showed that measures within the Cognition domain were correlated with age, and most measures were reliably associated with convergent Bayley-4 domain scores. A subset of children was recruited to complete the NBT a second time 1-14 days after their first session (n=220 recruited). Test-retest reliability as measured via Pearson correlations between scores obtained from the first session and retest session was moderate to high (see Table 1). Intra-domain correlations indicated that measures correlated well together within their respective domains (Expressive Language r=.58, Receptive Language rs>.22, Language rs>.22, EF-Cog rs>.16, Math rs>.30). These results support the use of NBT Cognition for examining age-related changes in early development in the specified cognitive constructs. Future analyses will include developing sub-domain level composites (Receptive Language, Expressive Language, Executive Function and Cognition, Early Math); a domain-level composite (Cognition); and convergent validity comparisons with the Ages and Stages Questionnaire (ASQ) categories. Finally, age-based normative scores weighted to demographic variables obtained from the American Community Survey 2022 5-Year values are in the process of being developed, which is critical for score interpretation within clinical, educational, and research settings. Upon release, the NBT Cognition domain will provide a comprehensive and modern solution for measuring the assessment of cognition in infants.

Paper #2
Development and initial validation of the NIH Baby Toolbox Social-Emotional Functioning assessments
Author information	Role
Courtney King Blackwell, Ph.D., Northwestern University, United States	Presenting author
Cindy Nowinski, Northwestern University, USA	Non-presenting author
Christina Bethell, Johns Hopkins University, USA	Non-presenting author
Amy Wetherby, Florida State University, USA	Non-presenting author
Jessica Sommerville, University of Toronto, USA	Non-presenting author
Lauren Wakschlag, Northwestern University, USA	Non-presenting author
Maxwell Mansolf, Northwestern University, USA	Non-presenting author
Elizabeth M. Dworak, Northwestern University, USA	Non-presenting author
Y. Catherine Han, Northwestern University, USA	Non-presenting author
Aaron J. Kaat, Northwestern University, USA	Non-presenting author
Richard Gershon, Northwestern University, USA	Non-presenting author
Abstract
Early social-emotional functioning is foundational for children’s health and development and a meaningful predictor of school readiness, academic achievement, social competence, and psychopathology. However, brief, easy-to-administer, and psychometrically robust measures for use in clinical and research settings are lacking. Here, we describe the NBT Social-Emotional Functioning domain. Based on an extensive literature review (466 articles) and expert survey (n>500), we identified six measurement subdomains and associated measures. To assess temperament, we selected the “very short form” Infant, Early Childhood, and Childhood Behavior Questionnaires (IBQ-R-VSF, Putnam et al., 2014; ECBQ-VSF, Putnam et al., 2010; CBQ-VSF; Putnam & Rothbart, 2006). We selected Patient Reported Outcome Measurement Information System (PROMIS®) Early Childhood (EC) Parent Report instruments to assess Negative Affect, Psychological Well-Being, Self-Regulation, and Social Relationships (Blackwell, Kallen et al., 2022; Blackwell, Lai et al., 2022). As no adequate measure of Social Communication was identified, we developed the Social Observation Measure (SOM) and accompanying Caregiver Checklist (CC). See Table 1 for descriptions of the measures. Measures were calibrated using item response theory methods, and construct validity was evaluated with intra- and inter-subdomain correlations and associations with co-administered legacy measures of child developmental milestones (i.e., the Ages and Stages Questionnaire – 3 [ASQ-3, Squires & Bricker, 2009] and Bayley-4 [Bayley & Aylward, 2019]). Intradomain correlations suggest each measure captures related but unique components, and interdomain associations show domains are distinct, as no correlations exceeded r=.60. For example, Negative Affect measures were positively correlated with each other (r=0.34-0.55) and negatively correlated with positive affect (r=-0.25 - -0.44). Moderate intra-domain correlations for Temperament measures of effortful control and surgency (r=0.26-0.57) and Self-Regulation measures (r=0.60) were found, and, to a lesser extent, these subdomains were correlated with each other (r=0.32-0.47). SOM and CC were positively correlated for each age group (younger: r=0.13-0.52; older: r=0.26), and the CC was modestly correlated with PROMIS EC Social Relationships measures (r=0.27-0.44). ANOVAs showed NBT Social-Emotional Functioning measures distinguished children at the ASQ categorizations of being on track for meeting developmental milestones, needing additional support and monitoring, and needing professional follow-up. NBT measures of effortful control (η^2=0.05-0.11), peer and child-caregiver relationships (η^2=0.04-0.06), SOM older (η^2=0.07-0.10), and CC (η^2=0.07-0.17) had medium to large associations with ASQ subscales of communication (η^2=0.05-0.16), personal-social (η^2=0.05-0.16), and problem solving (η^2=0.04-0.17). Significant positive Pearson correlations were found between NBT measures of Psychological Well-Being (r=0.25-0.34), Self-Regulation (r=0.33-0.47), Social Relationships (r=0.25-0.45), and Social Communication (r=0.38-0.67) with the Bayley-4 adaptive behavior subscales of expressive (r=0.33-0.68) and receptive (r=0.37-0.68) communication, interpersonal relationships (r=0.36-0.60), play and leisure (r=0.37-0.55), and daily living skills (r=0.35-0.66). SOM and CC were also significantly positively correlated with all Bayley-4 cognition, language, and motor subscales (r=0.33-0.76), as were measures of Self-Regulation (r=0.21-0.31). Overall, NBT Social-Emotional Functioning domain assessments provide a rigorous suite of measures assessing complementary but distinct subdomains that are both important outcomes as well as important contributors to early developmental milestones.

Paper #3
Video-based assessment of motor development (1-42 months) for the NIH Infant and Toddler Toolbox
Author information	Role
Dr. Karen E. Adolph, Ph.D., New York University, United States	Presenting author
Whitney Cole, New York University, USA	Non-presenting author
Miriam Novack, Northwestern University, USA	Non-presenting author
Y. Catherine Han, Northwestern University, USA	Non-presenting author
Jane Clark, University of Maryland, USA	Non-presenting author
Stacey Dusing, University of Southern California, USA	Non-presenting author
Jill Heathcock, The Ohio State University, USA	Non-presenting author
Orit Herzberg, New York University, USA	Non-presenting author
Jeff Lockman, University of Texas at Austin, USA	Non-presenting author
Laura Prosser, University of Pennsylvania, USA	Non-presenting author
Jaya Rachwani, Hunter College - CUNY, USA	Non-presenting author
Richard Gershon, Northwestern University, USA	Non-presenting author
Abstract
Existing assessments of motor development in infants and young children rely on specialized technology, caregiver-report, expert evaluation, specially purchased manipulatives, and/or lengthy administration. The motor assessment for the NIH “Baby Toolbox” minimizes these limitations. The Baby Toolbox assessment is a new, video-based, direct observation approach that can be administered and scored by non-experts in motor development, using commonly available objects and an iPad. The assessment takes < 10 minutes, from the first task until receiving composite and item-level scores. Two tasks assess key aspects of motor development from 1-42 months of age. “Get Up & Go” assesses postural transitions and locomotion. Children begin lying on their back and are encouraged to get up, locomote 3 meters to a small platform, climb onto the platform, and descend from the platform. “Reach To Eat” assesses manual actions and tool use. Children reach with each hand for a block or small snack, and use a spoon to bring food to their mouth. Both tasks focus on functional, everyday motor behaviors mastered by typically-developing children over the first few years of life. Both tasks require a suite of functional motor abilities, including head and trunk control, action planning, and affordance perception. And both tasks can be administered at multiple difficulty levels based on caregiver-report of children’s current skill level. Get Up & Go has an easier version for babies who cannot yet locomote and a harder version for children who can crawl or walk. Reach to Eat has an easier version for infants who cannot yet eat solid foods, a more challenging version for children who can eat “cheerios” or other snacks, and a harder version for children who can grasp a cheerio from a small base. Administrators videorecord children’s performance and are guided through a series of questions to score children’s actions based on immediate review of the video. For each question, the app includes drawings and written descriptions of possible behaviors, which the administrator matches to the selected video frame(s). Sliders capture the timing of particular behaviors. Target behaviors were selected based on prior work and expert opinion, then refined based on pilot testing of 70 children (ages 3-43 months). Norming, validation, and reliability data were collected as part of the larger Baby Toolbox Norming Study. The final sample includes 2515 children (1-48 months of age) stratified by maternal education, geographic location (census divisions), race and ethnicity, and language exposure (English- and Spanish-speaking). Composite motor scores are calculated for gross motor and fine motor subdomains. Both scores correlate positively with age (r = .82 for gross motor and r = .63 for fine motor) and with the Bayley-4 Motor growth scale values (r = .88 for gross motor and r = .63 for fine motor). In summary, with commonplace technology and everyday objects, non-experts can obtain detailed assessments of early motor skills based on directly observed behavior in a short amount of time. These assessments were designed for the Baby Toolbox, but our video-based, direct-observation approach could be applied to other domains of development.

⇦ Back to session