An official website of the United States government
ABSTRACT Given the foundational nature of infant visual attention and potential cascading effects on later development, studies of individual variability in developmental trajectories in a normative sample are needed. We longitudinally tested newborns (N= 77) at 1–2 and 3–4 weeks, then again at 2, 4, 6, 8 and 14 months of age, assessing individual differences in their attention. Newborns viewed live stimuli (facial gesturing, rotating disk), one at a time, for 3 min each. Older infants viewed a 10‐s side‐by‐side social–nonsocial video (people talking, rotating disk). We found short‐term developmental stability of interindividual differences in infants’ overall, social, and nonsocial attention, within the newborn period (1–4 weeks), and within the later infancy period (2–14 months). Additionally, we found that overall attention, but not social and nonsocial attention, was developmentally stable long term (newborn through 14 months). This novel finding that newborn overall attention predicts later overall attention through the first year suggests a robust individual difference. This study is a first step toward developing individual difference measures of social and nonsocial attention. Future studies need to understand why newborns vary in their attention and to identify the potential impact of this variability on later social and cognitive development.
more »
« less
This content will become publicly available on September 13, 2026
Automated detection of mouth opening in newborn infants
Automated behavioral measurement using machine learning is gaining ground in psychological research. Automated approaches have the potential to reduce the labor and time associated with manual behavioral coding, and to enhance measurement objectivity; yet their application in young infants remains limited. We asked whether automated measurement can accurately identify newborn mouth opening—a facial gesture involved in infants’ communication and expression—in videos of 29 newborns (age range 9-29 days, 55.2% female, 58.6% White, 51.7% Hispanic/Latino) during neonatal imitation testing. We employed a 3-dimensional cascade regression computer vision algorithm to automatically track and register newborn faces. The facial landmark coordinates of each frame were input into a Support Vector Machine (SVM) classifier, trained to recognize the presence and absence of mouth opening at the frame-level as identified by expert human coders. The SVM classifier was trained using leave-one-infant-out cross validation (training: N = 22 newborns, 95 videos, 354,468 frames), and the best classifier showed an average validation accuracy of 75%. The final SVM classifier was tested on different newborns from the training set (testing: N = 7 newborns, 29 videos, 118,615 frames) and demonstrated 76% overall accuracy in identifying mouth opening. An intraclass correlation coefficient of .81 among the SVM classifier and human experts indicated that the SVM classifier was, on a practical level, reliable with experts in quantifying newborns’ total rates of mouth opening across videos. Results highlight the potential of automated measurement approaches for objectively identifying the presence and absence of mouth opening in newborn infants.
more »
« less
- Award ID(s):
- 1653737
- PAR ID:
- 10636505
- Publisher / Repository:
- Springer
- Date Published:
- Journal Name:
- Behavior research methods
- ISSN:
- 1554-3528
- Subject(s) / Keyword(s):
- neonate behavioral coding gesture recognition video annotation facial expression social behavior social interaction
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Language-guided smart systems can help to design next-generation human-machine interactive applications. The dense text description is one of the research areas where systems learn the semantic knowledge and visual features of each video frame and map them to describe the video's most relevant subjects and events. In this paper, we consider untrimmed sports videos as our case study. Generating dense descriptions in the sports domain to supplement journalistic works without relying on commentators and experts requires more investigation. Motivated by this, we propose an end-to-end automated text-generator, SpecTextor, that learns the semantic features from untrimmed videos of sports games and generates associated descriptive texts. The proposed approach considers the video as a sequence of frames and sequentially generates words. After splitting videos into frames, we use a pre-trained VGG-16 model for feature extraction and encoding the video frames. With these encoded frames, we posit a Long Short-Term Memory (LSTM) based attention-decoder pipeline that leverages soft-attention mechanism to map the semantic features with relevant textual descriptions to generate the explanation of the game. Because developing a comprehensive description of the game warrants training on a set of dense time-stamped captions, we leverage two available public datasets: ActivityNet Captions and Microsoft Video Description. In addition, we utilized two different decoding algorithms: beam search and greedy search and computed two evaluation metrics: BLEU and METEOR scores.more » « less
-
Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months–3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing “LEFT” versus “RIGHT” and “ON” versus “OFF” looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.more » « less
-
Growing empirical evidence from the past quarter century reveals surprising sociality in newborns—infants in the first 28 postnatal days—including their ability to elicit and sustain contingent interactions with mutual gaze, social smiling, and sensitively timed, speech-like vocalizations. Newborns seem to have communicative expectations and behave as if they predict others’ goal-directed actions. Despite these discoveries, I review key barriers to progress in newborn developmental science. First, newborn social behavior research has almost exclusively focused on “average” development—based primarily on White, wealthy, English-speaking, Western, populations—treating interindividual differences as noise rather than meaningful, variability. Focusing almost exclusively on averages, especially with small sample sizes, ignores interindividual differences and hinders discoveries. Second, there are few studies of newborn sociality beyond the first postnatal week. In part, this gap in our understanding may be due to, and a consequence of, the mischaracterizations of newborns’ behaviors as passive, limited, disorganized, and low-level reflexes that are subcortically driven. Finally, researchers often assume that newborns’ behaviors are largely independent of experience. To the contrary, newborns’ need for nearly continuous social contact provides them with rich social learning opportunities, which have been shown to have lasting impacts on their development. Given the uniqueness and plasticity of this period, and their high vulnerability, developmental scientists are doing newborns a disservice by neglecting to characterize their social repertoires within and across diverse populations. Awareness of newborns’ social capacities will facilitate a more objective, accurate view of their social potential.more » « less
-
Background: Early-life pain is associated with adverse neurodevelopmental consequences; and current pain assessment practices are discontinuous, inconsistent, and highly dependent on nurses’ availability. Furthermore, facial expressions in commonly used pain assessment tools are not associated with brain-based evidence of pain. Purpose: To develop and validate a machine learning (ML) model to classify pain. Methods: In this retrospective validation study, using a human-centered design for Embedded Machine Learning Solutions approach and the Neonatal Facial Coding System (NFCS), 6 experienced neonatal intensive care unit (NICU) nurses labeled data from randomly assigned iCOPEvid (infant Classification Of Pain Expression video) sequences of 49 neonates undergoing heel lance. NFCS is the only observational pain assessment tool associated with brain-based evidence of pain. A standard 70% training and 30% testing split of the data was used to train and test several ML models. NICU nurses’ interrater reliability was evaluated, and NICU nurses’ area under the receiver operating characteristic curve (AUC) was compared with the ML models’ AUC. Results: Nurses weighted mean interrater reliability was 68% (63%-79%) for NFCS tasks, 77.7% (74%-83%) for pain intensity, and 48.6% (15%-59%) for frame and 78.4% (64%-100%) for video pain classification, with AUC of 0.68. The best performing ML model had 97.7% precision, 98% accuracy, 98.5% recall, and AUC of 0.98. Implications for Practice and Research: The pain classification ML model AUC far exceeded that of NICU nurses for identifying neonatal pain. These findings will inform the development of a continuous, unbiased, brain-based, nurse-in-the-loop Pain Recognition Automated Monitoring System (PRAMS) for neonates and infants.more » « less
