Search for: All records

Interactive media can be effectively enhanced through advancements in artificial intelligence (AI). Through a research–practice partnership with PBS KIDS, we integrated AI into popular PBS KIDS shows to support dialogic interactions that actively engage children in STEM learning. These interactions help ground abstract STEM concepts in socially meaningful experiences. In this work, we present an interactive web app designed to scaffold children’s learning and engagement at home. We discuss design considerations, challenges related to children’s speech recognition, and insights from iterative testing that informed the app’s current stage of development. 

Background Computational thinking (CT) is a crucial domain for children to develop in their early years. To increase children's access to CT learning resources, educational programs like PBS KIDS “Lyla in the Loop” have been developed to incorporate CT concepts through narrative structures where characters solve problems using the CT cycle. However, children need explicit guidance to effectively process both educational and narrative content. Engaging children in dialogues that connect educational content with the narrative has proven to enhance comprehension. Aims This study explores the effectiveness of using AI to enable this type of dialogues between children and a media character, supporting children in learning CT by connecting these concepts with everyday situations in “Lyla in the Loop.” Method Through a between-subject randomized control study with 160 children aged four to eight, we will compare children's learning and applications of CT concepts as well as narrative comprehension from AI-assisted dialogues to those who watched the broadcast version of the show without such dialogues. The study also examines the role of children's cognitive abilities and prior CT knowledge in their learning from the show, with or without AI-assisted dialogues. Expected results The findings could enhance our understanding of AI-based scaffolding strategies in children's media and offer practical implications for improving children's learning experiences. 

Recognizing the challenges bilingual children face in school readiness and the potential of bilingual dialogic shared reading in improving language and literacy, this study investigates the use of a bilingual conversational agent (CA) to enhance shared reading experiences in home environments. While current CAs hold promise in fostering young children's learning, they do not typically consider the linguistic and cultural needs of bilingual children and rarely involve parents intentionally. To this end, we developed a bilingual CA, embedded within ebooks, to support children's language learning and parent engagement for Latine Spanish-English bilingual families. A week-long home-based study with 15 families indicated that the bilingual CA elicited a high level of bilingual verbal engagement from children, thereby promoting their vocabulary acquisition. It also stimulated meaningful conversations among parents and children. This study provides design implications for developing CAs for bilingual children and parents. 

Deep reinforcement learning has demonstrated re- markable achievements across diverse domains such as video games, robotic control, autonomous driving, and drug discovery. Common methodologies in partially observable domains largely lean on end-to-end learning from high-dimensional observations, such as images, without explicitly reasoning about true state. We suggest an alternative direction, introducing the Partially Supervised Reinforcement Learning (PSRL) framework. At the heart of PSRL is the fusion of both supervised and unsupervised learning. The approach leverages a state estimator to distill supervised semantic state information from high-dimensional observations which are often fully observable at training time. This yields more interpretable policies that compose state predictions with control. In parallel, it captures an unsupervised latent representation. These two—the semantic state and the latent state—are then fused and utilized as inputs to a policy network. This juxtaposition offers practitioners a flexible and dynamic spectrum: from emphasizing supervised state information to integrating richer, latent insights. Extensive experimental results indicate that by merging these dual representations, PSRL offers a balance, enhancing interpretability while preserving, and often significantly outperforming, the performance benchmarks set by traditional methods in terms of reward and convergence speed.