More Like this

1. The Impact of Multi-Character Story Distribution and Gesture on Children’s Engagement

   Smith, Harrison Jesse ; Riley, Brian K. ; Reed, Lena ; Harrison, Vrindavan ; Walker, Marilyn ; Neff, Michael ( November 2019 , Interactive Storytelling (Lecture Notes in Computer Science))

   Effective storytelling relies on engagement and interaction. This work develops an automated software platform for telling stories to children and investigates the impact of two design choices on children’s engagement and willingness to interact with the system: story distribution and the use of complex gesture. A storyteller condition compares stories told in a third person, narrator voice with those distributed between a narrator and first-person story characters. Basic gestures are used in all our storytellings, but, in a second factor, some are augmented with gestures that indicate conversational turn changes, references to other characters and prompt children to ask questions.more »An analysis of eye gaze indicates that children attend more to the story when a distributed storytelling model is used. Gesture prompts appear to encourage children to ask questions, something that children did, but at a relatively low rate. Interestingly, the children most frequently asked “why” questions. Gaze switching happened more quickly when the story characters began to speak than for narrator turns. These results have implications for future agent-based storytelling system research.« less
2. Towards a Gesture-Based Story Authoring System: Design Implications from Feature Analysis of Iconic Gestures During Storytelling

   https://doi.org/10.1007/978-3-030-33894-7_38  

   Brown, S. A. ; Chu, S. L. ; Quek, F. ; Canaday, P. ; Li, Q. ; Loustau, T. ; Wu, S. ; Zhang, L. ( November 2019 , Lecture notes in computer science)

   Current systems that use gestures to enable storytelling tend to mostly rely on a pre-scripted set of gestures or the use of manipulative gestures with respect to tangibles. Our research aims to inform the design of gesture recognition systems for storytelling with implications derived from a feature-based analysis of iconic gestures that occur during naturalistic oral storytelling. We collected story retellings of a collection of cartoon stimuli from 20 study participants, and a gesture analysis was performed on videos of the story retellings focusing on iconic gestures. Iconic gestures are a type of representational gesture that provides information about objectsmore »such as their shape, location, or movement. The form features of the iconic gestures were analyzed with respect to the concepts that they portrayed. Patterns between the two were identified and used to create recommendations for patterns in gesture form a system could be primed to recognize.« less
3. A Model-free Affective Reinforcement Learning Approach to Personalization of an Autonomous Social Robot Companion for Early Literacy

   Park, H.W. ; Grover, I. ; Spaulding, S. ; Gomez, L. ; Breazeal, C. ( January 2019 , Proceedings of the ... AAAI Conference on Artificial Intelligence)

   Personalized education technologies capable of delivering adaptive interventions could play an important role in addressing the needs of diverse young learners at a critical time of school readiness. We present an innovative personalized social robot learning companion system that utilizes children’s verbal and nonverbal affective cues to modulate their engagement and maximize their long-term learning gains. We propose an affective reinforcement learning approach to train a personalized policy for each student during an educational activity where a child and a robot tell stories to each other. Using the personalized policy, the robot selects stories that are optimized for each child’smore »engagement and linguistic skill progression. We recruited 67 bilingual and English language learners between the ages of 4–6 years old to participate in a between-subjects study to evaluate our system. Over a three-month deployment in schools, a unique storytelling policy was trained to deliver a personalized story curriculum for each child in the Personalized group. We compared their engagement and learning outcomes to a Non-personalized group with a fixed curriculum robot, and a baseline group that had no robot intervention. In the Personalization condition, our results show that the affective policy successfully personalized to each child to boost their engagement and outcomes with respect to learning and retaining more target words as well as using more target syntax structures as compared to children in the other groups.« less
4. StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement

   https://doi.org/10.1145/3491102.3517479  

   Zhang, Zheng ; Xu, Ying ; Wang, Yanhao ; Yao, Bingsheng ; Ritchie, Daniel ; Wu, Tongshuang ; Yu, Mo ; Wang, Dakuo ; Li, Toby Jia-Jun ( April 2022 , Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems)

   Despite its benefits for children’s skill development and parent-child bonding, many parents do not often engage in interactive storytelling by having story-related dialogues with their child due to limited availability or challenges in coming up with appropriate questions. While recent advances made AI generation of questions from stories possible, the fully-automated approach excludes parent involvement, disregards educational goals, and underoptimizes for child engagement. Informed by need-finding interviews and participatory design (PD) results, we developed StoryBuddy, an AI-enabled system for parents to create interactive storytelling experiences. StoryBuddy’s design highlighted the need for accommodating dynamic user needs between the desire for parentmore »involvement and parent-child bonding and the goal of minimizing parent intervention when busy. The PD revealed varied assessment and educational goals of parents, which StoryBuddy addressed by supporting configuring question types and tracking child progress. A user study validated StoryBuddy’s usability and suggested design insights for future parent-AI collaboration systems.« less
5. Investigating the Effects of Self-Avatars and Story-Relevant Avatars on Children's Creative Storytelling

   https://doi.org/10.1145/3313831.3376331  

   Zarei, Niloofar ; Chu, Sharon Lynn ; Quek, Francis ; Rao, Nanjie 'Jimmy' ; Brown, Sarah Anne ( April 2020 , International Conference on Human Factors in Computing Systems)

   Storytelling is a critical step in the cognitive development of children. Particularly, this requires children to mentally project into the story context and to identify with the thoughts of the characters in their stories. We propose to support free imagination in creative storytelling through an enactment- based approach that allows children to embody an avatar and perform as the story character. We designed our story creation interface with two modes of avatar: the story-relevant avatar and the self-avatar, to investigate the effects of avatar design on the quality of children’s creative products. In our study with 20 child participants, themore »results indicate that self-avatars can create a stronger sense of identification and embodied presence, while story-relevant avatars can provide a scaffold for mental projection.« less