More Like this

1. DIM: Dyadic Interaction Modeling for Social Behavior Generation

   https://doi.org/10.1007/978-3-031-72913-3_27  

   Tran, Minh; Chang, Di; Siniukov, Maksim; Soleymani, Mohammad (December 2024, Springer Nature Switzerland)

   Leonardis, Aleš; Ricci, Eliss; Roth, Stefan; Russakovsky, Olga; Sattler, Torsten; Varol, Gul (Ed.)

   Human-human communication is like a delicate dance where listeners and speakers concurrently interact to maintain conversational dynamics. Hence, an effective model for generating listener nonverbal behaviors requires understanding the dyadic context and interaction. In this paper, we present an effective framework for creating 3D facial motions in dyadic interactions. Existing work consider a listener as a reactive agent with reflexive behaviors to the speaker’s voice and facial motions. The heart of our framework is Dyadic Interaction Modeling (DIM), a pre-training approach that jointly models speakers’ and listeners’ motions through masking and contrastive learning to learn representations that capture the dyadic context. To enable the generation of non-deterministic behaviors, we encode both listener and speaker motions into discrete latent representations, through VQ-VAE. The pre-trained model is further fine-tuned for motion generation. Extensive experiments demonstrate the superiority of our framework in generating listener motions, establishing a new state-of-the-art according to the quantitative measures capturing the diversity and realism of generated motions. Qualitative results demonstrate the superior capabilities of the proposed approach in generating diverse and realistic expressions, eye blinks and head gestures. 

   more » « less
2. Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation

   https://doi.org/10.1038/s41598-024-74244-4  

   Cui, Zixin; Sato, Tetsuya; Jackson, Austin; Jayarathna, Sampath; Itoh, Makoto; Yamani, Yusuke (December 2024, Scientific Reports)

   Abstract Real-world work environments require operators to perform multiple tasks with continual support from an automated system. Eye movement is often used as a surrogate measure of operator attention, yet conventional summary measures such as percent dwell time do not capture dynamic transitions of attention in complex visual workspace. This study analyzed eye movement data collected in a controlled a MATB-II task environment using gaze transition entropy analysis. In the study, human subjects performed a compensatory tracking task, a system monitoring task, and a communication task concurrently. The results indicate that both gaze transition entropy and stationary gaze entropy, measures of randomness in eye movements, decrease when the compensatory tracking task required more continuous monitoring. The findings imply that gaze transition entropy reflects attention allocation of operators performing dynamic operational tasks consistently. 

   more » « less
3. Are My Students Engaged? Nonverbal Interactions as an Indicator of Engagement in a Stadium-Style Lecture Hall

   https://doi.org/10.1080/0047231X.2024.2398386  

   States, Nicole E; Bruno, Carina; Martin, Karsten; Cole, Renée S (September 2024, Journal of College Science Teaching)

   Many large introductory classes are taught in stadium-style classrooms, which makes group work more difficult due to the room layout and immobile seating. These classrooms may create challenges for an instructor who wants to monitor student engagement because the layouts make it difficult to interact with the students as they work. Student nonverbal actions, such as eyes on the paper or an unsettled gaze, can be used to determine when students are actively engaged during group work. While other methods have been implemented to determine student actions during a class period, in larger settings these protocols require time-consuming data collection and cannot give in-the-moment feedback. In this study, student verbal and nonverbal interactions were analyzed and compared to determine the types of nonverbal interactions students take when collaboratively engaging in group work during lectures. It was found that a larger variety of nonverbal interactions, such as gesturing and leaning, were used when students were collaboratively working within their groups. Instructors of large enrollment classrooms can use the results of this work to aid in their facilitation of group work within stadium-style classrooms. 

   more » « less
4. Discovering the Fabric of Supportive Conversations: A Typology of Speaking Turns and Their Contingencies

   https://doi.org/10.1177/0261927X20953604  

   Bodie, Graham_D; Jones, Susanne_M; Brinberg, Miriam; Joyer, Amy_M; Solomon, Denise_Haunani; Ram, Nilam (August 2020, Journal of Language and Social Psychology)

   This study examines messages that problem disclosers and supportive listeners enact during conversations about everyday stressors. We coded 402 dyadic interactions between strangers, friends, and romantic couples using Stiles’s (1992) verbal response modes (VRM) and Burleson’s (1982) verbal person centeredness (PC) typology to explore whether (a) listener and discloser utterances coalesce into types of speaking turns, (b) listener turn types vary in person-centered quality, (c) listener turns relate to discloser responses, and (d) discloser responses relate to listener turns. Analyses revealed a typology for both listener and discloser turns: acknowledgment, advisement, question, elaboration, hedged disclosure, and reflection. The relative proportion of those types varied as a function of conversational role and relationship context, and these speech acts varied only minimally in PC. Configural frequency analyses revealed four greater-than-chance contingencies across data sets. The discussion highlights implications for a dyadic and dynamic understanding of supportive communication. 

   more » « less
5. The Roles of Moat Width and Outer Eyewall Contraction in Affecting the Timescale of Eyewall Replacement Cycle

   https://doi.org/10.1029/2024JD041488  

   Jiang, Jie; Wang, Yuqing (October 2024, Journal of Geophysical Research: Atmospheres)

   Abstract The timescale of eyewall replacement cycle (ERC) is critical for the prediction of intensity and structure changes of tropical cyclones (TCs) with concentric eyewall (CE) structures. Previous studies have indicated that the moat width can regulate the interaction between the inner and outer eyewalls and has a salient relationship with the ERC timescale. In this study, a series of sensitivity experiments are carried out to investigate the essential mechanisms resulting in the diversity of the duration of CEs using both simple and full‐physics models. Results reveal that a larger moat can induce stronger inflow under the same inner eyewall intensity by providing a longer distance for air parcels to accelerate in the boundary layer. Thus, there is greater inward absolute vorticity flux to sustain the inner eyewall. Besides, the equivalent potential temperature (θ_e) budget indicates that the vertical advection and surface flux of moist entropy can overbalance the negative contribution from the horizontal advection and lead to an increasing trend ofθ_ein the inner eyewall. This suggests that the thermodynamic process in the boundary layer is not indispensable to the inner eyewall weakening. It is also found that the contraction rate of the secondary eyewall, which directly influences the moat width, is subject to the activity of outer spiral rainbands. By directly introducing positive wind tendency outside the eyewall and indirectly promoting a vertically tilted eyewall structure, active convection in the outer region will impede or even suspend the contraction of the outer eyewall and hence extend the ERC timescale. 

   more » « less