More Like this

1. Hand Gesture Recognition for Blind Users by Tracking 3D Gesture Trajectory

   https://doi.org/10.1145/3613904.3642602  

   Khanna, Prerna; Ramakrishnan, IV; Jain, Shubham; Bi, Xiaojun; Balasubramanian, Aruna (May 2024, ACM)
2. Multi-objective adversarial gesture generation

   https://doi.org/10.1145/3359566.3360053  

   Ferstl, Ylva; Neff, Michael; McDonnell, Rachel (October 2019, Intelligent Virtual Agents)

   null (Ed.)
3. Abstract Meaning Representation for Gesture

   Richard Brutti, Lucia Donatelli (June 2022, Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies)
4. Tunable tension for gesture animation

   Michael Neff (September 2022, IVA '22: Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents)

   Variation in muscular tension has important expressive impacts on agent motion; however, it is difficult to tune simulations to achieve particular effects. With a focus on gesture animation, we introduce mass trackers, a lightweight approach that employs proportional derivative control to track point masses that define the position of each wrist. The restriction to point masses allows the derivation of response functions that support straightforward tuning of system behavior. Using the point mass as an end-effector for an inverse kinematics rig allows easy control of both loose and high tension arm motion. Examples illustrate the expressive variation that can be achieved with this tension modulation. Two perceptual studies confirm that these changes impact the overall level of tension perceived in the motion of a gesturing character and further explore the parameter space. Practical guidelines on tuning are discussed. 

   more » « less
5. Abstract Meaning Representation for Gesture

   Brutti, Richard; Donatelli, Lucia; Lai, Kenneth; Pustejovsky, James (June 2022, Proceedings of the Thirteenth Language Resources and Evaluation Conference)

   This paper presents Gesture AMR, an extension to Abstract Meaning Representation (AMR), that captures the meaning of gesture. In developing Gesture AMR, we consider how gesture form and meaning relate; how gesture packages meaning both independently and in interaction with speech; and how the meaning of gesture is temporally and contextually determined. Our case study for developing Gesture AMR is a focused human-human shared task to build block structures. We develop an initial taxonomy of gesture act relations that adheres to AMR{'}s existing focus on predicate-argument structure while integrating meaningful elements unique to gesture. Pilot annotation shows Gesture AMR to be more challenging than standard AMR, and illustrates the need for more work on representation of dialogue and multimodal meaning. We discuss challenges of adapting an existing meaning representation to non-speech-based modalities and outline several avenues for expanding Gesture AMR. 

   more » « less