More Like this

1. Object Motion Guided Human Motion Synthesis

   https://doi.org/10.1145/3618333  

   Li, Jiaman; Wu, Jiajun; Liu, C Karen (December 2023, ACM Transactions on Graphics)

   Modeling human behaviors in contextual environments has a wide range of applications in character animation, embodied AI, VR/AR, and robotics. In real-world scenarios, humans frequently interact with the environment and manipulate various objects to complete daily tasks. In this work, we study the problem of full-body human motion synthesis for the manipulation of large-sized objects. We propose Object MOtion guided human MOtion synthesis (OMOMO), a conditional diffusion framework that can generate full-body manipulation behaviors from only the object motion. Since naively applying diffusion models fails to precisely enforce contact constraints between the hands and the object, OMOMO learns two separate denoising processes to first predict hand positions from object motion and subsequently synthesize full-body poses based on the predicted hand positions. By employing the hand positions as an intermediate representation between the two denoising processes, we can explicitly enforce contact constraints, resulting in more physically plausible manipulation motions. With the learned model, we develop a novel system that captures full-body human manipulation motions by simply attaching a smartphone to the object being manipulated. Through extensive experiments, we demonstrate the effectiveness of our proposed pipeline and its ability to generalize to unseen objects. Additionally, as high-quality human-object interaction datasets are scarce, we collect a large-scale dataset consisting of 3D object geometry, object motion, and human motion. Our dataset contains human-object interaction motion for 15 objects, with a total duration of approximately 10 hours. 

   more » « less
2. Motion Tracking for Volumetric Motion Capture Data

   https://doi.org/10.1109/MASSW.2019.00025  

   Roberts, Derek; Zhu, Ying (November 2019, 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems Workshops (MASSW))

   null (Ed.)
3. Kinematic Motion Analysis with Volumetric Motion Capture

   https://doi.org/10.1109/IV56949.2022.00019  

   Zhu, Ying; Detig, Cameron; Kane, Steven; Lourie, Gary (July 2022, International Conference Information Visualisation (IV))

   Kinematic motion analysis is widely used in health-care, sports medicine, robotics, biomechanics, sports science, etc. Motion capture systems are essential for motion analysis. There are three types of motion capture systems: marker-based capture, vision-based capture, and volumetric capture. Marker-based motion capture systems can achieve fairly accurate results but attaching markers to a body is inconvenient and time-consuming. Vision-based, marker-less motion capture systems are more desirable because of their non-intrusiveness and flexibility. Volumetric capture is a newer and more advanced marker-less motion capture system that can reconstruct realistic, full-body, animated 3D character models. But volumetric capture has rarely been used for motion analysis because volumetric motion data presents new challenges. We propose a new method for conducting kinematic motion analysis using volumetric capture data. This method consists of a three-stage pipeline. First, the motion is captured by a volumetric capture system. Then the volumetric capture data is processed using the Iterative Closest Points (ICP) algorithm to generate virtual markers that track the motion. Third, the motion tracking data is imported into the biomechanical analysis tool OpenSim for kinematic motion analysis. Our motion analysis method enables users to apply numerical motion analysis to the skeleton model in OpenSim while also studying the full-body, animated 3D model from different angles. It has the potential to provide more detailed and in-depth motion analysis for areas such as healthcare, sports science, and biomechanics. 

   more » « less
4. Motion by mean curvature and Dyson Brownian Motion

   https://doi.org/10.1214/23-ECP540  

   Huang, Ching-Peng; Inauen, Dominik; Menon, Govind (January 2023, Electronic Communications in Probability)

   We construct Dyson Brownian motion for β ∈ (0, ∞] by adapting the extrinsic construc- tion of Brownian motion on Riemannian manifolds to the geometry of group orbits within the space of Hermitian matrices. When β is infinite, the eigenvalues evolve by Coulombic repulsion and the group orbits evolve by motion by (minus one half times) mean curvature. 

   more » « less
5. Editing Motion Graphics Video via Motion Vectorization and Transformation

   https://doi.org/10.1145/3618316  

   Zhang, Sharon; Ma, Jiaju; Wu, Jiajun; Ritchie, Daniel; Agrawala, Maneesh (December 2023, ACM Transactions on Graphics)

   Motion graphics videos are widely used in Web design, digital advertising, animated logos and film title sequences, to capture a viewer's attention. But editing such video is challenging because the video provides a low-level sequence of pixels and frames rather than higher-level structure such as the objects in the video with their corresponding motions and occlusions. We present amotion vectorizationpipeline for converting motion graphics video into an SVG motion program that provides such structure. The resulting SVG program can be rendered using any SVG renderer (e.g. most Web browsers) and edited using any SVG editor. We also introduce aprogram transformationAPI that facilitates editing of a SVG motion program to create variations that adjust the timing, motions and/or appearances of objects. We show how the API can be used to create a variety of effects including retiming object motion to match a music beat, adding motion textures to objects, and collision preserving appearance changes. 

   more » « less