skip to main content

Title: Wassersplines for Neural Vector Field‐Controlled Animation

Much of computer‐generated animation is created by manipulating meshes with rigs. While this approach works well for animating articulated objects like animals, it has limited flexibility for animating less structured free‐form objects. We introduce Wassersplines, a novel trajectory inference method for animating unstructured densities based on recent advances in continuous normalizing flows and optimal transport. The key idea is to train a neurally‐parameterized velocity field that represents the motion between keyframes. Trajectories are then computed by advecting keyframes through the velocity field. We solve an additional Wasserstein barycenter interpolation problem to guarantee strict adherence to keyframes. Our tool can stylize trajectories through a variety of PDE‐based regularizers to create different visual effects. We demonstrate our tool on various keyframe interpolation problems to produce temporally‐coherent animations without meshing or rigging.

more » « less
Author(s) / Creator(s):
 ;  ;  
Publisher / Repository:
Date Published:
Journal Name:
Computer Graphics Forum
Page Range / eLocation ID:
p. 31-41
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Optical flow-based video interpolation is a commonly used method to enrich video details by generating new intermediate frames from existing ones. However, it cannot accurately reproduce the trajectories of irregular and fast-moving objects. To achieve the accurate reconstruction of high-speed scenes, incorporating event information during the interpolation process is an effective method. However, existing methods are not suitable for scenarios where events are sparse. To implement high-quality video interpolation for these scenarios, this study incorporates event information into the interpolation process with three-fold ideas: a) treating the moving target as the foreground and using events to delineate the target area, b) matching foreground to background based on the event information to generate temporal frame between keyframes, and c) generating intermediate frames between keyframe and temporal frame with optical flow interpolation. Empirical studies indicate that owing to its efficient incorporation of event information, the proposed framework outperforms state-of-the-art methods in generating high-quality frames for video interpolation. 
    more » « less
  2. Umetani, N. ; Wojtan, C. ; Vouga, E. (Ed.)
    Most non-photorealistic rendering (NPR) methods for line drawing synthesis operate on a static shape. They are not tailored to process animated 3D models due to extensive per-frame parameter tuning needed to achieve the intended look and natural transition. This paper introduces a framework for interactive line drawing synthesis from animated 3D models based on a learned style space for drawing representation and interpolation. We refer to style as the relationship between stroke placement in a line drawing and its corresponding geometric properties. Starting from a given sequence of an animated 3D character, a user creates drawings for a set of keyframes. Our system embeds the raster drawings into a latent style space after they are disentangled from the underlying geometry. By traversing the latent space, our system enables a smooth transition between the input keyframes. The user may also edit, add, or remove the keyframes interactively, similar to a typical keyframe-based workflow. We implement our system with deep neural networks trained on synthetic line drawings produced by a combination of NPR methods. Our drawing-specific supervision and optimization-based embedding mechanism allow generalization from NPR line drawings to user-created drawings during run time. Experiments show that our approach generates high-quality line drawing animations while allowing interactive control of the drawing style across frames. 
    more » « less
  3. Abstract

    The origins of the upper branch of the Atlantic meridional overturning circulation (AMOC) are traced with backward‐in‐time Lagrangian trajectories, quantifying the partition of volume transport between different routes of entry from the Indo‐Pacific into the Atlantic. Particles are advected by the velocity field from a recent release of “Estimating the Circulation and Climate of the Ocean” (ECCOv4). This global time‐variable velocity field is a dynamically consistent interpolation of over 1 billion oceanographic observations collected between 1992 and 2015. Of the 13.6 Sverdrups (1 Sv = 106 m3/s) flowing northward across 6°S, 15% enters the Atlantic from Drake Passage, 35% enters from the straits between Asia and Australia (Indonesian Throughflow), and 49% comes from the region south of Australia (Tasman Leakage). Because of blending in the Agulhas region, water mass properties in the South Atlantic are not a good indicator of origin.

    more » « less
  4. Swimming at the microscale typically involves two modes of motion: mechanical propulsion and propulsion due to field interactions. During mechanical propulsion, particles swim by reconfiguring their geometry. When propelled by field interactions, body forces such as phoretic interactions drive mobility. In this work, we employ slender-body theory to explore how a bent rod actuator propels due to a mechanical mode of swimming via hinge articulations and due to a chemical mode of swimming via diffusiophoretic interactions with a solute field. Although previous theoretical studies have examined mechanical and chemical modes of swimming in isolation, the simultaneous investigation of both modes has remained unexplored. For the mechanical mode of swimming, our calculations, both numerical and analytical, recover Purcell’s scallop theorem and show that the bent rod actuator experiences zero net displacement during reciprocal motion. Additionally, we calculate the trajectories traced by a bent rod actuator under a non-reciprocal hinge articulation, revealing that these trajectories are influenced by the amplitude of the hinge articulation, geometric asymmetry, and the angular velocity distribution between the two arms of the bent rod actuator. We provide intuitive explanations for these effects using free-body diagrams. Furthermore, we explore the motion induced by simultaneous hinge articulations and self-diffusiophoresis. We observe that hinge articulations can modify the effective phoretic forces and torques acting on the bent rod actuator, either supporting or impeding propulsion. Additionally, during self-diffusiophoretic propulsion, reciprocal hinge articulations no longer result in zero net displacement. In summary, our findings chart a new direction for designing micron-sized objects that harness both mechanical and chemical modes of propulsion synchronously, offering a mechanism to enact control over trajectories.

    more » « less
  5. Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, brings substantial challenges due to infinite shape variations, non-rigid motions, and partial observability. We introduce ACID, an action-conditional visual dynamics model for volumetric deformable objects based on structured implicit neural representations. ACID integrates two new techniques: implicit representations for action-conditional dynamics and geodesics-based contrastive learning. To represent deformable dynamics from partial RGB-D observations, we learn implicit representations of occupancy and flow-based forward dynamics. To accurately identify state change under large non-rigid deformations, we learn a correspondence embedding field through a novel geodesics-based contrastive loss. To evaluate our approach, we develop a simulation framework for manipulating complex deformable shapes in realistic scenes and a benchmark containing over 17,000 action trajectories with six types of plush toys and 78 variants. Our model achieves the best performance in geometry, correspondence, and dynamics predictions over existing approaches. The ACID dynamics models are successfully employed for goal-conditioned deformable manipulation tasks, resulting in a 30% increase in task success rate over the strongest baseline. Furthermore, we apply the simulation-trained ACID model directly to real-world objects and show success in manipulating them into target configurations.

    more » « less