More Like this

1. MOVES: Manipulated Objects in Video Enable Segmentation

   https://doi.org/10.1109/CVPR52729.2023.00613  

   Higgins, Richard E.L.; Fouhey, David F. (June 2023, CVPR)

   Our method uses manipulation in video to learn to understand held-objects and hand-object contact. We train a system that takes a single RGB image and produces a pixel-embedding that can be used to answer grouping questions (do these two pixels go together) as well as hand-association questions (is this hand holding that pixel). Rather than painstakingly annotate segmentation masks, we observe people in realistic video data. We show that pairing epipolar geometry with modern optical flow produces simple and effective pseudo-labels for grouping. Given people segmentations, we can further associate pixels with hands to understand contact. Our system achieves competitive results on hand and hand-held object tasks. 

   more » « less
2. SimJEB: Simulated Jet Engine Bracket Dataset

   https://doi.org/10.1111/cgf.14353  

   Whalen, E.; Beyene, A.; Mueller, C. (August 2021, Computer Graphics Forum)

   Abstract This paper introduces the Simulated Jet Engine Bracket Dataset (SimJEB) [WBM21]: a new, public collection of crowdsourced mechanical brackets and accompanying structural simulations. SimJEB is applicable to a wide range of geometry processing tasks; the complexity of the shapes in SimJEB offer a challenge to automated geometry cleaning and meshing, while categorical labels and structural simulations facilitate classification and regression (i.e. engineering surrogate modeling). In contrast to existing shape collections, SimJEB's models are all designed for the same engineering function and thus have consistent structural loads and support conditions. On the other hand, SimJEB models are more complex, diverse, and realistic than the synthetically generated datasets commonly used in parametric surrogate model evaluation. The designs in SimJEB were derived from submissions to the GrabCAD Jet Engine Bracket Challenge: an open engineering design competition with over 700 hand‐designed CAD entries from 320 designers representing 56 countries. Each model has been cleaned, categorized, meshed, and simulated with finite element analysis according to the original competition specifications. The result is a collection of 381 diverse, high‐quality and application‐focused designs for advancing geometric deep learning, engineering surrogate modeling, automated cleaning and related geometry processing tasks. 

   more » « less
3. Learning Conditional Generative Models for Temporal Point Processes

   Xiao, S.; Xu, H.; Yan, J.; Farajtarbar, M.; Yang, X.; Song, L.; Zha, H. (January 2018, Proceedings of the AAAI Conference on Artificial Intelligence)

   Estimating the future event sequence conditioned on current observations is a long-standing and challenging task in temporal analysis. On one hand for many real-world problems the underlying dynamics can be very complex and often unknown. This renders the traditional parametric point process models often fail to fit the data for their limited capacity. On the other hand, long-term prediction suffers from the problem of bias exposure where the error accumulates and propagates to future prediction. Our new model builds upon the sequence to sequence (seq2seq) prediction network. Compared with parametric point process models, its modeling capacity is higher and has better flexibility for fitting real-world data. The main novelty of the paper is to mitigate the second challenge by introducing the likelihood-free loss based on Wasserstein distance between point processes, besides negative maximum likelihood loss used in the traditional seq2seq model. Wasserstein distance, unlike KL divergence i.e. MLE loss, is sensitive to the underlying geometry between samples and can robustly enforce close geometry structure between them. This technique is proven able to improve the vanilla seq2seq model by a notable margin on various tasks. 

   more » « less
4. Fermions in loop quantum gravity and resolution of doubling problem

   https://doi.org/10.1088/1361-6382/acf26b  

   Zhang, Cong; Liu, Hongguang; Han, Muxin (September 2023, Classical and Quantum Gravity)

   Abstract The fermion propagator is derived in detail from the model of fermion coupled to loop quantum gravity (LQG). As an ingredient of the propagator, the vacuum state is defined as the ground state of some effective fermion Hamiltonian under the background geometry given by a coherent state resembling the classical Minkowski spacetime. Moreover, as a critical feature of LQG, the superposition over graphs is employed to define the vacuum state. It turns out that the graph superposition leads to the propagator being the average of the propagators of the lattice field theory over various graphs so that all fermion doubler modes are suppressed in the propagator. This resolves the doubling problem in LQG. Our result suggests that the superposition nature of quantum geometry should, on the one hand, resolve the tension between fermion and the fundamental discreteness and, on the other hand, relate to the continuum limit of quantum gravity. 

   more » « less
5. Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulation

   https://doi.org/10.1109/ICRA.2018.8460496  

   Sundaralingam, Balakumar; Hermans, Tucker (May 2018, 10.1109/ICRA.2018.8460496)

   This paper explores the problem of autonomous, in-hand regrasping-the problem of moving from an initial grasp on an object to a desired grasp using the dexterity of a robot's fingers. We propose a planner for this problem which alternates between finger gaiting, and in-grasp manipulation. Finger gaiting enables the robot to move a single finger to a new contact location on the object, while the remaining fingers stably hold the object. In-grasp manipulation moves the object to a new pose relative to the robot's palm, while maintaining the contact locations between the hand and object. Given the object's geometry (as a mesh), the hand's kinematic structure, and the initial and desired grasps, we plan a sequence of finger gaits and object reposing actions to reach the desired grasp without dropping the object. We propose an optimization based approach and report in-hand regrasping plans for 5 objects over 5 in-hand regrasp goals each. The plans generated by our planner are collision free and guarantee kinematic feasibility. 

   more » « less