An official website of the United States government
This paper proposes SquiggleMilli, a system that approximates traditional Synthetic Aperture Radar (SAR) imaging on mobile millimeter-wave (mmWave) devices. The system is capable of imaging through obstructions, such as clothing, and under low visibility conditions. Unlike traditional SAR that relies on mechanical controllers or rigid bodies, SquiggleMilli is based on the hand-held, fluidic motion of the mmWave device. It enables mmWave imaging in hand-held settings by re-thinking existing motion compensation, compressed sensing, and voxel segmentation. Since mmWave imaging suffers from poor resolution due to specularity and weak reflectivity, the reconstructed shapes could be imperceptible by machines and humans. To this end, SquiggleMilli designs a machine learning model to recover the high spatial frequencies in the object to reconstruct an accurate 2D shape and predict its 3D features and category. We have customized SquiggleMilli for security applications, but the model is adaptable to other applications with limited training samples. We implement SquiggleMilli on off-the-shelf components and demonstrate its performance improvement over the traditional SAR qualitatively and quantitatively.
more »
« less
COHESIV: Contrastive Object and Hand Embeddings for Segmentation In Video
In this paper we learn to segment hands and hand-held objects from motion. Our system takes a single RGB image and hand location as input to segment the hand and hand-held object. For learning, we generate responsibility maps that show how well a hand’s motion explains other pixels’ motion in video. We use these responsibility maps as pseudo-labels to train a weakly-supervised neural network using an attention-based similarity loss and contrastive loss. Our system outperforms alternate methods, achieving good performance on the 100DOH, EPIC-KITCHENS, and HO3D datasets.
more »
« less
- Award ID(s):
- 2006619
- PAR ID:
- 10349042
- Date Published:
- Journal Name:
- Advances in neural information processing systems
- ISSN:
- 1049-5258
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Our method uses manipulation in video to learn to understand held-objects and hand-object contact. We train a system that takes a single RGB image and produces a pixel-embedding that can be used to answer grouping questions (do these two pixels go together) as well as hand-association questions (is this hand holding that pixel). Rather than painstakingly annotate segmentation masks, we observe people in realistic video data. We show that pairing epipolar geometry with modern optical flow produces simple and effective pseudo-labels for grouping. Given people segmentations, we can further associate pixels with hands to understand contact. Our system achieves competitive results on hand and hand-held object tasks.more » « less
-
This paper presents regional attraction of line segment maps, and hereby poses the problem of line segment detection (LSD) as a problem of region coloring. Given a line segment map, the proposed regional attraction first establishes the relationship between line segments and regions in the image lattice. Based on this, the line segment map is equivalently transformed to an attraction field map (AFM), which can be remapped to a set of line segments without loss of information. Accordingly, we develop an end-to-end framework to learn attraction field maps for raw input images, followed by a squeeze module to detect line segments. Apart from existing works, the proposed detector properly handles the local ambiguity and does not rely on the accurate identification of edge pixels. Comprehensive experiments on the Wireframe dataset and the YorkUrban dataset demonstrate the superiority of our method. In particular, we achieve an F-measure of 0.831 on the Wireframe dataset, advancing the state-of-the-art performance by 10.3 percent.more » « less
-
In this paper, we initiate the study of wave propagation in a recently proposed mathematical model for stretch-limited elastic strings. We consider the longitudinal motion of a simple class of uniform, semi-infinite, stretch-limited strings under no external force with finite end held fixed and prescribed tension at the infinite end. We study a class of motions such that the string has one inextensible segment, where the local stretch is maximized, and one extensible segment. The equations of motion reduce to a simple and novel shock front problem in one spatial variable for which we prove existence and uniqueness of local-in-time solutions for appropriate initial data. We then prove the orbital asymptotic stability of an explicit two-parameter family of piece-wise constant stretched motions. If the prescribed tension at the infinite end is increasing in time, our results provide an open set of initial data launching motions resulting in the string becoming fully inextensible and tension blowing up in finite time.more » « less
-
null (Ed.)Minimally invasive surgery is of high interest for interventional medicine since the smaller incisions can lead to less pain and faster recovery for patients. The current standard-of-care involves a range of affordable, manual, hand-held rigid tools, whose limited dexterity and range of adoptable shapes can prevent access to confined spaces. In contrast, recently developed roboticized tools that can provide increased accessibility and dexterity to navigate and perform complex tasks often come at the cost of larger, heavier, and grounded devices that are teleoperated, posing a new set of challenges. In this article, we propose a new hand-held concentric tube robot with an associated position control method that has the dexterity and precision of large roboticized devices, while maintaining the footprint of a traditional hand-held tool. The device shows human-in-the-loop control performance that meets the requirements of the targeted application, percutaneous abscess drainage. In addition, a small user study illustrates the advantage of combining rigid body motion of the device with more precise motions of the tip, thus showing the potential to bridge the gap between traditional hand-held tools and grounded robotic devices.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
