More Like this

1. Compressive phase retrieval: Optimal sample complexity with deep generative priors

   https://doi.org/10.1002/cpa.22155  

   Hand, Paul; Leong, Oscar; Voroninski, Vladislav (September 2023, Communications on Pure and Applied Mathematics)

   Abstract Advances in compressive sensing (CS) provided reconstruction algorithms of sparse signals from linear measurements with optimal sample complexity, but natural extensions of this methodology to nonlinear inverse problems have been met with potentially fundamental sample complexity bottlenecks. In particular, tractable algorithms for compressive phase retrieval with sparsity priors have not been able to achieve optimal sample complexity. This has created an open problem in compressive phase retrieval: under generic, phaseless linear measurements, are there tractable reconstruction algorithms that succeed with optimal sample complexity? Meanwhile, progress in machine learning has led to the development of new data‐driven signal priors in the form of generative models, which can outperform sparsity priors with significantly fewer measurements. In this work, we resolve the open problem in compressive phase retrieval and demonstrate that generative priors can lead to a fundamental advance by permitting optimal sample complexity by a tractable algorithm. We additionally provide empirics showing that exploiting generative priors in phase retrieval can significantly outperform sparsity priors. These results provide support for generative priors as a new paradigm for signal recovery in a variety of contexts, both empirically and theoretically. The strengths of this paradigm are that (1) generative priors can represent some classes of natural signals more concisely than sparsity priors, (2) generative priors allow for direct optimization over the natural signal manifold, which is intractable under sparsity priors, and (3) the resulting non‐convex optimization problems with generative priors can admit benign optimization landscapes at optimal sample complexity, perhaps surprisingly, even in cases of nonlinear measurements. 

   more » « less
2. Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias

   Stojanov, Stefan; Thai, Anh; Rehg, James M. (June 2021, IEEE Computer Society Conference on Computer Vision and Pattern Recognition)

   null (Ed.)

   It is widely accepted that reasoning about object shape is important for object recognition. However, the most powerful object recognition methods today do not explicitly make use of object shape during learning. In this work, motivated by recent developments in low-shot learning, findings in developmental psychology, and the increased use of synthetic data in computer vision research, we investigate how reasoning about 3D shape can be used to improve low-shot learning methods’ generalization performance. We propose a new way to improve existing low-shot learning approaches by learning a discriminative embedding space using 3D object shape, and using this embedding by learning how to map images into it. Our new approach improves the performance of image-only low-shot learning approaches on multiple datasets. We also introduce Toys4K, a 3D object dataset with the largest number of object categories currently available, which supports low-shot learning. 

   more » « less
3. A nonlocal prior in iterative CT reconstruction

   https://doi.org/10.1002/mp.17533  

   Shu, Ziyu; Entezari, Alireza (December 2024, Medical Physics)

   Abstract BackgroundComputed tomography (CT) reconstruction problems are always framed as inverse problems, where the attenuation map of an imaged object is reconstructed from the sinogram measurement. In practice, these inverse problems are often ill‐posed, especially under few‐view and limited‐angle conditions, which makes accurate reconstruction challenging. Existing solutions use regularizations such as total variation to steer reconstruction algorithms to the most plausible result. However, most prevalent regularizations rely on the same priors, such as piecewise constant prior, hindering their ability to collaborate effectively and further boost reconstruction precision. PurposeThis study aims to overcome the aforementioned challenge a prior previously limited to discrete tomography. This enables more accurate reconstructions when the proposed method is used in conjunction with most existing regularizations as they utilize different priors. The improvements will be demonstrated through experiments conducted under various conditions. MethodsInspired by the discrete algebraic reconstruction technique (DART) algorithm for discrete tomography, we find out that pixel grayscale values in CT images are not uniformly distributed and are actually highly clustered. Such discovery can be utilized as a powerful prior for CT reconstruction. In this paper, we leverage the collaborative filtering technique to enable the collaboration of the proposed prior and most existing regularizations, significantly enhancing the reconstruction accuracy. ResultsOur experiments show that the proposed method can work with most existing regularizations and significantly improve the reconstruction quality. Such improvement is most pronounced under limited‐angle and few‐view conditions. Furthermore, the proposed regularization also has the potential for further improvement and can be utilized in other image reconstruction areas. ConclusionsWe propose improving the performance of iterative CT reconstruction algorithms by applying the collaborative filtering technique along with a prior based on the densely clustered distribution of pixel grayscale values in CT images. Our experimental results indicate that the proposed methodology consistently enhances reconstruction accuracy when used in conjunction with most existing regularizations, particularly under few‐view and limited‐angle conditions. 

   more » « less
4. Optimal multi-object segmentation with novel gradient vector flow based shape priors

   https://doi.org/https://doi.org/10.1016/j.compmedimag.2018.08.004  

   Bai, Junjie; Shah, Abhay; Wu, Xiaodong (November 2018, Computerized medical imaging and graphics)

   Shape priors have been widely utilized in medical image segmentation to improve segmentation accuracy and robustness. A major way to encode such a prior shape model is to use a mesh representation, which is prone to causing self-intersection or mesh folding. Those problems require complex and expensive algorithms to mitigate. In this paper, we propose a novel shape prior directly embedded in the voxel grid space, based on gradient vector flows of a pre-segmentation. The flexible and powerful prior shape representation is ready to be extended to simultaneously segmenting multiple interacting objects with minimum separation distance constraint. The segmentation problem of multiple interacting objects with shape priors is formulated as a Markov Random Field problem, which seeks to optimize the label assignment (objects or background) for each voxel while keeping the label consistency between the neighboring voxels. The optimization problem can be efficiently solved with a single minimum s-t cut in an appropriately constructed graph. The proposed algorithm has been validated on two multi-object segmentation applications: the brain tissue segmentation in MRI images and the bladder/prostate segmentation in CT images. Both sets of experiments showed superior or competitive performance of the proposed method to the compared state-of-the-art methods. 

   more » « less
5. Fit2Form: 3D Generative Model for Robot Gripper Form Design

   Ha, Huy; Agrawal, Shubham; Song, Shuran (November 2020, Proceedings of the 2020 Conference on Robot Learning)

   The 3D shape of a robot’s end-effector plays a critical role in determining it’s functionality and overall performance. Many of today’s industrial applications rely on highly customized gripper design for a given task to ensure the system’s robustness and accuracy. However, the process of manual hardware design is both costly and time-consuming, and the quality of the design is also dependent on the engineer’s experience and domain expertise, which can easily be out-dated or inaccurate. The goal of this paper is to use machine learning algorithms to automate this design process and generate task-specific gripper designs that satisfy a set of pre-defined design objectives. We model the design objectives by training a Fitness network to predict their values for a pair of gripper fingers and a grasp object. This Fitness network is then used to provide training supervision to a 3D Generative network that produces a pair of 3D finger geometries for the target grasp object. Our experiments demonstrate that the proposed 3D generative design framework generates parallel jaw gripper finger shapes that achieve more stable and robust grasps as compared to other general-purpose and task-specific gripper design algorithms. 

   more » « less