More Like this

1. FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

   https://doi.org/10.1109/CVPR.2019.00665  

   Singh, Krishna Kumar; Ojha, Utkarsh; Lee, Yong Jae (June 2019, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR))

   We propose FineGAN, a novel unsupervised GAN framework, which disentangles the background, object shape, and object appearance to hierarchically generate images of fine-grained object categories. To disentangle the factors without any supervision, our key idea is to use information theory to associate each factor to a latent code, and to condition the relationships between the codes in a specific way to induce the desired hierarchy. Through extensive experiments, we show that FineGAN achieves the desired disentanglement to generate realistic and diverse images belonging to fine-grained classes of birds, dogs, and cars. Using FineGAN's automatically learned features, we also cluster real images as a first attempt at solving the novel problem of unsupervised fine-grained object category discovery. 

   more » « less
2. FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

   Singh, Krishna; Ojha, Utkarsh; Lee, Yong Jae (January 2019, IEEE Conference on Computer Vision and Pattern Recognition (CVPR))

   We propose FineGAN, a novel unsupervised GAN framework, which disentangles the background, object shape, and object appearance to hierarchically generate images of fine-grained object categories. To disentangle the factors without supervision, our key idea is to use information theory to associate each factor to a latent code, and to condition the relationships between the codes in a specific way to induce the desired hierarchy. Through extensive experiments, we show that FineGAN achieves the desired disentanglement to generate realistic and diverse images belonging to fine-grained classes of birds, dogs, and cars. Using FineGAN’s automatically learned features, we also cluster real images as a first attempt at solving the novel problem of unsupervised fine-grained object category discovery. Our code/models/demo can be found at https://github.com/kkanshul/finegan 

   more » « less
3. The Role of Shape for Domain Generalization on Sparsely-Textured Images

   https://doi.org/10.1109/CVPRW56347.2022.00560  

   Nazari, Narges Honarvar; Kovashka, Adriana (June 2022, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops)

   State-of-the-art object recognition methods do not generalize well to unseen domains. Work in domain generalization has attempted to bridge domains by increasing feature compatibility, but has focused on standard, appearance-based representations. We show the potential of shape-based representations to increase domain robustness. We compare two types of shape-based representations: one trains a convolutional network over edge features, and another computes a soft, dense medial axis transform. We show the complementary strengths of these representations for different types of domains, and the effect of the amount of texture that is preserved. We show that our shape-based techniques better leverage data augmentations for domain generalization, and are more effective at texture bias mitigation than shape-inducing augmentations. Finally, we show that when the convolutional network in state-of-the-art domain generalization methods is replaced with one that explicitly captures shape, we obtain improved results. 

   more » « less
4. Spatiotemporal 3D Models of Aging Fruit from Multi-view Time-Lapse Videos

   https://doi.org/10.1007/978-3-319-73603-7_38  

   Guo, Lintao; Quant, Hunter; Lamb, Nikolas; Lowit, Benjamin; Banerjee, Sean; Banerjee, Natasha Kholgade (February 2018, Multimedia Modeling (MMM))

   We provide an approach to reconstruct spatiotemporal 3D models of aging objects such as fruit containing time-varying shape and appearance using multi-view time-lapse videos captured by a microenvironment of Raspberry Pi cameras. Our approach represents the 3D structure of the object prior to aging using a static 3D mesh reconstructed from multiple photographs of the object captured using a rotating camera track. We manually align the 3D mesh to the images at the first time instant. Our approach automatically deforms the aligned 3D mesh to match the object across the multi-viewpoint time-lapse videos. We texture map the deformed 3D meshes with intensities from the frames at each time instant to create the spatiotemporal 3D model of the object. Our results reveal the time dependence of volume loss due to transpiration and color transformation due to enzymatic browning on banana peels and in exposed parts of bitten fruit. 

   more » « less
5. Shape From Sky: Polarimetric Normal Recovery Under the Sky

   Ichikawa, T. (June 2021, IEEE Conference on Computer Vision and Pattern Recognition)

   null (Ed.)

   The sky exhibits a unique spatial polarization pattern by scattering the unpolarized sun light. Just like insects use this unique angular pattern to navigate, we use it to map pixels to directions on the sky. That is, we show that the unique polarization pattern encoded in the polarimetric appearance of an object captured under the sky can be decoded to reveal the surface normal at each pixel. We derive a polarimetric reflection model of a diffuse plus mirror surface lit by the sun and a clear sky. This model is used to recover the per-pixel surface normal of an object from a single polarimetric image or from multiple polarimetric images captured under the sky at different times of the day. We experimentally evaluate the accuracy of our shape-from-sky method on a number of real objects of different surface compositions. The results clearly show that this passive approach to fine-geometry recovery that fully leverages the unique illumination made by nature is a viable option for 3D sensing. With the advent of quad-Bayer polarization chips, we believe the implications of our method span a wide range of domains. 

   more » « less