More Like this

1. Guided Image Inpainting: Replacing an Image Region by Pulling Content From Another Image

   https://doi.org/10.1109/WACV.2019.00166  

   Zhao, Yinan ; Price, Brian ; Cohen, Scott ; Gurari, Danna ( January 2019 , IEEE Winter conference on Applications in Computer Vision)
2. Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs

   https://doi.org/10.1109/icra48891.2023.10160815  

   Xia, Youya ; Monica, Josephine ; Chao, Wei-Lun ; Hariharan, Bharath ; Weinberger, Kilian Q ; Campbell, Mark ( May 2023 , IEEE International Conference on Robotics and Automation (ICRA))
3. Image-to-Image Translation for Autonomous Driving from Coarsely-Aligned Image Pairs

   Xia, Youya ; Monica, Josephine ; Chao, Wei-Lun ; Hariharan, Bharath ; Weinberger, Kilian ; Campbell, Mark ( May 2023 , Proceedings IEEE International Conference on Robotics and Automation)

   A self-driving car must be able to reliably handle adverse weather conditions (e.g., snowy) to operate safely. In this paper, we investigate the idea of turning sensor inputs (i.e., images) captured in an adverse condition into a benign one (i.e., sunny), upon which the downstream tasks (e.g., semantic segmentation) can attain high accuracy. Prior work primarily formulates this as an unpaired image-to-image translation problem due to the lack of paired images captured under the exact same camera poses and semantic layouts. While perfectly- aligned images are not available, one can easily obtain coarsely- paired images. For instance, many people drive the same routes daily in both good and adverse weather; thus, images captured at close-by GPS locations can form a pair. Though data from repeated traversals are unlikely to capture the same foreground objects, we posit that they provide rich contextual information to supervise the image translation model. To this end, we propose a novel training objective leveraging coarsely- aligned image pairs. We show that our coarsely-aligned training scheme leads to a better image translation quality and improved downstream tasks, such as semantic segmentation, monocular depth estimation, and visual localization. 

   more » « less
4. CryoETGAN: Cryo-Electron Tomography Image Synthesis via Unpaired Image Translation

   https://doi.org/10.3389/fphys.2022.760404  

   Wu, Xindi ; Li, Chengkun ; Zeng, Xiangrui ; Wei, Haocheng ; Deng, Hong-Wen ; Zhang, Jing ; Xu, Min ( March 2022 , Frontiers in Physiology)

   Cryo-electron tomography (Cryo-ET) has been regarded as a revolution in structural biology and can reveal molecular sociology. Its unprecedented quality enables it to visualize cellular organelles and macromolecular complexes at nanometer resolution with native conformations. Motivated by developments in nanotechnology and machine learning, establishing machine learning approaches such as classification, detection and averaging for Cryo-ET image analysis has inspired broad interest. Yet, deep learning-based methods for biomedical imaging typically require large labeled datasets for good results, which can be a great challenge due to the expense of obtaining and labeling training data. To deal with this problem, we propose a generative model to simulate Cryo-ET images efficiently and reliably: CryoETGAN. This cycle-consistent and Wasserstein generative adversarial network (GAN) is able to generate images with an appearance similar to the original experimental data. Quantitative and visual grading results on generated images are provided to show that the results of our proposed method achieve better performance compared to the previous state-of-the-art simulation methods. Moreover, CryoETGAN is stable to train and capable of generating plausibly diverse image samples. 

   more » « less
5. Image inpainting using sparse multiscale representations: Image recovery performance guarantees

   https://doi.org/10.1016/j.acha.2020.05.001  

   Guo, Kanghui ; Labate, Demetrio ; Rodriguez Ayllon, Jose Pedro ( September 2020 , Applied and Computational Harmonic Analysis)