Title: Fingerphoto Deblurring Using Attention-Guided Multi-Stage GAN
Using fingerphoto images acquired from mobile cameras, low-quality sensors, or crime scenes, it has become a challenge for automated identification systems to verify the identity due to various acquisition distortions. A significant type of photometric distortion that notably reduces the quality of a fingerphoto is the blurring of the image. This paper proposes a deep fingerphoto deblurring model to restore the ridge information degraded by the image blurring. As the core of our model, we utilize a conditional Generative Adversarial Network (cGAN) to learn the distribution of natural ridge patterns. We perform several modifications to enhance the quality of the reconstructed (deblurred) fingerphotos by our proposed model. First, we develop a multi-stage GAN to learn the ridge distribution in a coarse-to-fine framework. This framework enables the model to maintain the consistency of the ridge deblurring process at different resolutions. Second, we propose a guided attention module that helps the generator to focus mainly on blurred regions. Third, we incorporate a deep fingerphoto verifier as an auxiliary adaptive loss function to force the generator to preserve the ID information during the deblurring process. Finally, we evaluate the effectiveness of the proposed model through extensive experiments on multiple public fingerphoto datasets as well as real-world blurred fingerphotos. In particular, our method achieves 5.2 dB, 8.7%, and 7.6% improvement in PSNR, AUC, and EER, respectively, compared to a state-of-the-art deblurring method. more »« less
This paper studies a new convex variational model for denoising and deblurring images with multiplicative noise. Considering the statistical property of the multiplicative noise following Nakagami distribution, the denoising model consists of a data fidelity term, a quadratic penalty term, and a total variation regularization term. Here, the quadratic penalty term is mainly designed to guarantee the model to be strictly convex under a mild condition. Furthermore, the model is extended for the simultaneous denoising and deblurring case by introducing a blurring operator. We also study some mathematical properties of the proposed model. In addition, the model is solved by applying the primal-dual algorithm. The experimental results show that the proposed method is promising in restoring (blurred) images with multiplicative noise.
Sourav, Md_Sakib Galib; Yavari, Ehsan; Gao, Xiaomeng; Maskrey, James; Zheng, Yao; Lubecke, Victor M; Boric-Lubecke, Olga
(, Sensors)
Building occupancy information is significant for a variety of reasons, from allocation of resources in smart buildings to responding during emergency situations. As most people spend more than 90% of their time indoors, a comfortable indoor environment is crucial. To ensure comfort, traditional HVAC systems condition rooms assuming maximum occupancy, accounting for more than 50% of buildings’ energy budgets in the US. Occupancy level is a key factor in ensuring energy efficiency, as occupancy-controlled HVAC systems can reduce energy waste by conditioning rooms based on actual usage. Numerous studies have focused on developing occupancy estimation models leveraging existing sensors, with camera-based methods gaining popularity due to their high precision and widespread availability. However, the main concern with using cameras for occupancy estimation is the potential violation of occupants’ privacy. Unlike previous video-/image-based occupancy estimation methods, we addressed the issue of occupants’ privacy in this work by proposing and investigating both motion-based and motion-independent occupancy counting methods on intentionally blurred video frames. Our proposed approach included the development of a motion-based technique that inherently preserves privacy, as well as motion-independent techniques such as detection-based and density-estimation-based methods. To improve the accuracy of the motion-independent approaches, we utilized deblurring methods: an iterative statistical technique and a deep-learning-based method. Furthermore, we conducted an analysis of the privacy implications of our motion-independent occupancy counting system by comparing the original, blurred, and deblurred frames using different image quality assessment metrics. This analysis provided insights into the trade-off between occupancy estimation accuracy and the preservation of occupants’ visual privacy. The combination of iterative statistical deblurring and density estimation achieved a 16.29% counting error, outperforming our other proposed approaches while preserving occupants’ visual privacy to a certain extent. Our multifaceted approach aims to contribute to the field of occupancy estimation by proposing a solution that seeks to balance the trade-off between accuracy and privacy. While further research is needed to fully address this complex issue, our work provides insights and a step towards a more privacy-aware occupancy estimation system.
In this work we present a framework of designing iterative techniques for image deblurring in inverse problem. The new framework is based on two observations about existing methods. We used Landweber method as the basis to develop and present the new framework but note that the framework is applicable to other iterative techniques. First, we observed that the iterative steps of Landweber method consist of a constant term, which is a low-pass filtered version of the already blurry observation. We proposed a modification to use the observed image directly. Second, we observed that Landweber method uses an estimate of the true image as the starting point. This estimate, however, does not get updated over iterations. We proposed a modification that updates this estimate as the iterative process progresses. We integrated the two modifications into one framework of iteratively deblurring images. Finally, we tested the new method and compared its performance with several existing techniques, including Landweber method, Van Cittert method, GMRES (generalized minimal residual method), and LSQR (least square), to demonstrate its superior performance in image deblurring.
The underlying physics of imaging processes and associated instrumentation limitations mean that blurring artifacts are unavoidable in many applications such as astronomy, microscopy, radar and medical imaging. In several such imaging modalities, convolutional models are used to describe the blurring process; the observed image or function is a convolution of the true underlying image and a point spread function (PSF) which characterizes the blurring artifact. In this work, we propose and analyze a technique - based on convolutional edge detectors and Gaussian curve fitting - to approximate unknown Gaussian PSFs when the underlying true function is piecewise-smooth. For certain simple families of such functions, we show that this approximation is exponentially accurate. We also provide preliminary two dimensional extensions of this technique. These findings - confirmed by numerical simulations - demonstrate the feasibility of recovering accurate approximations to the blurring function, which serves as an important prerequisite to solving deblurring problems.
Alkhouri, Ismail; Liang, Shijun; Bell, Evan; Qu, Qing; Wang, Rongrong; Ravishankar, Saiprasad
(, Advances in Neural Information Processing Systems)
Recently, Deep Image Prior (DIP) has emerged as an effective unsupervised one-shot learner, delivering competitive results across various image recovery problems. This method only requires the noisy measurements and a forward operator, relying solely on deep networks initialized with random noise to learn and restore the structure of the data. However, DIP is notorious for its vulnerability to overfitting due to the overparameterization of the network. Building upon insights into the impact of the DIP input and drawing inspiration from the gradual denoising process in cutting-edge diffusion models, we introduce Autoencoding Sequential DIP (aSeqDIP) for image reconstruction. This method progressively denoises and reconstructs the image through a sequential optimization of network weights. This is achieved using an input-adaptive DIP objective, combined with an autoencoding regularization term. Compared to diffusion models, our method does not require training data and outperforms other DIP-based methods in mitigating noise overfitting while maintaining a similar number of parameter updates as Vanilla DIP. Through extensive experiments, we validate the effectiveness of our method in various image reconstruction tasks, such as MRI and CT reconstruction, as well as in image restoration tasks like image denoising, inpainting, and non-linear deblurring.
@article{osti_10496366,
place = {Country unknown/Code not available},
title = {Fingerphoto Deblurring Using Attention-Guided Multi-Stage GAN},
url = {https://par.nsf.gov/biblio/10496366},
DOI = {10.1109/ACCESS.2023.3301467},
abstractNote = {Using fingerphoto images acquired from mobile cameras, low-quality sensors, or crime scenes, it has become a challenge for automated identification systems to verify the identity due to various acquisition distortions. A significant type of photometric distortion that notably reduces the quality of a fingerphoto is the blurring of the image. This paper proposes a deep fingerphoto deblurring model to restore the ridge information degraded by the image blurring. As the core of our model, we utilize a conditional Generative Adversarial Network (cGAN) to learn the distribution of natural ridge patterns. We perform several modifications to enhance the quality of the reconstructed (deblurred) fingerphotos by our proposed model. First, we develop a multi-stage GAN to learn the ridge distribution in a coarse-to-fine framework. This framework enables the model to maintain the consistency of the ridge deblurring process at different resolutions. Second, we propose a guided attention module that helps the generator to focus mainly on blurred regions. Third, we incorporate a deep fingerphoto verifier as an auxiliary adaptive loss function to force the generator to preserve the ID information during the deblurring process. Finally, we evaluate the effectiveness of the proposed model through extensive experiments on multiple public fingerphoto datasets as well as real-world blurred fingerphotos. In particular, our method achieves 5.2 dB, 8.7%, and 7.6% improvement in PSNR, AUC, and EER, respectively, compared to a state-of-the-art deblurring method.},
journal = {IEEE Access},
volume = {11},
publisher = {IEEE},
author = {Joshi, Amol S. and Dabouei, Ali and Dawson, Jeremy and Nasrabadi, Nasser M.},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
An official website of the United States government
