More Like this

1. Evaluating Unsupervised Denoising Requires Unsupervised Metrics

   Morales, Adria Marcos; Leibovich, Matan; Mohan, Sreyas; Vincent, Joshua Lawrence; Haluai, Piyush; Tan, Mai; Crozier, Peter; Fernandez-Granda, Carlos (June 2023, Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR)

   Unsupervised denoising is a crucial challenge in real-world imaging applications. Unsupervised deep-learning methods have demonstrated impressive performance on benchmarks based on synthetic noise. However, no metrics are available to evaluate these methods in an unsupervised fashion. This is highly problematic for the many practical applications where ground-truth clean images are not available. In this work, we propose two novel metrics: the unsupervised mean squared error (MSE) and the unsupervised peak signal-to-noise ratio (PSNR), which are computed using only noisy data. We provide a theoretical analysis of these metrics, showing that they are asymptotically consistent estimators of the supervised MSE and PSNR. Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data. 

   more » « less
2. Evaluating Unsupervised Denoising Requires Unsupervised Metrics

   Marcos-Morales, Adria; Leibovich, Matan; Mohan, Sreyas; Lawrence Vincent, Joshua; Haluai, Piyush; Tan, Mai; Crozier, Peter; Fernandez-Granda, Carlos (May 2023, International Conference On Machine Learning 2023)

   Unsupervised denoising is a crucial challenge in real-world imaging applications. Unsupervised deep-learning methods have demonstrated impressive performance on benchmarks based on synthetic noise. However, no metrics are available to evaluate these methods in an unsupervised fashion. This is highly problematic for the many practical applications where ground-truth clean images are not available. In this work, we propose two novel metrics: the unsupervised mean squared error (MSE) and the unsupervised peak signal-to-noise ratio (PSNR), which are computed using only noisy data. We provide a theoretical analysis of these metrics, showing that they are asymptotically consistent estimators of the supervised MSE and PSNR. Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data. 

   more » « less
3. Evaluating Unsupervised Denoising Requires Unsupervised Metrics

   Marcos-Morales, Adria; Leibovich, Matan; Mohan, Sreyas; Vincent, Joshua Lawrence; Haluai, Piyush; Tan, Mai; Crozier, Peter; Fernandez-Granda, Carlos (August 2023, Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR)

   Unsupervised denoising is a crucial challenge in real-world imaging applications. Unsupervised deep-learning methods have demonstrated impressive performance on benchmarks based on synthetic noise. However, no metrics exist to evaluate these methods in an unsupervised fashion. This is highly problematic for the many practical applications where ground-truth clean images are not available. In this work, we propose two novel metrics: the unsupervised mean squared error (MSE) and the unsupervised peak signalto-noise ratio (PSNR), which are computed using only noisy data. We provide a theoretical analysis of these metrics, showing that they are asymptotically consistent estimators of the supervised MSE and PSNR. Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data. 

   more » « less
4. Evaluating Unsupervised Denoising Requires Unsupervised Metrics

   Marcos-Morales, Adria; Leibovich, Matan; Mohan, Sreyas; Vincent, Joshua Lawrence; Haluai, Piyush; Tan, Mai; Crozier, Peter A; Fernandez-Granda, Carlos (August 2023, Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR 2023 Vol. 202 Pages 23937-23957)

   Unsupervised denoising is a crucial challenge in real-world imaging applications. Unsupervised deep-learning methods have demonstrated impressive performance on benchmarks based on synthetic noise. However, no metrics exist to evaluate these methods in an unsupervised fashion. This is highly problematic for the many practical applications where ground-truth clean images are not available. In this work, we propose two novel metrics: the unsupervised mean squared error (MSE) and the unsupervised peak signal-to-noise ratio (PSNR), which are computed using only noisy data. We provide a theoretical analysis of these metrics, showing that they are asymptotically consistent estimators of the supervised MSE and PSNR. Controlled numerical experiments with synthetic noise confirm that they provide accurate approximations in practice. We validate our approach on real-world data from two imaging modalities: videos in raw format and transmission electron microscopy. Our results demonstrate that the proposed metrics enable unsupervised evaluation of denoising methods based exclusively on noisy data. 

   more » « less
5. Evaluating Unsupervised Denoising Requires Unsupervised Metrics

   Marcos-Morales, Adria; Leibovich, Matan; Mohan, Sreyas; Vincent, Joshua Lawrence; Haluai, Piyush; Tan, Mai; Crozier, Peter; Fernandez-Granda, Carlos (August 2023, Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR)