More Like this

1. Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation

   Zhou, Mingyuan; Zheng, Huangjie; Wang, Zhendong; Yin, Mingzhang; Huang, Hai (July 2023, International Conference on Machine Learning)

   We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator. SiD not only facilitates an exponentially fast reduction in Frechet inception distance (FID) during distillation but also approaches or even exceeds the FID performance of the original teacher diffusion models. By reformulating forward diffusion processes as semi-implicit distributions, we leverage three score-related identities to create an innovative loss mechanism. This mechanism achieves rapid FID reduction by training the generator using its own synthesized images, eliminating the need for real data or reverse-diffusion-based generation, all accomplished within significantly shortened generation time. Upon evaluation across four benchmark datasets, the SiD algorithm demonstrates high iteration efficiency during distillation and surpasses competing distillation approaches, whether they are one-step or few-step, data-free, or dependent on training data, in terms of generation quality. This achievement not only redefines the benchmarks for efficiency and effectiveness in diffusion distillation but also in the broader field of diffusion-based generation. The PyTorch implementation is available at https://github.com/mingyuanzhou/SiD. 

   more » « less
2. Evaluating generative networks using Gaussian mixtures of image features

   https://doi.org/10.1109/WACV56688.2023.00036  

   Luzi, Lorenzo; Marrero, Carlos Ortiz; Wynar, Nile; Baraniuk, Richard G.; Henry, Michael J. (February 2023, 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV))

   We develop a measure for evaluating the performance of generative networks given two sets of images. A popular performance measure currently used to do this is the Fréchet Inception Distance (FID). FID assumes that images featurized using the penultimate layer of Inception-v3 follow a Gaussian distribution, an assumption which cannot be violated if we wish to use FID as a metric. However, we show that Inception-v3 features of the ImageNet dataset are not Gaussian; in particular, every single marginal is not Gaussian. To remedy this problem, we model the featurized images using Gaussian mixture models (GMMs) and compute the 2-Wasserstein distance restricted to GMMs. We define a performance measure, which we call WaM, on two sets of images by using Inception-v3 (or another classifier) to featurize the images, estimate two GMMs, and use the restricted 2-Wasserstein distance to compare the GMMs. We experimentally show the advantages of WaM over FID, including how FID is more sensitive than WaM to imperceptible image perturbations. By modelling the non-Gaussian features obtained from Inception-v3 as GMMs and using a GMM metric, we can more accurately evaluate generative network performance. 

   more » « less
3. Advancements in Synthetic Generation of Contactless Palmprint Biometrics Using StyleGAN Models

   https://doi.org/10.3390/jcp4030032  

   Chowdhury, A_M Mahmud; Khondkar, Md_Jahangir Alam; Imtiaz, Masudul Haider (September 2024, Journal of Cybersecurity and Privacy)

   Deep learning models have demonstrated significant advantages over traditional algorithms in image processing tasks like object detection. However, a large amount of data are needed to train such deep networks, which limits their application to tasks such as biometric recognition that require more training samples for each class (i.e., each individual). Researchers developing such complex systems rely on real biometric data, which raises privacy concerns and is restricted by the availability of extensive, varied datasets. This paper proposes a generative adversarial network (GAN)-based solution to produce training data (palm images) for improved biometric (palmprint-based) recognition systems. We investigate the performance of the most recent StyleGAN models in generating a thorough contactless palm image dataset for application in biometric research. Training on publicly available H-PolyU and IIDT palmprint databases, a total of 4839 images were generated using StyleGAN models. SIFT (Scale-Invariant Feature Transform) was used to find uniqueness and features at different sizes and angles, which showed a similarity score of 16.12% with the most recent StyleGAN3-based model. For the regions of interest (ROIs) in both the palm and finger, the average similarity scores were 17.85%. We present the Frechet Inception Distance (FID) of the proposed model, which achieved a 16.1 score, demonstrating significant performance. These results demonstrated StyleGAN as effective in producing unique synthetic biometric images. 

   more » « less
4. Advancing ai-powered medical image synthesis: Insights from medvqa-gi challenge using clip, fine-tuned stable diffusion, and dream-booth + lora

   Ojonugwa, E; Rahman, M; Khalifa, F (September 2024, CEUR workshop proceedings)

   Faggioli, G; Ferro, N; Galuščáková, P; Herrera, A (Ed.)

   The MEDVQA-GI challenge addresses the integration of AI-driven text-to-image generative models in medical diagnostics, aiming to enhance diagnostic capabilities through synthetic image generation. Existing methods primarily focus on static image analysis and lack the dynamic generation of medical imagery from textual descriptions. This study intends to partially close this gap by introducing a novel approach based on fine-tuned generative models to generate dynamic, scalable, and precise images from textual descriptions. Particularly, our system integrates fine-tuned Stable Diffusion and DreamBooth models, as well as Low-Rank Adaptation (LORA), to generate high-fidelity medical images. The problem is around two sub-tasks namely: image synthesis (IS) and optimal prompt production (OPG). The former creates medical images via verbal prompts, whereas the latter provides prompts that produce high-quality images in specified categories. The study emphasizes the limitations of traditional medical image generation methods, such as hand sketching, constrained datasets, static procedures, and generic models. Our evaluation measures showed that Stable Diffusion surpasses CLIP and DreamBooth + LORA in terms of producing high-quality, diversified images. Specifically, Stable Diffusion had the lowest Fréchet Inception Distance (FID) scores (0.099 for single center, 0.064 for multi-center, and 0.067 for combined), indicating higher image quality. Furthermore, it had the highest average Inception Score (2.327 across all datasets), indicating exceptional diversity and quality. This advances the field of AI-powered medical diagnosis. Future research will concentrate on model refining, dataset augmentation, and ethical considerations for efficiently implementing these advances into clinical practice. 

   more » « less
5. An Empirical Analysis of Diffusion, Autoencoders, and Adversarial Deep Learning Models for Predicting Dementia Using High-Fidelity MRI

   https://doi.org/10.1109/ACCESS.2024.3354724  

   Gajjar, Pranshav; Garg, Manav; Desai, Shivani; Chhinkaniwala, Hitesh; Sanghvi, Harshal A; Patel, Riki H; Gupta, Shailesh; Pandya, Abhijit S (January 2024, IEEE Access)

   Saif, Mehrdad (Ed.)

   This study explores cutting-edge computational technologies and intelligent methods to create realistic synthetic data, focusing on dementia-centric Magnetic Resonance Imaging (MRI) scans related to Alzheimer’s and Parkinson’s diseases. The research delves into Generative Adversarial Networks (GANs), Variational Autoencoders, and Diffusion Models, comparing their efficacy in generating synthetic MRI scans. Using datasets from Alzheimer’s and Parkinson’s patients, the study reveals intriguing findings. In the Alzheimer dataset, diffusion models produced non-dementia images with the lowest Frechet Inception Distance (FID) score at 92.46, while data-efficient GANs excelled in generating dementia images with an FID score of 178.53. In the Parkinson dataset, data-efficient GANs achieved remarkable FID scores of 102.71 for dementia images and 129.77 for non-dementia images. The study also introduces a novel aspect by incorporating a classification study, validating the generative metrics. DenseNets, a deep learning architecture, exhibited superior performance in disease detection compared to ResNets. Training both models on images generated by diffusion models further improved results, with DenseNet achieving accuracies of 80.84% and 92.42% in Alzheimer’s and Parkinson’s disease detection, respectively. The research not only presents innovative generative architectures but also emphasizes the importance of classification metrics, providing valuable insights into the synthesis and detection of neurodegenerative diseases through advanced computational techniques. 

   more » « less