More Like this

1. Density estimation using deep generative neural networks

   https://doi.org/10.1073/pnas.2101344118  

   Liu, Qiao; Xu, Jiaze; Jiang, Rui; Wong, Wing Hung (April 2021, Proceedings of the National Academy of Sciences)

   null (Ed.)

   Density estimation is one of the fundamental problems in both statistics and machine learning. In this study, we propose Roundtrip, a computational framework for general-purpose density estimation based on deep generative neural networks. Roundtrip retains the generative power of deep generative models, such as generative adversarial networks (GANs) while it also provides estimates of density values, thus supporting both data generation and density estimation. Unlike previous neural density estimators that put stringent conditions on the transformation from the latent space to the data space, Roundtrip enables the use of much more general mappings where target density is modeled by learning a manifold induced from a base density (e.g., Gaussian distribution). Roundtrip provides a statistical framework for GAN models where an explicit evaluation of density values is feasible. In numerical experiments, Roundtrip exceeds state-of-the-art performance in a diverse range of density estimation tasks. 

   more » « less
2. Learning Neural Contextual Bandits through Perturbed Rewards

   Jia, Yiling; Zhang, Weitong; Zhou, Dongruo; Gu, Quanquan; Wang, Hongning (April 2022, The Tenth International Conference on Learning Representations (ICLR'2022))

   Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts. But because their exploration has to be performed in the entire neural network parameter space to obtain nearly optimal regret, the resulting computational cost is prohibitively high. We perturb the rewards when updating the neural network to eliminate the need of explicit exploration and the corresponding computational overhead. We prove that a O(d\sqrt{T}) regret upper bound is still achievable under standard regularity conditions, where $$T$$ is the number of rounds of interactions and $$\tilde{d}$$ is the effective dimension of a neural tangent kernel matrix. Extensive comparisons with several benchmark contextual bandit algorithms, including two recent neural contextual bandit models, demonstrate the effectiveness and computational efficiency of our proposed neural bandit algorithm. 

   more » « less
3. A dimension-reduced variational approach for solving physics-based inverse problems using generative adversarial network priors and normalizing flows

   https://doi.org/10.1016/j.cma.2023.116682  

   Dasgupta, Agnimitra; Patel, Dhruv V; Ray, Deep; Johnson, Erik A; Oberai, Assad A (February 2024, Computer Methods in Applied Mechanics and Engineering)

   We propose a novel modular inference approach combining two different generative models — generative adversarial networks (GAN) and normalizing flows — to approximate the posterior distribution of physics-based Bayesian inverse problems framed in high-dimensional ambient spaces. We dub the proposed framework GAN-Flow. The proposed method leverages the intrinsic dimension reduction and superior sample generation capabilities of GANs to define a low-dimensional data-driven prior distribution. Once a trained GAN-prior is available, the inverse problem is solved entirely in the latent space of the GAN using variational Bayesian inference with normalizing flow-based variational distribution, which approximates low-dimensional posterior distribution by transforming realizations from the low-dimensional latent prior (Gaussian) to corresponding realizations of a low-dimensional variational posterior distribution. The trained GAN generator then maps realizations from this approximate posterior distribution in the latent space back to the high-dimensional ambient space. We also propose a two-stage training strategy for GAN-Flow wherein we train the two generative models sequentially. Thereafter, GAN-Flow can estimate the statistics of posterior-predictive quantities of interest at virtually no additional computational cost. The synergy between the two types of generative models allows us to overcome many challenges associated with the application of Bayesian inference to large-scale inverse problems, chief among which are describing an informative prior and sampling from the high-dimensional posterior. GAN-Flow does not involve Markov chain Monte Carlo simulation, making it particularly suitable for solving large-scale inverse problems. We demonstrate the efficacy and flexibility of GAN-Flow on various physics-based inverse problems of varying ambient dimensionality and prior knowledge using different types of GANs and normalizing flows. Notably, one of the applications we consider involves a 65,536-dimensional inverse problem of phase retrieval wherein an object is reconstructed from sparse noisy measurements of the magnitude of its Fourier transform. 

   more » « less
4. A Roadmap to Holographic Focused Ultrasound Approaches for Generating Gradient Thermal Patterns

   https://doi.org/10.1002/cnm.70055  

   Cengiz, Ceren; Eger, Zekeriya_Ender; Pewekar, Mihir; Acar, Pinar; Legon, Wynn; Shahab, Shima (June 2025, International Journal for Numerical Methods in Biomedical Engineering)

   ABSTRACT In therapeutic focused ultrasound (FUS), such as thermal ablation and hyperthermia, effective acousto‐thermal manipulation requires precise targeting of complex geometries, sound wave propagation through irregular structures, and selective focusing at specific depths. Acoustic holographic lenses (AHLs) provide a distinctive capability to shape acoustic fields into precise, complex, and multifocal FUS‐thermal patterns. Acknowledging the under‐explored potential of AHLs in shaping ultrasound‐induced heating patterns, this study introduces a roadmap for acousto‐thermal modeling in the design of AHLs. Three primary modeling approaches are studied and contrasted using four distinct shape groups for the imposed target field. They include pressure‐based time reversal (TR) (basic (BSC‐TR) and iterative (ITER‐TR)), temperature‐based (inverse heat transfer optimization (IHTO‐TR)), and machine learning (ML)‐based (generative adversarial network (GaN) and GaN with feature (Feat‐GAN)) methods. Novel metrics, including image quality, thermal efficiency, thermal control, and computational time, are introduced, providing each method's strengths and weaknesses. The importance of evaluating target pattern complexity, thermal and pressure requirements, and computational resources is highlighted. As a further step, two case studies: (1) transcranial FUS and (2) liver hyperthermia, demonstrate the practical use of acoustic holography in therapeutic settings. This paper offers a practical reference for selecting modeling approaches based on therapeutic goals and modeling requirements. Alongside established methods like BSC‐TR and ITER‐TR, new techniques IHTO‐TR, GaN, and Feat‐GaN are introduced. BSC‐TR serves as a baseline, while ITER‐TR enables refinement based on target shape characteristics. IHTO‐TR supports thermal control, GaN offers rapid solutions under fixed conditions, and Feat‐GaN provides adaptability across varying application settings. 

   more » « less
5. PIM-TGAN: A Processing-in-Memory Accelerator for Ternary Generative Adversarial Networks

   https://doi.org/10.1109/ICCD.2018.00048  

   Rakin, Adnan Siraj; Angizi, Shaahin; He, Zhezhi; Fan, Deliang (October 2018, 2018 IEEE 36th International Conference on Computer Design (ICCD))

   Generative Adversarial Network (GAN) has emerged as one of the most promising semi-supervised learning methods where two neural nets train themselves in a competitive environment. In this paper, as far as we know, we are the first to present a statistically trained Ternarized Generative Adversarial Network (TGAN) with fully ternarized weights (i.e. -1,0,+1) to massively reduce the need for computation and storage resources in the conventional GAN structures. In the proposed TGAN, the computationally expensive convolution operations (i.e. Multiplication and Accumulation) in both generator and discriminator's forward path are converted into hardware-friendly Addition/Subtraction operations. Accordingly, we propose a Processing-in-Memory accelerator for TGAN called (PIM-TGAN) based on Spin-Orbit Torque Magnetic Random Access Memory (SOT-MRAM) computational sub-arrays to efficiently accelerate the training process of GAN within non-volatile memory. In addition, we propose a parallelism technique to further enhance the training efficiency of TGAN. Our device-to-architecture co-simulation results show that, with almost the same inception score to the baseline GAN with floating point number weights on different data-sets, the proposed PIM-TGAN can obtain ~25.6× better energy-efficiency and 22× speedup compared to GPU platform averagely, and, 9.2× better energy-efficiency and 5.4× speedup over the best processing-in-ReRAM accelerators. 

   more » « less