More Like this

1. Cross-frequency training with adversarial learning for radar micro-Doppler signature classification (Rising Researcher)

   https://doi.org/10.1117/12.2559155  

   Gurbuz, Sevgi Zubeyde; Rahman, Mohammad M.; Kurtoglu, Emre; Macks, Trevor; Fioranelli, Francesco (May 2020, Proc. SPIE 11408, Radar Sensor Technology XXIV)

   Deep neural networks have become increasingly popular in radar micro-Doppler classification; yet, a key challenge, which has limited potential gains, is the lack of large amounts of measured data that can facilitate the design of deeper networks with greater robustness and performance. Several approaches have been proposed in the literature to address this problem, such as unsupervised pre-training and transfer learning from optical imagery or synthetic RF data. This work investigates an alternative approach to training which involves exploitation of “datasets of opportunity” – micro-Doppler datasets collected using other RF sensors, which may be of a different frequency, bandwidth or waveform - for the purposes of training. Specifically, this work compares in detail the cross-frequency training degradation incurred for several different training approaches and deep neural network (DNN) architectures. Results show a 70% drop in classification accuracy when the RF sensors for pre-training, fine-tuning, and testing are different, and a 15% degradation when only the pre-training data is different, but the fine-tuning and test data are from the same sensor. By using generative adversarial networks (GANs), a large amount of synthetic data is generated for pre-training. Results show that cross-frequency performance degradation is reduced by 50% when kinematically-sifted GAN-synthesized signatures are used in pre-training. 

   more » « less
2. Deep Learning on a Data Diet: Finding Important Examples Early in Training

   Paul, Mansheej; Ganguli, Surya; Dziugaite, Karolina G (December 2021, NeurIPS)

   Recent success in deep learning has partially been driven by training increasingly overparametrized networks on ever larger datasets. It is therefore natural to ask: how much of the data is superfluous, which examples are important for general- ization, and how do we find them? In this work, we make the striking observation that, in standard vision datasets, simple scores averaged over several weight ini- tializations can be used to identify important examples very early in training. We propose two such scores—the Gradient Normed (GraNd) and the Error L2-Norm (EL2N) scores—and demonstrate their efficacy on a range of architectures and datasets by pruning significant fractions of training data without sacrificing test accuracy. In fact, using EL2N scores calculated a few epochs into training, we can prune half of the CIFAR10 training set while slightly improving test accuracy. Furthermore, for a given dataset, EL2N scores from one architecture or hyper- parameter configuration generalize to other configurations. Compared to recent work that prunes data by discarding examples that are rarely forgotten over the course of training, our scores use only local information early in training. We also use our scores to detect noisy examples and study training dynamics through the lens of important examples—we investigate how the data distribution shapes the loss surface and identify subspaces of the model’s data representation that are relatively stable over training. 

   more » « less
3. MutualNet: Adaptive ConvNet via Mutual Learning from Network Width and Resolution

   Yang, Taojiannan; Zhu, Sijie; Chen, Chen; Yan, Shen; Zhang, Mi; Willis, Andrew (January 2020, European Conference on Computer Vision)

   We propose the width-resolution mutual learning method (MutualNet) to train a network that is executable at dynamic resource constraints to achieve adaptive accuracy-efficiency trade-offs at runtime. Our method trains a cohort of sub-networks with different widths (i.e., number of channels in a layer) using different input resolutions to mutually learn multi-scale representations for each sub-network. It achieves consistently better ImageNet top-1 accuracy over the state-of-the-art adaptive network US-Net under different computation constraints, and outperforms the best compound scaled MobileNet in EfficientNet by 1.5%. The superiority of our method is also validated on COCO object detection and instance segmentation as well as transfer learning. Surprisingly, the training strategy of MutualNet can also boost the performance of a single network, which substantially outperforms the powerful AutoAugmentation in both efficiency (GPU search hours: 15000 vs. 0) and accuracy (ImageNet: 77.6% vs. 78.6%). Code is available at https://github.com/ aoyang1122/MutualNet 

   more » « less
4. Effectively using public data in privacy preserving Machine learning

   Milad Nasr, Saeed Mahloujifar (July 2023, Proceedings of the 40th International Conference on Machine Learning)

   Andreas Krause, Emma Brunskill (Ed.)

   Differentially private (DP) machine learning techniques are notorious for their degradation of model utility (e.g., they degrade classification accuracy). A recent line of work has demonstrated that leveraging public data can improve the trade-off between privacy and utility when training models with DP guaranteed. In this work, we further explore the potential of using public data in DP models, showing that utility gains can in fact be significantly higher than what shown in prior works. Specifically, we introduce DOPE-SGD, a modified DP-SGD algorithm that leverages public data during its training. DOPE-SGD uses public data in two complementary ways: (1) it uses advance augmentation techniques that leverages public data to generate synthetic data that is effectively embedded in multiple steps of the training pipeline; (2) it uses a modified gradient clipping mechanism (which is a standard technique in DP training) to change the origin of gradient vectors using the information inferred from available public and synthetic data, therefore boosting utility. We also introduce a technique to ensemble intermediate DP models by leveraging the post processing property of differential privacy to further improve the accuracy of the predictions. Our experimental results demonstrate the effectiveness of our approach in improving the state-of-the-art in DP machine learning across multiple datasets, network architectures, and application domains. For instance, assuming access to 2,000 public images, and for a privacy budget of 𝜀=2,𝛿=10−5, our technique achieves an accuracy of 75.1 on CIFAR10, significantly higher than 68.1 achieved by the state of the art. 

   more » « less
5. Deep Learning of Forced Convection Heat Transfer

   https://doi.org/10.1115/1.4052893  

   Kang, Munku; Kwon, Beomjin (February 2022, Journal of Heat Transfer)

   Abstract We present the deep learning model for internal forced convection heat transfer problems. Conditional generative adversarial networks (cGAN) are trained to predict the solution based on a graphical input describing fluid channel geometries and initial flow conditions. Without interactively solving the physical governing equations, a trained cGAN model rapidly approximates the flow temperature, Nusselt number (Nu), and friction factor (f) of a flow in a heated channel over Reynolds number ranging from 100 to 27,750. For an effective training, we optimize the dataset size, training epoch, and a hyperparameter λ. The cGAN model exhibited an accuracy up to 97.6% when predicting the local distributions of Nu and f. We also show that the trained cGAN model can predict for unseen fluid channel geometries such as narrowed, widened, and rotated channels if the training dataset is properly augmented. A simple data augmentation technique improved the model accuracy up to 70%. This work demonstrates the potential of deep learning approach to enable cost-effective predictions for thermofluidic processes. 

   more » « less