More Like this

1. Knowledge Distillation-Based TinyML Model for Breast Cancer Detection Using Real and Wasserstein GAN-Generated Microwave Imaging Data

   https://doi.org/10.1109/EMBC58623.2025.11253875  

   Khalid, Nazish; Dagli, Cihan; Obafemi-Ajayi, Tayo; Wunsch, Donald (July 2025, IEEE)

   Tiny Machine Learning (TinyML) offers a transformative approach to efficient, intelligent cancer diagnostics on edge devices. However, designing optimized TinyML models for breast cancer diagnosis using Microwave Imaging datasets remains a challenge due to domain-specific customization requirements. To address this, we propose a novel Knowledge Distillation framework, where a teacher model, built using residual convolutional neural networks, transfers its knowledge to a lightweight student model, enhancing efficiency while maintaining high accuracy. The teacher model achieves a 95.42% accuracy on test data. Through knowledge distillation, the student model attains 95.32% accuracy while achieving a 96% reduction in model size, significantly enhancing computational efficiency without compromising performance. Without distillation, the student model reaches a testing accuracy of 86.5%. A major contribution of this work is the generation of a highquality synthetic breast cancer dataset to address the scarcity of microwave imaging data. We employ a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) to synthesize realistic breast cancer scans, ensuring reliability through rigorous validation with a one-class Support Vector Machine. By training all models on a combination of real and synthetic datasets, our approach enhances robustness, making TinyML-powered breast cancer detection more viable for realworld deployment. 

   more » « less
2. Learning Terrain-Aware Bipedal Locomotion via Reduced-Dimensional Perceptual Representations

   https://doi.org/10.1109/TCST.2026.3664022  

   Castillo, Guillermo A; Lodha, Himanshu; Hereid, Ayonga (January 2026, IEEE Transactions on Control Systems Technology)

   This work introduces a hierarchical strategy for terrain-aware bipedal locomotion that integrates reduced-dimensional perceptual representations to enhance the reinforcement learning (RL)-based high-level (HL) policies for real-time gait generation. Unlike end-to-end approaches, our framework leverages latent terrain encodings via a convolutional variational autoencoder (CNN-VAE) alongside reduced-order robot dynamics, optimizing the locomotion decision process with a compact state. We systematically analyze the impact of latent space dimensionality on learning efficiency and policy robustness. In addition, we extend our method to be history-aware, incorporating sequences of recent terrain observations into the latent representation to improve robustness. To address real-world feasibility, we introduce a distillation method to learn the latent representation directly from depth camera images and provide preliminary hardware validation by comparing simulated and real sensor data. We further validate our framework using the high-fidelity agility robotics (ARs) simulator, incorporating realistic sensor noise, state estimation, and actuator dynamics. The results confirm the robustness and adaptability of our method, underscoring its potential for hardware deployment. 

   more » « less
3. ORVIT: Near-Optimal Online Distributionally Robust Reinforcement Learning

   https://doi.org/10.1609/aaai.v40i25.39273  

   Ghosh, Debamita; Atia, George K; Wang, Yue (March 2026, Proceedings of the AAAI Conference on Artificial Intelligence)

   Reinforcement learning (RL) faces significant challenges in real-world deployments due to the sim-to-real gap, where policies trained in simulators often underperform in practice due to mismatches between training and deployment conditions. Distributionally robust RL addresses this issue by optimizing worst-case performance over an uncertainty set of environments and providing an optimized lower bound on deployment performance. However, existing studies typically assume access to either a generative model or offline datasets with broad coverage of the deployment environment—assumptions that limit their practicality in unknown environments without prior knowledge. In this work, we study the more realistic and challenging setting of online distributionally robust RL, where the agent interacts only with a single unknown training environment while aiming to optimize its worst-case performance. We focus on general f-divergence-based uncertainty sets, including chi-squared and KL divergence balls, and propose a computationally efficient algorithm with sublinear regret guarantees under minimal assumptions. Furthermore, we establish a minimax lower bound on regret of online learning, demonstrating the near-optimality of our approach. Extensive experiments across diverse environments further confirm the robustness and efficiency of our algorithm, validating our theoretical findings. 

   more » « less
4. Slimmed Asymmetrical Contrastive Learning and Cross Distillation for Lightweight Model Training

   Meng, Jian; Yang, Li; Lee, Kyungmin; Shin, Jinwoo; Fan, Deliang; Seo, Jae-sun (December 2023, Thirty-seventh Conference on Neural Information Processing Systems)

   Contrastive learning (CL) has been widely investigated with various learning mech- anisms and achieves strong capability in learning representations of data in a self-supervised manner using unlabeled data. A common fashion of contrastive learning on this line is employing large-sized encoders to achieve comparable performance as the supervised learning counterpart. Despite the success of the labelless training, current contrastive learning algorithms failed to achieve good performance with lightweight (compact) models, e.g., MobileNet, while the re- quirements of the heavy encoders impede the energy-efficient computation, espe- cially for resource-constrained AI applications. Motivated by this, we propose a new self-supervised CL scheme, named SACL-XD, consisting of two technical components, Slimmed Asymmetrical Contrastive Learning (SACL) and Cross- Distillation (XD), which collectively enable efficient CL with compact models. While relevant prior works employed a strong pre-trained model as the teacher of unsupervised knowledge distillation to a lightweight encoder, our proposed method trains CL models from scratch and outperforms them even without such an expensive requirement. Compared to the SoTA lightweight CL training (dis- tillation) algorithms, SACL-XD achieves 1.79% ImageNet-1K accuracy improve- ment on MobileNet-V3 with 64⇥ training FLOPs reduction. Code is available at https://github.com/mengjian0502/SACL-XD. 

   more » « less
5. A Closer Look at Rehearsal-Free Continual Learning

   https://doi.org/10.1109/CVPRW59228.2023.00239  

   Smith, James Seale; Tian, Junjiao; Halbe, Shaunak; Hsu, Yen-Chang; Kira, Zsolt (June 2023, IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW))

   Continual learning is a setting where machine learning models learn novel concepts from continuously shifting training data, while simultaneously avoiding degradation of knowledge on previously seen classes which may disappear from the training data for extended periods of time (a phenomenon known as the catastrophic forgetting problem). Current approaches for continual learning of a single expanding task (aka class-incremental continual learning) require extensive rehearsal of previously seen data to avoid this degradation of knowledge. Unfortunately, rehearsal comes at a cost to memory, and it may also violate data-privacy. Instead, we explore combining knowledge distillation and parameter regularization in new ways to achieve strong continual learning performance without rehearsal. Specifically, we take a deep dive into common continual learning techniques: prediction distillation, feature distillation, L2 parameter regularization, and EWC parameter regularization. We first disprove the common assumption that parameter regularization techniques fail for rehearsal-free continual learning of a single, expanding task. Next, we explore how to leverage knowledge from a pre-trained model in rehearsal-free continual learning and find that vanilla L2 parameter regularization outperforms EWC parameter regularization and feature distillation. Finally, we explore the recently popular ImageNet-R benchmark, and show that L2 parameter regularization implemented in self-attention blocks of a ViT transformer outperforms recent popular prompting for continual learning methods. 

   more » « less