More Like this

1. Online Dynamics Learning for Predictive Control with an Application to Aerial Robots

   Jiahao, Tom Z.; Chee, Kong Yao; Hsieh, M. Ani (May 2023, Proceedings of The 6th Conference on Robot Learning)

   Liu, Karen; Kulic, Dana; Ichnowski, Jeff (Ed.)

   In this work, we consider the task of improving the accuracy of dynamic models for model predictive control (MPC) in an online setting. Although prediction models can be learned and applied to model-based controllers, these models are often learned offline. In this offline setting, training data is first collected and a prediction model is learned through an elaborated training procedure. However, since the model is learned offline, it does not adapt to disturbances or model errors observed during deployment. To improve the adaptiveness of the model and the controller, we propose an online dynamics learning framework that continually improves the accuracy of the dynamic model during deployment. We adopt knowledge-based neural ordinary differential equations (KNODE) as the dynamic models, and use techniques inspired by transfer learning to continually improve the model accuracy. We demonstrate the efficacy of our framework with a quadrotor, and verify the framework in both simulations and physical experiments. Results show that our approach can account for disturbances that are possibly time-varying, while maintaining good trajectory tracking performance. 

   more » « less
2. Learning Off-policy for Online Planning

   Sikchi, Harshit; Zhou, Wenxuan; Held, David (January 2021, Conference on Robot Learning (CoRL))

   Reinforcement learning (RL) in low-data and risk-sensitive domains requires performant and flexible deployment policies that can readily incorporate constraints during deployment. One such class of policies are the semi-parametric H-step lookahead policies, which select actions using trajectory optimization over a dynamics model for a fixed horizon with a terminal value function. In this work, we investigate a novel instantiation of H-step lookahead with a learned model and a terminal value function learned by a model-free off-policy algorithm, named Learning Off-Policy with Online Planning (LOOP). We provide a theoretical analysis of this method, suggesting a tradeoff between model errors and value function errors and empirically demonstrate this tradeoff to be beneficial in deep reinforcement learning. Furthermore, we identify the "Actor Divergence" issue in this framework and propose Actor Regularized Control (ARC), a modified trajectory optimization procedure. We evaluate our method on a set of robotic tasks for Offline and Online RL and demonstrate improved performance. We also show the flexibility of LOOP to incorporate safety constraints during deployment with a set of navigation environments. We demonstrate that LOOP is a desirable framework for robotics applications based on its strong performance in various important RL settings. 

   more » « less
3. Stochastic Communication and Motion Planning via Learned Abstract World Representations

   Dadvar, Mehdi; Majd, Keyvan; Fainekos, Georgios; Srivastava, Siddharth (September 2025, CoRL)

   The increasing deployment of robots alongside humans necessitates sophisticated communication and motion planning to ensure safety and task achievability in social navigation scenarios. Existing methods often rely heavily on historical data and extensive expert hand-coding, which limits their scalability and generalizability. This paper introduces a novel framework that models social navigation as a Markov Decision Process (MDP), utilizing Conditional Abstraction Trees (CATs) to learn dynamic abstract world representations and policies that focus on critical aspects of interaction. In the offline phase, the framework operates within a simulator, while in the online phase, it deploys the learned representations and policies in real-world scenarios for ongoing refinement and adaptation. Integral to our approach is a Dynamic Bayesian Network (DBN) based human sensor and belief model that accounts for humans’ imperfect perception to enhance the prediction of human motion. We evaluated our method through extensive simulations and user studies involving physical experiments, demonstrating its effectiveness in managing critical interactions and ensuring safety and task completion across various scenarios. 

   more » « less
4. ClickTrain: Efficient and Accurate End-to-End Deep Learning Training via Fine-Grained Architecture-Preserving Pruning

   https://doi.org/10.1145/3447818.3459988  

   Zhang, Chengming; Yuan, Geng; Niu, Wei; Tian, Jiannan; Jin, Sian; Zhuang, Donglin; Jiang, Zhe; Wang, Yanzhi; Ren, Bin; Song, Shuaiwen Leon; et al (June 2021, The 35th ACM International Conference on Supercomputing (ICS 2021))

   Convolutional neural networks (CNNs) are becoming increasingly deeper, wider, and non-linear because of the growing demand on prediction accuracy and analysis quality. The wide and deep CNNs, however, require a large amount of computing resources and processing time. Many previous works have studied model pruning to improve inference performance, but little work has been done for effectively reducing training cost. In this paper, we propose ClickTrain: an efficient and accurate end-to-end training and pruning framework for CNNs. Different from the existing pruning-during-training work, ClickTrain provides higher model accuracy and compression ratio via fine-grained architecture-preserving pruning. By leveraging pattern-based pruning with our proposed novel accurate weight importance estimation, dynamic pattern generation and selection, and compiler-assisted computation optimizations, ClickTrain generates highly accurate and fast pruned CNN models for direct deployment without any extra time overhead, compared with the baseline training. ClickTrain also reduces the end-to-end time cost of the pruning-after-training method by up to 2.3X with comparable accuracy and compression ratio. Moreover, compared with the state-of-the-art pruning-during-training approach, ClickTrain provides significant improvements both accuracy and compression ratio on the tested CNN models and datasets, under similar limited training time. 

   more » « less
5. Online Joint Multi-Metric Adaptation from Frequent Sharing-Subset Mining for Person Re-Identification

   https://doi.org/10.1109/CVPR42600.2020.00298  

   Zhou, Jiahuan; Su, Bing; Wu, Ying (June 2020, IEEE/CVF Conf. on Computer Vision and Pattern Recognition)

   Person Re-IDentification (P-RID), as an instance-level recognition problem, still remains challenging in computer vision community. Many P-RID works aim to learn faithful and discriminative features/metrics from offline training data and directly use them for the unseen online testing data. However, their performance is largely limited due to the severe data shifting issue between training and testing data. Therefore, we propose an online joint multi-metric adaptation model to adapt the offline learned P-RID models for the online data by learning a series of metrics for all the sharing-subsets. Each sharing-subset is obtained from the proposed novel frequent sharing-subset mining module and contains a group of testing samples which share strong visual similarity relationships to each other. Unlike existing online P-RID methods, our model simultaneously takes both the sample-specific discriminant and the set-based visual similarity among testing samples into consideration so that the adapted multiple metrics can refine the discriminant of all the given testing samples jointly via a multi-kernel late fusion framework. Our proposed model is generally suitable to any offline learned P-RID baselines for online boosting, the performance improvement by our model is not only verified by extensive experiments on several widely-used P-RID benchmarks (CUHK03, Market1501, DukeMTMC-reID and MSMT17) and state-of-the-art P-RID baselines but also guaranteed by the provided in-depth theoretical analyses. 

   more » « less