Autonomous vehicles are cyber-physical systems that combine embedded computing and deep learning with physical systems to perceive the world, predict future states, and safely control the vehicle through changing environments. The ability of an autonomous vehicle to accurately predict the motion of other road users across a wide range of diverse scenarios is critical for both motion planning and safety. However, existing motion prediction methods do not explicitly model contextual information about the environment, which can cause significant variations in performance across diverse driving scenarios. To address this limitation, we proposeCASTNet: a dynamic, context-aware approach for motion prediction that (i) identifies the current driving context using a spatio-temporal model, (ii) adapts an ensemble of motion prediction models to fit the current context, and (iii) applies novel trajectory fusion methods to combine predictions output by the ensemble. This approach enables CASTNet to improve robustness by minimizing motion prediction error across diverse driving scenarios. CASTNet is highly modular and can be used with various existing image processing backbones and motion predictors. We demonstrate how CASTNet can improve both CNN-based and graph-learning-based motion prediction approaches and conduct ablation studies on the performance, latency, and model size for various ensemble architecture choices. In addition, we propose and evaluate several attention-based spatio-temporal models for context identification and ensemble selection. We also propose a modular trajectory fusion algorithm that effectively filters, clusters, and fuses the predicted trajectories output by the ensemble. On the nuScenes dataset, our approach demonstrates more robust and consistent performance across diverse, real-world driving contexts than state-of-the-art techniques.
more »
« less
T-Foresight: Interpret moving strategies based on context-aware trajectory prediction
Trajectory prediction and interpretation are crucial in various domains for optimizing movements in complex environments. However, understanding how diverse contextual factors—environmental, physical, and social—influence moving strategies is challenging due to their multifaceted nature, which complicates quantification and the derivation of actionable insights. We introduce an interpretable analytics workflow that addresses these challenges by innovatively leveraging ensemble learning for context-aware trajectory prediction. Multiple base predictors simulate diverse moving strategies, while a decision-making model assesses the suitability of each predictor in specific contexts. This approach quantifies the impact of contextual factors by interpreting the decision-making model’s predictions and reveals possible moving strategies through the aggregation of base predictors’ outputs. The workflow comes with T-Foresight, an interactive visualization interface that empowers stakeholders to explore predictions, interpret contextual influences, and devise and compare moving strategies effectively. We evaluate our approach in the domain of eSports, specifically MOBA games. Through case studies with professional analysts, we demonstrate T-Foresight’s effectiveness in illustrating player moving strategies and providing insights into top-tier tactics. A user study further confirms its usefulness in helping average players uncover and understand advanced strategies.
more »
« less
- Award ID(s):
- 2427770
- PAR ID:
- 10698148
- Publisher / Repository:
- Elsevier
- Date Published:
- Journal Name:
- Visual Informatics
- Volume:
- 9
- Issue:
- 3
- ISSN:
- 2468-502X
- Page Range / eLocation ID:
- 100261
- Subject(s) / Keyword(s):
- Moving strategy, Interpretability, Visual analytics, Context-aware trajectory prediction, eSports data
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We introduce the Smoothed Online Optimization for Target Tracking (SOOTT) problem, a new framework that integrates three key objectives in online decision-making under uncertainty: (1) tracking cost for following a dynamically moving target, (2) adversarial perturbation cost for withstanding unpredictable disturbances, and (3) switching cost for penalizing abrupt changes in decisions. This formulation captures real-world scenarios, such as elastic and inelastic workload scheduling in AI clusters, where operators must balance long-term service-level agreements for elastic workloads, like LLM training, against sudden demand spikes for inelastic workloads, like real-time inference. We first present BEST, a robust algorithm with provable competitive guarantees for SOOTT. To enhance practical performance, we introduce CoRT, a learning-augmented variant that incorporates untrusted black-box predictions (e.g., from ML models) into its decision process. Our theoretical analysis shows that CoRT strictly improves over BEST when predictions are accurate, while maintaining robustness under arbitrary prediction errors. We validate our approach through a case study on workload scheduling, demonstrating that both algorithms effectively balance trajectory tracking, decision smoothness, and resilience to external disturbances.more » « less
-
Longitudinal experiential data offers rich insights into dynamic human states, yet building models that generalize across diverse contexts remains challenging. This paper addresses how to best represent multi-modal longitudinal experiential data as text and formulate prediction tasks to maximize large language model (LLM) cross-distribution generalization. We propose ConText-LE, a framework grounded in linguistic and cognitive theories of contextual meaning-making, which systematically investigates text representation strategies and output formulations for robust behavioral forecasting. Our novel Meta-Narrative representation synthesizes complex temporal patterns into semantically rich narratives, while Prospective Narrative Generation reframes prediction as a generative task aligned with LLMs' inherent contextual understanding capabilities. Through comprehensive experiments on three diverse longitudinal datasets, we address the critical but underexplored challenge of cross-distribution generalization in mental health and educational behavior forecasting. We demonstrate that combining Meta-Narrative input with Prospective Narrative Generation significantly outperforms existing LLM-based approaches, achieving up to 12.28% improvement in out-of-distribution accuracy and up to 11.99% improvement in F1 scores over binary classification methods. Bidirectional evaluation and architectural ablation studies confirm the robustness of our approach, establishing ConText-LE as an effective framework for developing reliable behavioral forecasting systems across temporal and contextual shifts.more » « less
-
Machine-learning (ML) approaches have proven to be of great utility in modern materials innovation pipelines. Generally, ML models are trained on predetermined past data and then used to make predictions for new test cases. Active-learning, however, is a paradigm in which ML models can direct the learning process itself through providing dynamic suggestions/queries for the “next-best experiment.” In this work, the authors demonstrate how an active-learning framework can aid in the discovery of polymers possessing high glass transition temperatures ( T g ). Starting from an initial small dataset of polymer T g measurements, the authors use Gaussian process regression in conjunction with an active-learning framework to iteratively add T g measurements of candidate polymers to the training dataset. The active-learning framework employs one of three decision making strategies (exploitation, exploration, or balanced exploitation/exploration) for selection of the “next-best experiment.” The active-learning workflow terminates once 10 polymers possessing a T g greater than a certain threshold temperature are selected. The authors statistically benchmark the performance of the aforementioned three strategies (against a random selection approach) with respect to the discovery of high- T g polymers for this particular demonstrative materials design challenge.more » « less
-
As robotics integrates into high-risk construction environments, cognitive load and emotional dynamics emerge as critical safety factors influencing Human–Robot Collaboration (HRC). While technical coordination is well-studied, comprehensive reviews of these psychological safety factors remain limited. This study addresses this gap through a mixed-method systematic review that combines scientometric mapping with an in-depth qualitative synthesis of 55 peer-reviewed journal articles. Five dominant research themes were identified: cognitive load and human factors, intelligent systems and cognitive architectures, social-emotional interaction, AI-driven decision-making, and emerging enabling technologies. More importantly, the findings demonstrate that cognitive and emotional processes are not independent but interact dynamically to shape safety outcomes. Building on these insights, the study develops an integrated cognitive–emotional safety framework that models how contextual factors, cognitive responses, and emotional states jointly influence worker safety during collaboration. The review further reveals a geographically concentrated evidence base, highlighting the need for broader, more inclusive research to support globally applicable safety strategies. The study advances safety science by repositioning cognitive and emotional dynamics as central, manageable risk factors in HRC. It calls for future research to develop task-specific cognitive load models, incorporate emotional safety into system design, and implement adaptive, human-aware robotic systems capable of proactively mitigating safety risks. This work provides a foundational framework for advancing safety science in technology-enhanced work environments.more » « less
An official website of the United States government

