Attention:The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, November 12th until 12:00 AM ET on Friday, November 13th due to maintenance. We apologize for the inconvenience.


Search for: All records

Award ID contains: 2245607

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. Free, publicly-accessible full text available May 24, 2027
  2. Free, publicly-accessible full text available May 24, 2027
  3. Free, publicly-accessible full text available May 24, 2027
  4. Free, publicly-accessible full text available February 16, 2027
  5. Reliable gesture interfaces are essential for coordinating distributed robot teams in the field. However, models trained in a single domain often perform poorly when confronted with new users, different sensors, or unfamiliar environments. To address this challenge, we propose a memory-efficient replay-based domain incremental learning (DIL) framework, ReDIaL, that adapts to sequential domain shifts while minimizing catastrophic forgetting. Our approach employs a frozen encoder to create a stable latent space and a clustering-based exemplar replay strategy to retain compact, representative samples from prior domains under strict memory constraints. We evaluate the framework on a multi-domain air-marshalling gesture recognition task, where an in-house dataset serves as the initial training domain and the NATOPS dataset provides 20 cross-user domains for sequential adaptation. During each adaptation step, training data from the current NATOPS subject is interleaved with stored exemplars to retain prior knowledge while accommodating new knowledge variability. Across 21 sequential domains, our approach attains 97.34% accuracy on the domain incremental setting, exceeding pooled fine-tuning (91.87%), incremental fine-tuning (80.92%), and Experience Replay (94.20%) by +5.47, +16.42, and +3.14 percentage points, respectively. Performance also approaches the joint-training upper bound (98.18%), which represents the ideal case where data from all domains are available simultaneously. These results demonstrate that memory-efficient latent exemplar replay provides both strong adaptation and robust retention, enabling practical and trustworthy gesture-based human–robot interaction in dynamic real-world deployments. 
    more » « less
  6. Long-horizon tasks in unstructured environments are notoriously challenging for robots because they require the prediction of extensive action plans with thousands of steps while adapting to ever-changing conditions by reasoning among multimodal sensing spaces. Humans can efficiently tackle such compound problems by breaking them down into easily reachable abstract sub-goals, significantly reducing complexity. Inspired by this ability, we explore how we can enable robots to acquire sub-goal formulation skills for long-horizon tasks and generalize them to novel situations and environments. To address these challenges, we propose the Zero-shot Abstract Sub-goal Framework (ZAS-F), which empowers robots to decompose overarching action plans into transferable abstract sub-goals, thereby providing zero-shot capability in new task conditions. ZAS-F is an imitation-learning-based method that efficiently learns a task policy from a few demonstrations. The learned policy extracts abstract features from multimodal and extensive temporal observations and subsequently uses these features to predict task-agnostic sub-goals by reasoning about their latent relations. We evaluated ZAS-F in radio frequency identification (RFID) inventory tasks across various dynamic environments, a typical long-horizon task requiring robots to handle unpredictable conditions, including unseen objects and structural layouts. Ourexperiments demonstrated that ZAS-F achieves a learning efficiency 30 times higher than previous methods, requiring only 8k demonstrations. Compared to prior approaches, ZAS-F achieves a 98.3% scanning accuracy while significantly reducing the training data requirement. Further, ZAS-F demonstrated strong generalization, maintaining a scan success rate of 99.4% in real-world deployment without additional finetuning. In long-term operations spanning 100 rooms, ZAS-F maintained consistent performance compared to short-term tasks, highlighting its robustness against compounding errors. These results establish ZAS-F as an efficient and adaptable solution for long-horizon robotic tasks in unstructured environments. 
    more » « less