skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Friday, May 2 until 12:00 AM ET on Saturday, May 3 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Dynamical Measurement of Team Resilience
Resilient teams overcome sudden, dynamic changes by enacting rapid, adaptive responses that maintain system effectiveness. We analyzed two experiments on human-autonomy teams (HATs) operating a simulated remotely piloted aircraft system (RPAS) and correlated dynamical measures of resilience with measures of team performance. Across both experiments, HATs experienced automation and autonomy failures, using a Wizard of Oz paradigm. Team performance was measured in multiple ways, using a mission-level performance score, a target processing efficiency score, a failure overcome score, and a ground truth resilience score. Novel dynamical systems metrics of resilience measured the timing of system reorganization in response to failures across RPAS layers, including vehicle, controls, communications layers, and the system overall. Time to achieve extreme values of reorganization and novelty of reorganization were consistently correlated with target processing efficiency and ground truth resilience across both studies. Correlations with mission-level performance and the overcome score were apparent but less consistent. Across both studies, teams displayed greater system reorganization during failures compared to routine task conditions. The second experiment revealed differential effects of team training focused on coordination coaching and trust calibration. These results inform the measurement and training of resilience in HATs using objective, real-time resilience analysis.  more » « less
Award ID(s):
2019805 1828010
PAR ID:
10499591
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Journal of Cognitive Engineering and Decision Making
Date Published:
Journal Name:
Journal of Cognitive Engineering and Decision Making
Volume:
17
Issue:
4
ISSN:
1555-3434
Page Range / eLocation ID:
351 to 382
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; (, Human Factors: The Journal of the Human Factors and Ergonomics Society)
    ObjectiveThis study examines low-, medium-, and high-performing Human-Autonomy Teams’ (HATs’) communication strategies during various technological failures that impact routine communication strategies to adapt to the task environment. BackgroundTeams must adapt their communication strategies during dynamic tasks, where more successful teams make more substantial adaptations. Adaptations in communication strategies may explain how successful HATs overcome technological failures. Further, technological failures of variable severity may alter communication strategies of HATs at different performance levels in their attempts to overcome each failure. MethodHATs in a Remotely Piloted Aircraft System-Synthetic Task Environment (RPAS-STE), involving three team members, were tasked with photographing targets. Each triad had two randomly assigned participants in navigator and photographer roles, teaming with an experimenter who simulated an AI pilot in a Wizard of Oz paradigm. Teams encountered two different technological failures, automation and autonomy, where autonomy failures were more challenging to overcome. ResultsHigh-performing HATs calibrated their communication strategy to the complexity of the different failures better than medium- and low-performing teams. Further, HATs adjusted their communication strategies over time. Finally, only the most severe failures required teams to increase the efficiency of their communication. ConclusionHAT effectiveness under degraded conditions depends on the type of communication strategies enacted by the team. Previous findings from studies of all-human teams apply here; however, novel results suggest information requests are particularly important to HAT success during failures. ApplicationUnderstanding the communication strategies of HATs under degraded conditions can inform training protocols to help HATs overcome failures. 
    more » « less
  2. ; ; ; ; ; (, Human Factors: The Journal of the Human Factors and Ergonomics Society)
    Objective This work examines two human–autonomy team (HAT) training approaches that target communication and trust calibration to improve team effectiveness under degraded conditions. Background Human–autonomy teaming presents challenges to teamwork, some of which may be addressed through training. Factors vital to HAT performance include communication and calibrated trust. Method Thirty teams of three, including one confederate acting as an autonomous agent, received either entrainment-based coordination training, trust calibration training, or control training before executing a series of missions operating a simulated remotely piloted aircraft. Automation and autonomy failures simulating degraded conditions were injected during missions, and measures of team communication, trust, and task efficiency were collected. Results Teams receiving coordination training had higher communication anticipation ratios, took photos of targets faster, and overcame more autonomy failures. Although autonomy failures were introduced in all conditions, teams receiving the calibration training reported that their overall trust in the agent was more robust over time. However, they did not perform better than the control condition. Conclusions Training based on entrainment of communications, wherein introduction of timely information exchange through one team member has lasting effects throughout the team, was positively associated with improvements in HAT communications and performance under degraded conditions. Training that emphasized the shortcomings of the autonomous agent appeared to calibrate expectations and maintain trust. Applications Team training that includes an autonomous agent that models effective information exchange may positively impact team communication and coordination. Training that emphasizes the limitations of an autonomous agent may help calibrate trust. 
    more » « less
  3. ; ; ; ; ; ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    Workplace research suggests that roughly equal communication between teammates is positively associated with team effectiveness. A distinction between teams in these studies and distributed action teams is the degree of role specialization and context-driven communication which may entail unequal degrees of communication. Yet, distributed action teams may have more equal footing to provide inputs in contexts such as mission planning or briefings. Twenty-two ad hoc teams participated in a simulated ground combat vehicle task in which teams conducted six-missions and briefed before each mission. We used team performance, team situation awareness, team workload, and team resilience as team effectiveness criteria. Balanced degrees of communication in mission briefs were correlated with performance and resilience measures, and largely uncorrelated with situation-awareness and workload measures. The overall amount of communication was also largely uncorrelated with all effectiveness measures. The results suggest that communication balance in mission briefs may help predict effectiveness in action teams. 
    more » « less
  4. ; ; ; ; ; ; ; ; (, IEEE Systems and Information Engineering Design Symposium)
    This paper introduces a new testbed called Cooking with Humans and Autonomy in Overcooked!2 for studying Performance and Teaming (CHAOPT). A validation study was conducted to examine the viability of Overcooked!2 as a research platform to explore teamwork and communication in human-autonomy teams. Unique measures derived from this platform such as productive chef actions (PCA), team expertise score and chef role contribution (CRC) distinguished performance between levels and players. Our findings demonstrate that we can derive meaningful team process, performance and communication measures and that the interactions within Overcooked!2 meet the requirements of psychological fidelity of teaming research. 
    more » « less
  5. ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    This research explores the potential of shared leadership as a complementary approach to function allocation within Human-Autonomy Teams (HATs), particularly in large, multi-agent contexts. Through a literature review of shared leadership in all-human organizational contexts, conducted using three databases, Web of Science, Engineering Village, and Google Scholar, the article identifies and outlines two mechanisms in shared leadership—decentralization and mutual influence—that appear promising for improving team processes and outcomes in HATs. This research underscores the need for developing frameworks or engineering guidance incorporating these mechanisms to enhance adaptability and performance in complex and dynamic environments. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email