Resilient teams overcome sudden, dynamic changes by enacting rapid, adaptive responses that maintain system effectiveness. We analyzed two experiments on human-autonomy teams (HATs) operating a simulated remotely piloted aircraft system (RPAS) and correlated dynamical measures of resilience with measures of team performance. Across both experiments, HATs experienced automation and autonomy failures, using a Wizard of Oz paradigm. Team performance was measured in multiple ways, using a mission-level performance score, a target processing efficiency score, a failure overcome score, and a ground truth resilience score. Novel dynamical systems metrics of resilience measured the timing of system reorganization in response to failures across RPAS layers, including vehicle, controls, communications layers, and the system overall. Time to achieve extreme values of reorganization and novelty of reorganization were consistently correlated with target processing efficiency and ground truth resilience across both studies. Correlations with mission-level performance and the overcome score were apparent but less consistent. Across both studies, teams displayed greater system reorganization during failures compared to routine task conditions. The second experiment revealed differential effects of team training focused on coordination coaching and trust calibration. These results inform the measurement and training of resilience in HATs using objective, real-time resilience analysis.
more »
« less
Team Communication in Pre-Mission Briefs and Effectiveness in Distributed Action Teams
Workplace research suggests that roughly equal communication between teammates is positively associated with team effectiveness. A distinction between teams in these studies and distributed action teams is the degree of role specialization and context-driven communication which may entail unequal degrees of communication. Yet, distributed action teams may have more equal footing to provide inputs in contexts such as mission planning or briefings. Twenty-two ad hoc teams participated in a simulated ground combat vehicle task in which teams conducted six-missions and briefed before each mission. We used team performance, team situation awareness, team workload, and team resilience as team effectiveness criteria. Balanced degrees of communication in mission briefs were correlated with performance and resilience measures, and largely uncorrelated with situation-awareness and workload measures. The overall amount of communication was also largely uncorrelated with all effectiveness measures. The results suggest that communication balance in mission briefs may help predict effectiveness in action teams.
more »
« less
- Award ID(s):
- 1828010
- PAR ID:
- 10432730
- Date Published:
- Journal Name:
- Proceedings of the Human Factors and Ergonomics Society Annual Meeting
- Volume:
- 66
- Issue:
- 1
- ISSN:
- 2169-5067
- Page Range / eLocation ID:
- 1596 to 1600
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Team workload is a team-level construct considered similar to, but not reducible to, individual workload and mediated by team coordination. Despite this, the conceptualization and measurement of team workload in action teams lags behind that of individual workload. In most empirical studies, team workload is often simply considered as the sum or average of individual team members’ workload. However, unique characteristics of action teams, such as interdependence and heterogeneity, suggest that traditional approaches to conceptualizing and measuring team workload may be inadequate or even misleading. As such, innovative approaches are required to accurately capture this complex construct. This paper presents the development of a simulation designed to investigate the influence of interdependence and demand levels on team workload measures within a 3-person action-team command and control scenario. Preliminary results, which suggest that our manipulations are effective, are provided and discussed.more » « less
-
Background Effective team communication is crucial for managing medical emergencies like malignant hyperthermia (MH), but current assessment methods fail to capture the dynamic and temporal nature of teamwork processes. The lack of reliable measures to inform feedback to teams is likely limiting the overall effectiveness of simulation training. This study demonstrates the application of ordered network analysis (ONA) to model communication sequences during the simulated MH scenario. Methods Twenty-two anesthesiologists participated in video-recorded MH simulations. Each scenario involved one participant as the primary anesthesiologist with confederates in supporting roles. Team communication was coded using the Team Reflection Behavioral Observation (TuRBO) framework, capturing behaviors related to information gathering, evaluation, planning, and implementation. ONA modeled the sequences of these coded behaviors as dynamic networks. Teams were classified as high- or low-performing based on timely dantrolene administration and appropriate MH treatment actions. Network visualizations and statistical tests compared communication patterns between groups. Results Five of 22 teams (23%) were high-performing. ONA revealed high-performers transitioned more effectively from situation assessment (information seeking/evaluation) to planning and implementation, while low-performers cycled between assessment behaviors without progressing (p = 0.04, Cohen’s d = 1.72). High-performers demonstrated stronger associations between invited input, explicitly assessing the situation, stating plans, and implementation. Conclusions Integrating video coding with ONA provides an innovative approach for examining team behaviors. Leveraging ONA can uncover patterns in communication timing and sequences, guiding targeted interventions to improve team coordination in various real-world clinical and simulated settings (e.g., operating room, EMS, ICU).more » « less
-
null (Ed.)Fully immersive virtual reality, with the unique ability to replicate the real world, could potentially aid in real-time communication. Geographically separated teams can collaborate using virtual reality. To test the viability of using virtual reality for remote collaboration, we designed a system called “WeRSort” where teams sorted cards in a virtual environment. Participants performed the task as a team of 2 in one of three conditions-controls-only condition, generic embodiment and full embodiment. Objective measures of performance, time and percentage match with master cards showed no significant difference. Subjective measures of presence and system usability also showed no statistical significance. However, overall work-load obtained from NASA-TLX showed that fully immersive virtual reality resulted in lower workload in comparison with the other two. Qualitative data was collected and analyzed to understand collaboration using the awareness evaluation model.more » « less
-
ObjectiveWe explore the relationships between objective communication patterns displayed during virtual team meetings and established, qualitative measures of team member effectiveness. BackgroundA key component of teamwork is communication. Automated measures of objective communication patterns are becoming more feasible and offer the ability to measure and monitor communication in a scalable, consistent and continuous manner. However, their validity in reflecting meaningful measures of teamwork processes are not well established, especially in real-world settings. MethodWe studied real-world virtual student teams working on semester-long projects. We captured virtual team meetings using the Zoom video conferencing platform throughout the semester and periodic surveys comprising peer ratings of team member effectiveness. Leveraging audio transcripts, we examined relationships between objective measures of speaking time, silence gap duration and vocal turn-taking and peer ratings of team member effectiveness. ResultsSpeaking time, speaking turn count, degree centrality and (marginally) speaking turn duration, but not silence gap duration, were positively related to individual-level team member effectiveness. Time in dyadic interactions and interaction count, but not interaction length, were positively related to dyad-level team member effectiveness. ConclusionOur study highlights the relevance of objective measures of speaking time and vocal turn-taking to team member effectiveness in virtual project-based teams, supporting the validity of these objective measures and their use in future research. ApplicationOur approach offers a scalable, easy-to-use method for measuring communication patterns and team member effectiveness in virtual teams and opens the opportunity to study these patterns in a more continuous and dynamic manner.more » « less