More Like this

1. How Coordinated Visual Attention on a Target Area of Interest is Impacted by a Change in Workload Over Time

   https://doi.org/10.1177/1071181320641083  

   Devlin, Shannon P.; Flynn, Jake R.; Riggs, Sara L. (December 2020, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)

   null (Ed.)

   Data-rich environments rely on operators to collaborate, especially in light of workload changes. This work explores the relationship between the operators’ shared visual attention patterns on a target area of interest (AOI), i.e. the AOI causing a workload change, and how it affects collaborative performance. Eye tracking data was collected from ten pairs of participants who completed two scenarios, the first being low workload and the second being high workload, in an unmanned aerial vehicle (UAV) command and control testbed. Then, best and worst performing pairs were compared in terms of two shared visual attention metrics: (1) percent gaze overlap and (2) the phi coefficient for the target AOI. The results showed that coordinated visits to and from the target AOI were associated with better performance during high workload. These results suggest including quantitative measures of visual attention can be indicators of the adaptation process in real- time. 

   more » « less
2. Scan-based eye tracking measures are predictive of workload transition performance

   Shannon P Devlin, Noelle L (November 2022, Applied Ergonomics)

   Given there is no unifying theory or design guidance for workload transitions, this work investigated how visual attention allocation patterns could inform both topics, by understanding if scan-based eye tracking metrics could predict workload transition performance trends in a context-relevant domain. The eye movements of sixty Naval flight students were tracked as workload transitioned at a slow, medium, and fast pace in an unmanned aerial vehicle testbed. Four scan-based metrics were significant predictors across the different growth curve models of response time and accuracy. Stationary gaze entropy (a measure of how dispersed visual attention transitions are across tasks) was predictive across all three transition rates. The other three predictive scan-based metrics captured different aspects of visual attention, including its spread, directness, and duration. The findings specify several missing details in both theory and design guidance, which is unprecedented, and serves as a basis of future workload transition research. 

   more » « less
3. Scan-based eye tracking measures are predictive of workload transition performance

   https://doi.org/10.1016/j.apergo.2022.103829  

   Devlin, S.P; Brown, N.L.; Drollinger, S.; Sibley, C.; Alami, J.; Riggs, S.L. (November 2022, Applied Ergonomics)

   Given there is no unifying theory or design guidance for workload transitions, this work investigated how visual attention allocation patterns could inform both topics, by understanding if scan-based eye tracking metrics could predict workload transition performance trends in a context-relevant domain. The eye movements of sixty Naval flight students were tracked as workload transitioned at a slow, medium, and fast pace in an unmanned aerial vehicle testbed. Four scan-based metrics were significant predictors across the different growth curve models of response time and accuracy. Stationary gaze entropy (a measure of how dispersed visual attention transitions are across tasks) was predictive across all three transition rates. The other three predictive scan-based metrics captured different aspects of visual attention, including its spread, directness, and duration. The findings specify several missing details in both theory and design guidance, which is unprecedented, and serves as a basis of future workload transition research. 

   more » « less
4. Effects of Workload and Workload Transitions on Attention Allocation in a Dual-Task Environment: Evidence From Eye Tracking Metrics

   https://doi.org/10.1177/1555343419892184  

   Moacdieh, Nadine_Marie; Devlin, Shannon_P; Jundi, Hussein; Riggs, Sara_Lu (January 2020, Journal of Cognitive Engineering and Decision Making)

   High mental workload, in addition to changes in workload, can negatively affect operators, but it is not clear how sudden versus gradual workload transitions influence performance and visual attention allocation. This knowledge is important as sudden shifts in workload are common in multitasking domains. The objective of this study was to investigate, using performance and eye tracking metrics, how constant versus variable levels of workload affect operators in the context of a dual-task paradigm. An unmanned aerial vehicle command and control simulation varied task load between low, high, gradually transitioning from low to high, and suddenly transitioning from low to high. Performance on a primary and secondary task and several eye tracking measures were calculated. There was no significant difference between sudden and gradual workload transitions in terms of performance or attention allocation overall; however, both sudden and gradual workload transitions changed participants’ strategy in dealing with the primary and secondary task as compared to low/high workload. Also, eye tracking metrics that are not frequently used, such as transition rate and stationary entropy, provided more insight into performance differences. These metrics can potentially be used to better understand operators’ strategies and could form the basis of an adaptive display. 

   more » « less
5. Integrated Eye-Tracking and EEG Data Collection and Synchronization for Virtual Reality-Based Spatial Ability Assessments

   https://doi.org/10.54941/ahfe1004499  

   Salehi, Faezeh; Razavi, Moein; Smith, Mason; Dixit, Manish (January 2024, Intelligent Human Systems Integration (IHSI 2024), Vol. 119, 2024, 348–360)

   In the realm of virtual reality (VR) research, the synergy of methodological advancements, technical innovation, and novel applications is paramount. Our work encapsulates these facets in the context of spatial ability assessments conducted within a VR environment. This paper presents a comprehensive and integrated framework of VR, eye-tracking, and electroencephalography (EEG), which seamlessly combines measuring participants’ behavioral performance and simultaneously collecting time-stamped eye tracking and EEG data to enable understanding how spatial ability is impacted in certain conditions and if such conditions demand increased attention and mental allocation. This framework encompasses the measurement of participants’ gaze pattern (e.g., fixation and saccades), EEG data (e.g., Alpha, Beta, Gamma, and Theta wave patterns), and psychometric and behavioral test performance. On the technical front, we utilized the Unity 3D game engine as the core for running our spatial ability tasks by simulating altered conditions of space exploration. We simulated two types of space exploration conditions: (1) microgravity condition in which participants’ idiotropic (body) axis is in statically and dynamically misaligned with their visual axis; and (2) conditions of Martian terrain that offers a visual frame of reference (FOR) but with limited and unfamiliar landmarks objects. We specifically targeted assessing human spatial ability and spatial perception. To assess spatial ability, we digitalized behavioral tests of Purdue Spatial Visualization Test: Rotations (PSVT: R), the Mental Cutting Test (MCT), and the Perspective Taking Ability (PTA) test and integrated them into the VR settings to evaluate participants’ spatial visualization, spatial relations, and spatial orientation ability, respectively. For spatial perception, we applied digitalized versions of size and distance perception tests to measure participants’ subjective perception of size and distance. A suite of C# scripts orchestrated the VR experience, enabling real-time data collection and synchronization. This technical innovation includes the integration of data streams from diverse sources, such as VIVE controllers, eye-tracking devices, and EEG hardware, to ensure a cohesive and comprehensive dataset. A pivotal challenge in our research was synchronizing data from EEG, eye tracking, and VR tasks to facilitate comprehensive analysis. To address this challenge, we employed the Unity interface of the OpenSync library, a tool designed to unify disparate data sources in the fields of psychology and neuroscience. This approach ensures that all collected measures share a common time reference, enabling meaningful analysis of participant performance, gaze behavior, and EEG activity. The Unity-based system seamlessly incorporates task parameters, participant data, and VIVE controller inputs, providing a versatile platform for conducting assessments in diverse domains. Finally, we were able to collect synchronized measurements of participants’ scores on the behavioral tests of spatial ability and spatial perception, their gaze data and EEG data. In this paper, we present the whole process of combining the eye-tracking and EEG workflows into the VR settings and collecting relevant measurements. We believe that our work not only advances the state-of-the-art in spatial ability assessments but also underscores the potential of virtual reality as a versatile tool in cognitive research, therapy, and rehabilitation. 

   more » « less