More Like this

1. Neural 3D Gaze: 3D Pupil Localization and Gaze Tracking based on Anatomical Eye Model and Neural Refraction Correction

   https://doi.org/10.1109/ISMAR55827.2022.00053  

   Lu, Conny; Chakravarthula, Praneeth; Liu, Kaihao; Liu, Xixiang; Li, Siyuan; Fuchs, Henry (October 2022, Proccedings of the 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR))

   Eye tracking has already made its way to current commercial wearable display devices, and is becoming increasingly important for virtual and augmented reality applications. However, the existing model-based eye tracking solutions are not capable of conducting very accurate gaze angle measurements, and may not be sufficient to solve challenging display problems such as pupil steering or eyebox expansion. In this paper, we argue that accurate detection and localization of pupil in 3D space is a necessary intermediate step in model-based eye tracking. Existing methods and datasets either ignore evaluating the accuracy of 3D pupil localization or evaluate it only on synthetic data. To this end, we capture the first 3D pupilgaze-measurement dataset using a high precision setup with head stabilization and release it as the first benchmark dataset to evaluate both 3D pupil localization and gaze tracking methods. Furthermore, we utilize an advanced eye model to replace the commonly used oversimplified eye model. Leveraging the eye model, we propose a novel 3D pupil localization method with a deep learning-based corneal refraction correction. We demonstrate that our method outperforms the state-of-the-art works by reducing the 3D pupil localization error by 47.5% and the gaze estimation error by 18.7%. Our dataset and codes can be found here: link. 

   more » « less
2. Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality

   https://doi.org/10.1145/3654705  

   Barkevich, Kevin; Bailey, Reynold; Diaz, Gabriel J (May 2024, Proceedings of the ACM on Computer Graphics and Interactive Techniques)

   Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate. 

   more » « less
3. Beyond gaze: preliminary analysis of pupil dilation and blink rates in an fMRI study of program comprehension

   https://doi.org/10.1145/3216723.3216726  

   Peitek, Norman; Siegmund, Janet; Parnin, Chris; Apel, Sven; Brechmann, André (January 2018, Proceedings of the Workshop on Eye Movements in Programming (EMIP'18))

   Researchers have been employing psycho-physiological measures to better understand program comprehension, for example simultaneous fMRI and eye tracking to validate top-down comprehension models. In this paper, we argue that there is additional value in eye-tracking data beyond eye gaze: Pupil dilation and blink rates may offer insights into programmers' cognitive load. However, the fMRI environment may influence pupil dilation and blink rates, which would diminish their informative value. We conducted a preliminary analysis of pupil dilation and blink rates of an fMRI experiment with 22 student participants. We conclude from our preliminary analysis that the correction for our fMRI environment is challenging, but possible, such that we can use pupil dilation and blink rates to more reliably observe program comprehension. 

   more » « less
4. Improved vergence and accommodation via Purkinje Image tracking with multiple cameras for AR glasses

   https://doi.org/10.1109/ISMAR50242.2020.00058  

   Lu, Conny; Chakravarthula, Praneeth; Tao, Yujie; Chen, Steven; Fuchs, Henry (November 2020, 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR))

   null (Ed.)

   We present a personalized, comprehensive eye-tracking solution based on tracking higher-order Purkinje images, suited explicitly for eyeglasses-style AR and VR displays. Existing eye-tracking systems for near-eye applications are typically designed to work for an on-axis configuration and rely on pupil center and corneal reflections (PCCR) to estimate gaze with an accuracy of only about 0.5°to 1°. These are often expensive, bulky in form factor, and fail to estimate monocular accommodation, which is crucial for focus adjustment within the AR glasses. Our system independently measures the binocular vergence and monocular accommodation using higher-order Purkinje reflections from the eye, extending the PCCR based methods. We demonstrate that these reflections are sensitive to both gaze rotation and lens accommodation and model the Purkinje images’ behavior in simulation. We also design and fabricate a user-customized eye tracker using cheap off-the-shelf cameras and LEDs. We use an end-to-end convolutional neural network (CNN) for calibrating the eye tracker for the individual user, allowing for robust and simultaneous estimation of vergence and accommodation. Experimental results show that our solution, specifically catering to individual users, outperforms state-of-the-art methods for vergence and depth estimation, achieving an accuracy of 0.3782°and 1.108 cm respectively. 

   more » « less
5. Eye tracking measures of bicyclists’ behavior and perception: A systematic review

   https://doi.org/10.1016/j.trf.2024.08.026  

   Ma, Shiyu; Zhang, Wenwen; Noland, Robert B; Andrews, Clinton J (November 2024, Transportation Research Part F: Traffic Psychology and Behaviour)

   With improved portability and affordability, eye tracking devices have facilitated an expanding range of cycling experiments aimed at understanding cycling behavior and potential risks. Given the complexity of cyclists’ visual behavior and gaze measurements, we provide a comprehensive review with three key focuses: 1) the adoption and interpretation of various gaze metrics derived from cycling experiments, 2) a summary of the findings of those experiments, and 3) identifying areas for future research. A systematic review of three databases yielded thirty-five articles that met our inclusion criteria. Our review results show that cycling experiments with eye tracking allow analysis of the viewpoint of the cyclist and reactions to the built environment, road conditions, navigation behavior, and mental workload and/or stress levels. Our review suggests substantial variation in research objectives and the consequent selection of eye-tracking devices, experimental design, and which gaze metrics are used and interpreted. A variety of general gaze metrics and gaze measurements related to Areas of Interest (AOI) are applied to infer cyclists’ mental workload/stress levels and attention allocation respectively. The diversity of gaze metrics reported in the literature makes cross-study comparisons difficult. Areas for future research, especially potential integration with computer vision are also discussed. 

   more » « less