Title: Comprehensive study of driver behavior monitoring systems using computer vision and machine learning techniques
The flourishing realm of advanced driver-assistance systems (ADAS) as well as autonomous vehicles (AVs) presents exceptional opportunities to enhance safe driving. An essential aspect of this transformation involves monitoring driver behavior through observable physiological indicators, including the driver’s facial expressions, hand placement on the wheels, and the driver’s body postures. An artificial intelligence (AI) system under consideration alerts drivers about potentially unsafe behaviors using real-time voice notifications. This paper offers an all-embracing survey of neu ral network-based methodologies for studying these driver bio-metrics, presenting an exhaustive examination of their advantages and drawbacks. The evaluation includes two relevant datasets, separately categorizing ten different in-cabinet behaviors, providing a systematic classification for driver behaviors detection. The ultimate aim is to inform the development of driver behavior monitoring systems. This survey is a valuable guide for those dedicated to enhancing vehicle safety and preventing accidents caused by careless driving. The paper’s structure encompasses sections on autonomous vehicles, neural networks, driver behavior analysis methods, dataset utilization, and final findings and future suggestions, ensuring accessibility for audiences with diverse levels of understanding regarding the subject matter.  more » « less
Award ID(s):
2231200
PAR ID:
10492030
Author(s) / Creator(s):
Publisher / Repository:
Springer Publisher
Date Published:
Journal Name:
Journal on big data
Volume:
11
Issue:
32
ISSN:
2579-0048
Page Range / eLocation ID:
1-44
Subject(s) / Keyword(s):
Driver behavior monitoring system, Driver behavior classification, Computer vision, Machine learning, Autonomous vehicles
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Autonomous Vehicle (AV) technology has the potential to significantly improve driver safety. Unfortunately, driver could be reluctant to ride with AVs due to the lack of trust and acceptance of AV’s driving styles. The present study investigated the impact of driver’s driving style (aggressive/defensive) and the designed driving styles of AVs (aggressive/defensive) on driver’s trust, acceptance, and take-over behavior in fully autonomous vehicles. Thirty-two participants were classified into two groups based on their driving styles using the Aggressive Driving Scale and experienced twelve scenarios in either an aggressive AV or a defensive AV. Results revealed that drivers’ trust, acceptance, and takeover frequency were significantly influenced by the interaction effects between AV’s driving style and driver’s driving style. The findings implied that driver’s individual differences should be considered in the design of AV’s driving styles to enhance driver’s trust and acceptance of AVs and reduce undesired take over behaviors. 
    more » « less
  2. Autonomous vehicles are expected to improve road safety and efficiency in future transportation systems. A driving simulator study was designed to identify driving styles and the cooperation between human drivers and other AVs. The study captured driver’s following behavior in a fully autonomous driving environment at unsignalized intersections. Participants were asked to make a series of maneuvers (straight through intersection, left turn, and right turn) in two different speed conditions (30, 40 mph) and two different traffic density conditions (with or without other traffic). Analysis of Variance showed that drivers had a significantly larger deviation (defined as the area between two trajectories) during left turn maneuvers when they were traveling at higher speeds. Moreover, the first turning operation had smaller deviation than the second turning operation. The findings have implications for the design of driver-assistance guidance systems in future mixed autonomous and non-autonomous traffic flows. 
    more » « less
  3. As automated vehicles (AVs) become more intelligent, the concept of teaming with a semi-autonomous AV has gained considerable traction. However, few empirical studies exist that quantify the performance of an AV-(human) driver team. To contribute to the existing work in this area, the purpose of this study was to examine how the availability of both a vehicle and driver agent, or the unavailability of one agent, influenced teaming in driving. The vehicle’s individual goal: manage the dynamic driving task. The driver’s individual goal: complete a work-related document-editing task. Together, the vehicle would ask the driver about potential roadway dangers and the driver would respond to the vehicle’s queries. Overall, introducing availability allowed for function allocations where drivers had individual goals separate from driving. Study results provide initial insights into operationalizing the future vision of AVs where safety and driver productivity during commutes are maximized. 
    more » « less
  4. Social scientists have argued that autonomous vehicles (AVs) need to act as effective social agents; they have to respond implicitly to other drivers’ behaviors as human drivers would. In this paper, we investigate how contingent driving behavior in AVs influences human drivers’ experiences. We compared three algorithmic driving models: one trained on human driving data that responds to interactions (a familiar contingent behavior) and two artificial models that intend to either always-yield or never-yield regardless of how the interaction unfolds (non-contingent behaviors). Results show a statistically significant relationship between familiar contingent behavior and positive driver experiences, reducing stress while promoting the decisive interactions that mitigate driver hesitance. The direct relationship between familiar contingency and positive experience indicates that AVs should incorporate socially familiar driving patterns through contextually-adaptive algorithms to improve the chances of successful deployment and acceptance in mixed human-AV traffic environments. 
    more » « less
  5. Modern vehicles increasingly rely on advanced driver monitoring systems (DMS) to ensure safety and enhance the driving experience. These systems assess driver status to prevent accidents caused by fatigue, inattentiveness, or intoxication. While some DMS applications process video data on vehicle, many rely on edge or cloud-based solutions, raising significant privacy concerns due to the storage of sensor data from vehicles. Existing approaches, such as de-identification and homomorphic encryption, either impose heavy computational overhead on vehicles or insufficiently address privacy. To overcome these limitations, we present the Privacy-preserving Driver Monitoring System (PDMS), a novel framework based on the additive secret sharing theory and privacy-preserving Transformer-based deep learning models. PDMS creates randomized secret shares from driver’s facial video data on vehicle, processes them independently through privacy-preserving Transformer models on edges, and securely aggregates partial results on vehicle, ensuring vehicles’ sensor data and final results remain protected. This approach reduces the computational load on the vehicle, enabling cost-effective and scalable DMS solutions that protect the privacy of the driver both in transit and in processing. Our contributions include the design and optimization of the PDMS system, incorporating privacy-preserving DNN layers that are capable of processing randomized secret shares. Furthermore, we present a practical system that utilizes a vision transformer (ViT)-based gaze estimation model, demonstrating the effectiveness of PDMS through comprehensive experiments. 
    more » « less