Due to drivers' increasing demand for in-vehicle entertainment options and the rapid development of technologies, modern automobiles are more than mechanical machines used for transportation (Eiza & Ni, 2017;Zhang, Petit, & Roberts, 2019). Vehicles are embedded with numerous electronic components, making them more advanced and automated, and allowing the possibility of internal and external vehicle connections. While intended to facilitate driving, the evolution also brings up cybersecurity issues: modern vehicles are vulnerable to cyberattacks.
Hackers can propagate cyberattacks on vehicles through several avenues, including physical and remote access (Hodge, Hauck, Gupta, & Bennett, 2019). A Jeep Cherokee was hacked by two researchers remotely from their basement and a Mitsubishi Outlander was hacked through manipulations between its mobile app and the Wi-Fi access point (Eiza & Ni, 2017;Zhang et al., 2019). Moreover, almost 100 million Volkswagen vehicles manufactured between 1995 and 2016 were vulnerable to remote, keyless-entry hacks (Garcia, Oswald, Kasper, & Pavlidès, 2016;Zhang et al., 2019).
The increasing number of electronic components, external connections, and communications embedded in vehicles account for their vulnerability to cyberattacks (Larson & Nilsson, 2008). Research has begun to address the issue by theoretically analyzing the attack surface and listing the potential outcomes. As a consequence, general recommendations regarding system design have been proposed to prevent future vehicles from cyberattacks. However, a prominent feature of cyberattacks is their randomness and unpredictability (Petit & Shladover, 2014). Combined with how fast technologies evolve and vehicle systems update, there is no panacea to prevent all vehicle cyberattacks (Zhang et al., 2019). Additionally, it is believed that the majority of software and hardware systems in vehicles are not protected against manipulations and that existing automotive systems tend to be fragile (Koscher et al., 2010;Wolf, Weimerskirch, & Wollinger, 2007;Zhang et al., 2019).
Because drivers are those who directly interact with vehicles and would respond to any potential cyberattacks, it is critical to consider their role and investigate their response behavior under such situations. Doing so would allow us to assess the consequences of vehicle cyberattacks in terms of driver safety (Cranor, 2008;Zhang et al., 2019). Yet, past research has not thoroughly studied the drivers' behavior in the context of vehicle cyberattacks.
While there is no one-size-fits-all solution to prevent vehicle cyberattacks, training and warning systems are effective in helping drivers deal with unexpected and hazardous situations (Zhang et al., 2019). Therefore, we hypothesized that if drivers could be trained on vehicle cybersecurity and receive warning messages when encountering vehicle cyberattacks, they may safely respond to cyberattack-inducted situations. Regarding drivers' response in safety-critical situations, various measures such as their glance behavior toward the side mirrors or the rear mirror, pulling the car over, and using information from inside of the vehicle for assistance are indicators as to whether the situation is properly and safely handled (Classen et al., 2010).
The present study examined how drivers respond to vehicle cyberattacks through a driving simulator study. A secondary goal was to investigate how training and warning systems affect drivers' response behavior toward such attacks. By doing so, we aimed to quantify and characterize drivers' behavior under such safety-critical situations and eventually, offer insights into vehicle cybersecurity from the perspective of drivers themselves as well as possible solutions that consider both the drivers and vehicle systems.
Phase I of this project focused on the iterative development of the training and warning systems using the human centered design process. Phase II evaluated the effectiveness of the training and warning systems in a driving simulator study and is described below.
A total of 32 participants (aged 18-26; 23 males, 7 females, and 2 non-conforming individuals) were recruited from a university campus and the surrounding town using flyers and email advertisements. A power analysis showed that with a sample size of 32 and an effect size 0.38, when setting the alpha error to 0.05, the power is 0.8. The average age of the participants was 20.4 years (SD = 2.0 years). Only individuals with a valid United States driving license were included in the study. Participants were randomly split into four groups: training only, warnings only, training + warnings, and no intervention (control group).
A fixed-based RTI (Realtime Technologies Inc.) driving simulator consisting of a fully equipped 2013 Ford Fusion surrounded by six screens with a 330-degree field of view was used for the study (Figure 1). The cab has two dynamic sidemirrors, providing participants with realistic side and rear views of the scenarios. In the car, there is a fully customizable virtual dashboard and center stack.
The driving environment consisted of long sections of roadway with four straight sections and four curves -a loopwith no traffic lights or stop signs, and a speed limit of 35 (Figure 3). Drivers experienced a total of four scenarios/drives: the first was a baseline drive whereas the next three drives contained cybersecurity events. All cybersecurity events occurred when the driver was entering the last straight section. Table 1 lists and describes the three cybersecurity events in the order they were presented to participants. The cybersecurity events were chosen based on past literature (McCarthy, Harnett, & Carter, 2014).
Participants in the training and training + warning group received training before being exposed to the cybersecurity events (i.e., between the first two drives). This training informed participants of the dangers of vehicle cyberattacks and how to respond to cyberattacks. Participants were tested on their knowledge using scenes (e.g., "What would you do if your vehicle changed lanes by itself?"). Participants in the warning and training + warning group had in-vehicle warning messages appear, as shown in Table 1. The warning was dependent on the severity of the cybersecurity event and was issued approximately 5 seconds after the cybersecurity event began. The warnings were designed and developed based on the feedback that we received from our previous user-centered design experiments as well as general guideline for warning design (Baldwin & Lewis, 2014;Wogalter & Mayhorn, 2005).
To ascertain which behaviors were indicative of a driver responding to a cyberattack, the research team viewed and annotated a portion of the videos. For example, looking in the rearview mirror for an emergency vehicle during the Siren scenario or exhibiting large steering wheel movements when trying to gain control during the Lane Change scenario were both expected reactions. A total of 11 behaviors (i.e., dependent variables) were included in the analysis: hand hesitation, foot hesitation, changing the number of hands on wheel, checking the side mirror, checking the rearview mirror, looking at the dashboard for airbag or check engine message, looking at the dashboard for a warning message, large steering wheel movement, changing lanes, pulling over, and slamming the brakes. For the 11 dependent variables, the Spearman's correlation coefficient was evaluated, and all pairs were not found to be correlated except for the "hand hesitation" and "changing number of hands on wheel" (ρ = 0.22, p = 0.03).
Participants' responses were determined from the videos recorded in Sim Observer. Coding started from the moment the cybersecurity event began. For each video, a researcher noted the time when the participant did one of the 11 predetermined reactions. To ensure accuracy and reliability, three researchers coded the videos for the same two participants and any discrepancies were rectified.
Sirens, similar to a police car or ambulance, begin to play.
Audio states "Your comfort and convenience might be impacted. Distraction might be caused". Image appears on the dashboard.
A single high-pitched beep sounds and the two warning signs illuminate on the dashboard.
Audio states "Your comfort and convenience might be impacted. Distraction might be caused". Image appears on the dashboard.
The vehicle is suddenly controlled by the experimenter, who repeatedly moves the vehicle from the left lane to the right.
Audio states "Your safety is compromised. Slow down and prepare to pull over". Image appears on the dashboard.
For all dependent variables, the equality of the mean and variance was examined to determine the appropriate model: either Poisson or logistic regression. Six variables were best suited for Poisson regression, including the count of: hand hesitation, foot hesitation, changing number of hands on wheel, changing lanes, looking at dashboard for airbag or check engine messages, and slamming on brakes. The other 5 variables were modeled using logistic regression where the counts were converted to a binary format: 0 -if the behavior never occurred or 1 -if the behavior ever occurred.
With both Poisson and logistic regression models, there were two independent variables: group (control, training, warning, and training + warning) and drive (Sirens, Dashboard Signs, and Lane Change). As these were both categorical variables, we selected a reference level of control for the group variable and Dashboard Signs for the drive variable.
Table 2 indicates the average number of times each participant engaged in each of the activities per drive (for the first 6 activities that were modeled using Poisson regression). For the last activity that was modeled using logistic regression, Table 3Table 2 indicates the percentage of participants who engaged in the activity for the lane change drive. No large steering wheel movements were recorded for the Siren or Dashboard Signs drives.
Table 2: Average number of times each participant engaged in the activity per drive for six of the dependent variables sections. These four significant dependent variables were all modeled using logistic regression and we present the percentage of participants who engaged in a certain activity.
Figure 4 summarizes results for looking at the dashboard to check for warning messages. The group factor was significant: the warning group, versus the training + warning group, decreases the log odds of looking at the dashboard by 1.42 (! = 1.42, SE = 0.71, z-score = -2, p < 0.05). The goodness of fit test was not significant (p = 0.1), implying that the model fit the data.
Figure 5 summarizes results for checking the rearview mirror. The siren factor was significant: the Sirens scenario, when compared to the Dashboard Signs scenario, increases the log odds of checking the side mirror by 3.22 (! = 3.22, SE = 0.82, z-score = 3.92, p < 0.01). The goodness of fit test was not significant (p = 0.3), implying that the model fit the data.
Figure 6 summarizes results for checking the side mirror. The lane change factor was significant: the Lane Change scenario, relative to the Dashboard Signs scenario, decreases the log odds of checking the side mirror by 1.4 (! = 1.4, SE = 0.68, z-score = -2.06, p = 0.04). The goodness of fit test was not significant (p = 0.76), implying that the model fit the data.
The objective of this study was to investigate how drivers respond to cyberattacks and to determine if training and warning systems affect drivers' responses to cyberattacks. We conducted a simulator study with 32 participants wherein some drivers received training on how to respond to cyberattacks and some drivers received warning messages (on the dashboard) about a cyberattack. Participants experienced three cyberattacks of differing severity levels.
Of the 11 behaviors that were examined, significant differences between the groups and drivers were only exhibited among four behaviors: looking at the dashboard for warning messages, checking the rearview mirror, checking the side view mirror, and pulling over. Those who received warning messages viewed them whenever they appeared. When participants heard a siren similar to an emergency vehicle, many checked their rearview mirrors, presumably to see if there was an emergency vehicle behind them. Similarly, when the vehicle began to abruptly change lanes by itself, participants were less likely to check the side view mirror. With respect to pulling over when a cybersecurity event occurred, those who received any form of training were much more likely to pull over, regardless of the scenario. Pulling over, for most participants, concluded the drive as they put the car in park. However, in the Sirens scenario, some participants pulled over, waited until the sirens stopped, continued to drive, but pulled over again once the sirens restarted.
Implications from this indicate the utility of training for improving drivers' responses to cyberattacks. Even a short training session-our training was approximately 10 minutesleads drivers to be more cautious when their vehicle behaves unexpectedly. Providing simple messages on the dashboard capture drivers' attention, but do not necessarily lead to a change in behavior. There were stark differences in how drivers responded to cyberattacks across the scenarios: the sound of emergency sirens leads drivers to check their rearview and side mirrors much more so than other scenarios.
Though this study highlighted important findings when it comes to drivers' responses to cyberattacks, there are limitations and opportunities for future work. First, when coding participant videos, the researcher had to accommodate for a limited field of view. Future work should consider combining video data with driving behavior data (e.g., vehicle speed) to ascertain drivers' responses. The lack of an effect for warnings indicates they can be improved. Relatedly, a larger study with more participants and more cybersecurity events would allow for greater generalizability. Last, drivers in this study were young; future work can consider a different driving demographic, particularly those with more driving experience or who drive a vehicle on a day to day basis (e.g., for work).
Proceedings of the 2022 HFES 66th International Annual Meeting
Copyright 2022 by Human Factors and Ergonomics Society. All rights reserved. 10.1177/1071181322661506 Proceedings of the 2022 HFES 66th International Annual Meeting