In the automotive industry, a revolution is taking place: the rise of Autonomous Vehicles (AVs). AVs have been demonstrated to lower transportation costs and energy consumption, improve travel convenience and comfort, and reduce traffic accidents and congestion [1]. However, the Autonomous Driving (AD) system, which serves as the "brain" of an AV to make driving decisions, may have vulnerabilities that could lead to severe security threats to road safety. For example, vulnerabilities in the localization module, whose outputs are critical for route planning and navigation, can be exploited by attackers to manipulate the routes of AVs [2]. In fact, attacks on the localization will not only affect AVs' route planning and navigation but also have direct effects in other modules such as the perception module to cause false detection.
The perception module processes sensor inputs to perceive the surrounding obstacles and traffic lights, which are necessary for planning a safe driving path and making the correct driving decision. Typically, the perception sensors (i.e., camera, LiDAR, radar, ultrasonic sensor) in the AD system [3], [4] have wide ranges of views. For example, AVs are usually equipped with high-resolution cameras that are capable of detecting obstacles as far as 100 meters or more [3]. However, in this work, we discover an interesting design consideration, which is common in AD systems, that enables an attacker to blind or misguide the perception without tampering the perception sensor inputs themselves.
The root cause for such a vulnerability is the design of Region-of-Interests (ROIs). ROI is a strategy commonly employed in AD perception that utilizes information from the localization to narrow the detection scope in the sensor Workshop on Automotive and Autonomous Vehicle Security (AutoSec) 2021 Depending on the sensor availability, the localization module can be configured in GPS mode or Multi-Sensor Fusion (MSF) mode [9]. The GPS mode directly takes positioning from GPS, while the MSF mode combines GPS and LiDAR for more accurate and robust positioning. For example, Baidu Apollo, by default, uses MSF in the localization; however, it also provides a GPS mode when there is no LiDAR installed on the AV. Tesla Autopilot, as a Level-2 AD system [7], also do not use LiDARs and thus relies on GPS for localization [10].
The perception module incorporates sensors such as cameras, radars, LiDARs, and ultrasonic sensors to recognize vehicles and pedestrians surrounding the AV. In addition, it also performs important tasks such as TL detection.
TL detection and ROI design. TL detection is an essential feature for Level-4 AVs. If an AV cannot detect and recognize TLs, it may violate traffic rules resulting in some catastrophic consequences, such as running red lights and causing car accidents. Recently, as Level-2 AV companies aiming to achieve higher level driving autonomy, companies such as Tesla starts to add TL detection into their AD systems [11]. TL detection leverages Deep Neural Networks (DNNs) to detect and classify TLs in the camera images [3], [4]. For example, Baidu Apollo applies two DNNs for TL detection; one for the object detection to recognize the TL object in the image, and the other for the classification to recognize the light color [3].
Since the camera image may contain multiple TLs, it is thus necessary to identify the correct TL for the current intersection. To address this, AD systems commonly adopt an ROI design [3], [4], which projects the current TL in the world coordinates obtained from the High-Definition (HD) map to image coordinates based on the localization outputs (Fig. 1). To compensate for any localization or HD map inaccuracies, an ROI area with an empirically determined radius or height/width centered at the projected image coordinates is selected for TL detection. Such an ROI design not only helps prevent ambiguous TL detection but also reduces the computation overhead. Fig. 2 show an example of the ROI in Baidu Apollo. As shown, the ROI anchor box is the projected position of the TL. However, due to inaccuracies in localization and HD map, the projected position is not perfectly aligned with the actual TL. Thus, Baidu Apollo defines a larger rectangle as the ROI for TL detection ( 2 in Fig. 2). After that, it crops the ROI area from the image and applies DNNs to recognize all TL bounding boxes and their colors in the ROI. Finally, the TL bounding box ( 3 in Fig. 2), which is the closest to the ROI anchor box, is selected as the TL detection result.
Civilian GPS systems are known to be vulnerable to spoofing attacks due to the lack of signal authentication in the infrastructure. In the spoofing attack, the attacker broadcasts fake satellite signals to the victim GPS receiver to deceive it into resolving false positions specified by the attacker [5]. So far, GPS spoofing has been demonstrated to be feasible on various systems, including smartphones [12], drones [5], and vehicles such as Tesla [10].
The attack capabilities of GPS spoofing are different in GPS-based localization and MSF-based localization. In GPS mode, GPS inputs are the only positioning source for localization, and thus GPS spoofing can directly control the localization outputs in the AD system. In MSF mode, since the positioning from GPS will be fused with LiDAR, GPS spoofing cannot set an arbitrary position in the localization output. Nevertheless, Shen et al. [6] have demonstrated that they can inject a large deviation in the MSF-based localization outputs using GPS spoofing.
Threat model. Similar to the threat model used in prior works [6], [12], we assume a car-following model where the attacker launches GPS spoofing attack while tailgating the victim AV. We assume the attacker can arbitrarily manipulate the victim's localization outputs under GPS spoofing, which is valid for AVs using only GPS for localization, such as Baidu Apollo in GPS mode and Tesla. We also assume the attacker can track the physical position of the victim AV, which is feasible if the attacker's vehicle is also an AV [6].
Attack goal. The goal of our attack is to influence the position of ROI in the perception module via GPS spoofing, and thus cause the victim AV to detect a wrong TL to run the red light or fail to detect any TLs to stop at the green light (i.e., denial-of-service or DoS).
Attack insight. Although from high-level design, the localization and perception modules appear to be independent of each other, the perception module in fact heavily relies on the localization, especially for the ROI logic in TL detection as mentioned in §II-A. Since the localization output directly determines the ROI that the TL detection pipeline will be performed on, any errors in the localization would result in a wrong ROI and may cause incorrect TL detection.
Attack design. To perform the ROI attack on TL detection, the attacker launches GPS spoofing when the victim AV is in front of an intersection. When the victim AV drives towards the intersection, a longitudinal error in the localization would naturally have a smaller effect on the ROI position compared to a lateral deviation. Thus, to maximize the effect on the ROI, we design the attack as spoofing a lateral distance away from the physical position of the victim AV, e.g., the spoofing distance shown on the Fig. 3. Fig. 3 illustrates the ROI attack for TL detection, which is composed of a reconnaissance stage and a spoofing stage. In the reconnaissance stage, the attacker examines the current TL status and determines the best attack parameters to be applied. Depending on the TL status, the attacker will use different spoofing distances to achieve the desired attack goals. For example, when the TL is red, the attacker needs to spoof a distance such that the ROI includes another green TL in the camera view to cause false detection. Alternatively, if the TL is green, the attacker can simply spoof a distance large enough such that the ROI contains no TLs, which would trigger a safe stop in the planning since the AV knows from the HD map that there is a TL in the front but fails to detect one. In addition, we set an attack range threshold for determining the timing to launch the spoofing. If the attacker starts spoofing when the victim is too far or too close to the intersection, the victim might have already been deviated out of the road boundary that prevented it from reaching the intersection or have already detected the correct TL color and committed the correct driving decision. After that, the attacker closely monitors the victim's position and spoofs the corresponding lateral distance when the victim enters the attack range.
V. EVALUATION Experimental setup. Due to the high cost of evaluating self-driving algorithms on real AVs, we follow the common practice for AV testing and perform a simulation-based evaluation, in which we run Baidu Apollo in an industrygrade AV simulator, LGSVL [13]. To facilitate AV simulation, LGSVL provides photo-realistic simulation environments with diverse road structures and a wide range of vehicle models. In our evaluation, we use the Shalun map and Lincoln MKZ vehicle. The Shalun map models a common two-lane road with multiple intersections along the road. We use Lincoln MKZ since it contains the compatible sensor configurations for Baidu Apollo. We simulate our attack on Baidu Apollo version 5.0, which is the latest version fully supported by LGSVL.
To simulate the attack consequences, we create two concrete attack scenarios: red light detection and green light detection. Specifically, for red light detection, we set the front TL in the first intersection to red and the back TL in the second intersection to green; for green light detection, we set the front TL to green and the back TL to red. We have confirmed that in benign driving, the AV will always correctly detect the front TLs, and make the correct driving decisions, i.e., stop before the intersection or drive through the intersection.
Evaluation metric. In our evaluation, we explore the attack effectiveness of the ROI attack under different attack range thresholds and spoofing distances, aiming to find the parameters that can achieve the highest attack effectiveness. We define a successful attack case as the victim AV mis-detects the front TL and commits the wrong driving decisions, i.e., running the red light or stopping in front of the green light. Since both Baidu Apollo and LGSVL involve random factors such as messaging delays, we calculate a success rate by repeating the simulation for 5 times for each attack parameter.
Attack construction. For the ease of evaluation, we implement the attack logic as an independent module in Baidu Apollo to receive original GPS inputs from LGSVL. If the victim AV within the attack range threshold, we apply the spoofing distance to the original GPS positions. After that, we publish the spoofed GPS inputs to the localization module.
Results. As mentioned in §IV, the red light detection scenario has a more restricted spoofing distance requirement since the attacker needs to spoof the GPS such that the ROI covers the back green TL but not the front red TL. Thus, we start by exploring the attack parameters for this scenario. The left figure in Fig. 4 shows the attack success rates when using different attack range thresholds, i.e., the distance from the AV to the TL that triggers the attack. During the experiments, a spoofing distance of 3 meters is applied to the victim's GPS inputs. As shown, the best attack range threshold falls between 24 and 28 meters. When using a small attack range threshold (e.g., 22 meters), the attack fails because the victim is too close to the intersection, and it already executed the stop decision based on the TL detection prior to the attack. On the other hand, if the attack range threshold is too large (e.g., over 28 meters), the victim might already be deviated out of road boundary since the AD system is constantly correcting any deviation between the localization and the planned trajectory [6].
The right figure in Fig. 4 shows the attack success rates when using different spoofing distances. Based on the previous experiments on the attack range threshold, we launch the attack when the victim AV is 26 meters away from the TL. As shown in Fig. 4, a spoofing distance of smaller than or equal to 2 meters is too small to move the red TL out of the ROI, so the victim can still detect it. On the other hand, when the spoofing distance is larger than 4.5 meters, both front and back TLs are moved out of the ROI, causing the victim to fail to detect any TL and simply stop in front of the intersection. When the spoofing distance is 3-4.5 meters, the attack success rate is at least 80%, where the front red TL is moved out of the ROI, but the back green TL is still in and thus detected by the victim AV, causing it to continue driving forward to run the red light. In particular, a 100% attack success rate is achieved when using a spoofing distance of 3 meters.
In the green light detection scenario, instead of controlling the ROI to be at a particular position, the attacker can simply spoof the ROI to the area without any TLs. In such a case, the TL detector will report "unknown" as no TLs exist in the ROI. Consequently, the planning module in the AD system issues a stop decision conservatively. This results in a DoS attack since the victim stops at the green TL. Similarly, we evaluate this scenario and found that our ROI attack can achieve a 100% success rate when using a spoofing distance of 5 meters.
Attack demos. We create two attack demos to illustrate the severe consequences of the ROI attack. Fig. 5 and Fig. 6 show the snapshots when attacking the red and green light detections, respectively. The left sub-figures show the camera images with the ROI annotations, and the right sub-figures show the corresponding driving decisions in Baidu Apollo. As shown, our attack can successfully fool the victim AV to detect a wrong TL or fail to detect any TLs due to the ROI position shifts caused by GPS spoofing. Such an attack poses great dangers to road safety since it can cause the AV to violate traffic rules and may even lead to car crashing consequences, e.g.,
Related work. To the best of our knowledge, we are the first to attack ROI and TL detection in AD systems. Since TL detection relies on DNNs and camera inputs, thus attacks to these two are direct threats to TL detection. Prior work used physical-world perturbations to fool the DNNs. For example, Eykholt et al. [14] uses stickers with carefully devised patterns to fool the stop sign detection. However, TLs are not as accessible as stop signs since they are usually installed at a high place and thus it is unclear how the attacker can put the stickers without alerting other drivers and law enforcement. Another line of research explored physical sensor attacks to blind the camera by shooting strong lights to it [15]. However, such attacks cannot cause the victim to be a red light runner and can only lead to denial-of-service. In this work, the ROI attack does not require any physical perturbations to the TLs and can achieve both attack goals.
Limitations and future work. As the first study of ROI attacks in AD perception using GPS spoofing, we start by attacking AD system with a GPS-based localization, where GPS is the only positioning resource. Indeed, for GPS-based localization, it is possible to simply spoof the GPS to a road without any TLs, such that the victim is not in a state where it realizes that it should sense a TL, resulting in red light running. However, this is not the case for MSF-based localization, which is predominantly adopt in today's Level-4 AVs, since GPS alone can no longer dictate the localization output. Although MSF is considered to be more robust against GPS spoofing, prior work has demonstrated that MSF-based localization can be deviated by as large as 10 meters using GPS spoofing [6]. In the future, we plan to incorporate the MSF attack to improve the practicality of the ROI attack. Another limitation is that we only evaluate the attack on one AD system, i.e., Baidu Apollo [3]. Although Baidu Apollo implements a representative TL ROI design and is already in production, it is necessary to also evaluate on other TL ROI implementations (e.g., Autoware [4]) to show the generality of our attack. We leave this as a future work. Besides, since the attack parameters are highly dependent on factors such as victim's speed and road shape, more diverse driving scenarios are needed to evaluate the applicability of our attack. In addition, currently the best attack parameters are found via exhaustive search. In the next step, we plan to use the victim AV/TL positions and camera configurations to calculate the best spoofing distances to be applied in different scenarios.
VII. CONCLUSION In this paper, we perform the first security analysis on perception ROI design and take the TL detection as a case study. We discover that using GPS spoofing can cause the AV to recognize a wrong TL or fail to detect any TLs. Our study shows that strategic GPS spoofing can achieve a 100% success rate to fool TL detection in the AV, leading to red-light running or denial-of-service consequences, which can be detrimental to road safety. The future of AVs is bright, but more security investigations are still needed in the AD systems. We hope our research can bring more attention to the security threats to AVs, improving AD systems for safer AVs for the public.
February 2021, Virtual ISBN 1-891562-68-1 https://dx.doi.org/10.14722/autosec.2021.23029 www.ndss-symposium.org