skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, October 10 until 2:00 AM ET on Friday, October 11 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the NSF-PAR

Title: Task-aware risk estimation of perception failures for autonomous vehicle
Safety and performance are key enablers for autonomous driving: on the one hand we want our autonomous vehicles (AVs) to be safe, while at the same time, their performance (e.g., comfort or progression) is key to adoption. To effectively walk the tight-rope between safety and performance, AVs need to be risk-averse, but not entirely risk-avoidant. To facilitate safe-yet-performant driving, in this paper, we develop a task-aware risk estimator that assesses the risk a perception failure poses to the AV's motion plan. If the failure has no bearing on the safety of the AV's motion plan, then regardless of how egregious the perception failure is, our task-aware risk estimator considers the failure to have a low risk; on the other hand, if a seemingly benign perception failure severely impacts the motion plan, then our estimator considers it to have a high risk. In this paper, we propose a task-aware risk estimator to decide whether a safety maneuver needs to be triggered. To estimate the task-aware risk, first, we leverage the perception failure - detected by a perception monitor - to synthesize an alternative plausible model for the vehicle's surroundings. The risk due to the perception failure is then formalized as the "relative" risk to the AV's motion plan between the perceived and the alternative plausible scenario. We employ a statistical tool called copula, which models tail dependencies between distributions, to estimate this risk. The theoretical properties of the copula allow us to compute probably approximately correct (PAC) estimates of the risk. We evaluate our task-aware risk estimator using NuPlan and compare it with established baselines, showing that the proposed risk estimator achieves the best F1-score (doubling the score of the best baseline) and exhibits a good balance between recall and precision, i.e., a good balance of safety and performance.  more » « less
Award ID(s):
2044973
NSF-PAR ID:
10492596
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Robotics: Science and Systems (RSS)
Date Published:
Journal Name:
Robotics: Science and Systems (RSS)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ( , ICC 2021 - IEEE International Conference on Communications)
    null (Ed.)
    Autonomous vehicles (AVs) rely on on-board sensors and computation capabilities to drive on the road with limited or no human intervention. However, autonomous driving decisions can go wrong for numerous reasons, leading to accidents on the road. The AVs lack a proper forensics investigation framework, which is essential for various reasons such as resolving insurance disputes, investigating attacks, compliance with autonomous driving safety guidelines, etc. To design robust and safe AVs, identifying the actual reason behind any incident involving the AV is crucial. Hence, it is essential to collect meaningful logs from different autonomous driving modules and store them in a secure and tamper-proof way. In this paper, we propose AVGuard, a forensic investigation framework that collects and stores the autonomous driving logs. The framework can generate and verify proofs to ensure the integrity of collected logs while preventing collusion attacks among multiple dishonest parties. The stored logs can be used later by investigators to identify the exact incident. Our proof-of-concept implementation shows that the framework can be integrated with autonomous driving modules efficiently without any significant overheads. 
    more » « less
  2. ; ; ; ; ; ( , IEEE Communications surveys and tutorials)
    By enabling autonomous vehicles (AVs) to share data while driving, 5G vehicular communications allow AVs to collaborate on solving common autonomous driving tasks. AVs often rely on machine learning models to perform such tasks; as such, collaboration requires leveraging vehicular communications to improve the performance of machine learning algorithms. This paper provides a comprehensive literature survey of the intersection between machine learning for autonomous driving and vehicular communications. Throughout the paper, we explain how vehicle-to-vehicle (V2V) and vehicle-to-everything (V2X) communications are used to improve machine learning in AVs, answering five major questions regarding such systems. These questions include: 1) How can AVs effectively transmit data wirelessly on the road? 2) How do AVs manage the shared data? 3) How do AVs use shared data to improve their perception of the environment? 4) How do AVs use shared data to drive more safely and efficiently? and 5) How can AVs protect the privacy of shared data and prevent cyberattacks? We also summarize data sources that may support research in this area and discuss the future research potential surrounding these five questions. 
    more » « less
  3. ( , IISE Annual Conference Proceedings)
    A. Ghate, K. Krishnaiyer (Ed.)
    Deaths due to road traffic accidents are one of the leading causes of death in the United States. Furthermore, the economic impact of road traffic accidents accounts for about 3% of the United States' annual gross domestic product (GDP). In the past decade, extensive research has focused on autonomous vehicles (AVs). This technology is said to help prevent traffic accidents while promoting road traffic safety. This study aims to investigate the safety performance of AVs and identify the significant risk factors associated with the AV collisions. The study considers more than 200 crashes involving AVs and includes vehicle factors, environmental factors, collision type and crash severity. Multinomial logistic regression was conducted with collision type. The results showed no statistically significant risk factors to crash severity. However, movement preceding to collision contributes significantly to collision type. When both vehicles are moving, there's a higher likelihood of an angled collision, 47% to be exact. The other scenario which demonstrates a high probability of an angled collision is when the AV vehicle is not moving while the other is moving. The highest probability for a rear-end collision to occur is when the AV vehicle is not moving while the other is moving. This scenario makes up 55% of the entire rear-end collisions. As for the second-highest proportion, both vehicles moving, it consists of 42%. The research shall help reduce AV involved collisions and increase driving safety. 
    more » « less
  4. ; ( , Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    null (Ed.)
    Autonomous Vehicle (AV) technology has the potential to significantly improve driver safety. Unfortunately, driver could be reluctant to ride with AVs due to the lack of trust and acceptance of AV’s driving styles. The present study investigated the impact of driver’s driving style (aggressive/defensive) and the designed driving styles of AVs (aggressive/defensive) on driver’s trust, acceptance, and take-over behavior in fully autonomous vehicles. Thirty-two participants were classified into two groups based on their driving styles using the Aggressive Driving Scale and experienced twelve scenarios in either an aggressive AV or a defensive AV. Results revealed that drivers’ trust, acceptance, and takeover frequency were significantly influenced by the interaction effects between AV’s driving style and driver’s driving style. The findings implied that driver’s individual differences should be considered in the design of AV’s driving styles to enhance driver’s trust and acceptance of AVs and reduce undesired take over behaviors. 
    more » « less
  5. ; ( , IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR))
    After the 2017 TuSimple Lane Detection Challenge, its dataset and evaluation based on accuracy and F1 score have become the de facto standard to measure the performance of lane detection methods. While they have played a major role in improving the performance of lane detection methods, the validity of this evaluation method in downstream tasks has not been adequately researched. In this study, we design 2 new driving-oriented metrics for lane detection: End-to-End Lateral Deviation metric (E2E-LD) is directly formulated based on the requirements of autonomous driving, a core downstream task of lane detection; Per-frame Simulated Lateral Deviation metric (PSLD) is a lightweight surrogate metric of E2E-LD. To evaluate the validity of the metrics, we conduct a large-scale empirical study with 4 major types of lane detection approaches on the TuSimple dataset and our newly constructed dataset Comma2k19-LD. Our results show that the conventional metrics have strongly negative correlations (≤-0.55) with E2E-LD, meaning that some recent improvements purely targeting the conventional metrics may not have led to meaningful improvements in autonomous driving, but rather may actually have made it worse by overfitting to the conventional metrics. As autonomous driving is a security/safety-critical system, the underestimation of robustness hinders the sound development of practical lane detection models. We hope that our study will help the community achieve more downstream task-aware evaluations for lane detection. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email