More Like this

1. Model, Data and Reward Repair: Trusted Machine Learning for Markov Decision Processes

   https://doi.org/10.1109/DSN-W.2018.00064  

   Ghosh, Shalini; Jha, Susmit; Tiwari, Ashish; Lincoln, Patrick; Zhu, Xiaojin (January 2018, 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W))

   When machine learning (ML) algorithms are used in mission-critical domains (e.g., self-driving cars, cyber security) or life-critical domains (e.g., surgical robotics), it is often important to ensure that the learned models satisfy some high-level correctness requirements — these requirements can be instantiated in particular domains via constraints like safety (e.g., a robot arm should not come within five meters of any human operator during any phase of performing an autonomous operation) or liveness (e.g., a car should eventually cross a 4-way intersection). Such constraints can be formally described in propositional logic, first order logic or temporal logics such as Probabilistic Computation Tree Logic (PCTL)[31]. For example, in a lane change controller we can enforce the following PCTL safety property on seeing a slow-moving truck in front: Pr>0.99[F(changedLane or reducedSpeed)] , where F is the eventually operator in PCTL logic — this property states that the car should eventually change lanes or reduce speed with high probability (greater than 0.99). Trusted Machine Learning (TML) refers to a learning methodology that ensures that the specified properties are satisfied. 

   more » « less
2. Analysis of Driver Behavior in Mixed Autonomous and Non-autonomous Traffic Flows

   https://doi.org/10.1177/1071181322661305  

   Liu, Jundi; Boyle, Linda Ng (September 2022, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)

   Autonomous vehicles are expected to improve road safety and efficiency in future transportation systems. A driving simulator study was designed to identify driving styles and the cooperation between human drivers and other AVs. The study captured driver’s following behavior in a fully autonomous driving environment at unsignalized intersections. Participants were asked to make a series of maneuvers (straight through intersection, left turn, and right turn) in two different speed conditions (30, 40 mph) and two different traffic density conditions (with or without other traffic). Analysis of Variance showed that drivers had a significantly larger deviation (defined as the area between two trajectories) during left turn maneuvers when they were traveling at higher speeds. Moreover, the first turning operation had smaller deviation than the second turning operation. The findings have implications for the design of driver-assistance guidance systems in future mixed autonomous and non-autonomous traffic flows. 

   more » « less
3. VPP: The Vulnerability-Proportional Protection Paradigm Towards Reliable Autonomous Machines

   Zishen, Wan; Yiming, Gan; Bo, Yu3; Arijit, Raychowdhury; Yuhao, Zhu (June 2023, workshop)

   The next ubiquitous computing platform, after personal computers and smartphones, is likely one of the autonomous natures, such as drones, robots, and self-driving cars, which have moved from mere lab concepts to permeating almost every aspect of our soci- ety [16, 20, 25]. Behind the proliferation of autonomous machines is the critical need to ensure reliability [7, 22–24]. Almost every vendor, be it in the software, hardware, or systems segment, has to conform to functional safety standards when shipping products for automotives. Today’s resiliency solutions to autonomous machines, however, all make fundamental trade-offs between resiliency and cost, which manifests as high overhead in performance, energy, and chip area. For instance, hardware modular redundancy provides high safety but more than doubles the area and energy cost [1]. The reason is that today’s solutions are of the “one-size-fits-all” nature: they use the same protection scheme throughout the entire computing stack of autonomous machines. As a result, they have to accommodate the least robust component, leading to a high protection overhead. The insight of this paper is that for a resiliency solution to pro- vide high protection coverage while introducing little cost, we must exploit the inherent robustness variations in the domain-specific autonomous machine computing. In particular, we show that the different autonomous machine kernels differ significantly in their inherent robustness and performance. Building on top of that, we propose a Vulnerable-Proportional Protection (VPP) design paradigm, in which the protection budget, be it spatially (e.g., modular re- dundancy) or temporally (e.g., re-execution), should be inversely proportional to the inherent robustness of a task in the autonomous machine system. In stark contrast to the existing “one-size-fits-all” strategy, VPP wisely allocates the protection budget, thus achieving the same protection coverage with little overhead, which provides a blueprint design paradigm towards reliable autonomous machines 

   more » « less
4. Concurrent Error Detection in Embedded Digital Control of Nonlinear Autonomous Systems Using Adaptive State Space Checks

   https://doi.org/10.1109/ITC44778.2020.9325229  

   Momtaz, Md Imran; Amarnath, Chandramouli N; Chatterjee, Abhijit (November 2020, International Test Conference)

   null (Ed.)

   The advent of pervasive autonomous systems such as self-driving cars and drones has raised questions about their safety and trustworthiness. This is particularly relevant in the event of on-board subsystem errors or failures. In this research, we show how encoded Extended Kalman Filter can be used to detect anomalous behaviors of critical components of nonlinear autonomous systems: sensors, actuators, state estimation algorithms and control software. As opposed to prior work that is limited to linear systems or requires the use of cumbersome machine learned checks with fixed detection thresholds, the proposed approach necessitates the use of time-varying checks with dynamically adaptive thresholds. The method is lightweight in comparison to existing methods (does not rely on machine learning paradigms) and achieves high coverage as well as low detection latency of errors. A quadcopter and an automotive steer-by-wire system are used as test vehicles for the research and simulation and hardware results indicate the overhead, coverage and error detection latency benefits of the proposed approach. 

   more » « less
5. Misconfiguration Software Testing for Failure Emergence in Autonomous Driving Systems

   https://doi.org/10.1145/3660792  

   Chen, Yuntianyi; Huai, Yuqi; Li, Shilong; Hong, Changnam; Garcia, Joshua (July 2024, Proceedings of the ACM on Software Engineering)

   The optimization of a system’s configuration options is crucial for determining its performance and functionality, particularly in the case of autonomous driving software (ADS) systems because they possess a multitude of such options. Research efforts in the domain of ADS have prioritized the development of automated testing methods to enhance the safety and security of self-driving cars. Presently, search-based approaches are utilized to test ADS systems in a virtual environment, thereby simulating real-world scenarios. However, such approaches rely on optimizing the waypoints of ego cars and obstacles to generate diverse scenarios that trigger violations, and no prior techniques focus on optimizing the ADS from the perspective of configuration. To address this challenge, we present a framework called ConfVE, which is the first automated configuration testing framework for ADSes. ConfVE’s design focuses on the emergence of violations through rerunning scenarios generated by different ADS testing approaches under different configurations, leveraging 9 test oracles to enable previous ADS testing approaches to find more types of violations without modifying their designs or implementations and employing a novel technique to identify bug-revealing violations and eliminate duplicate violations. Our evaluation results demonstrate that ConfVE can discover 1,818 unique violations and reduce 74.19% of duplicate violations. 

   more » « less