skip to main content


Title: Certified Perception for Autonomous Cars
We present a method for establishing confidence in the decisions of an autonomous car which accounts for errors not only in control but also in perception. The key idea is that the controller generates a certificate, which is a kind its proposed action is safe. of proof that its interpretation of the scene is accurate and its proposed action is safe. Checking the certificate is faster and simpler than generating it, which allows for a monitor that comprises a much smaller trusted base than the system as a whole. Simulation experiments suggest that the approach is practical.  more » « less
Award ID(s):
1801399
NSF-PAR ID:
10272149
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
6th Workshop On Monitoring And Testing Of Cyber-Physical Systems,
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Certified control is a new architectural pattern for achieving high assurance of safety in autonomous cars. As with a traditional safety controller or interlock, a separate component oversees safety and intervenes to prevent safety violations. This component (along with sensors and actuators) comprises a trusted base that can ensure safety even if the main controller fails. But in certified control, the interlock does not use the sensors directly to determine when to intervene. Instead, the main controller is given the responsibility of presenting the interlock with a certificate that provides evidence that the proposed next action is safe. The interlock checks this certificate, and intervenes only if the check fails. Because generating such a certificate is usually much harder than checking one, the interlock can be smaller and simpler than the main controller, and thus assuring its correctness is more feasible. 
    more » « less
  2. null (Ed.)
    Often—for example in war games, strategy video games, and financial simulations—the game is given to us only as a black-box simulator in which we can play it. In these settings, since the game may have unknown nature action distributions (from which we can only obtain samples) and/or be too large to expand fully, it can be difficult to compute strategies with guarantees on exploitability. Recent work (Zhang and Sandholm 2020) resulted in a notion of certificate for extensive-form games that allows exploitability guarantees while not expanding the full game tree. However, that work assumed that the black box could sample or expand arbitrary nodes of the game tree at any time, and that a series of exact game solves (via, for example, linear programming) can be conducted to compute the certificate. Each of those two assumptions severely restricts the practical applicability of that method. In this work, we relax both of the assumptions. We show that high-probability certificates can be obtained with a black box that can do nothing more than play through games, using only a regret minimizer as a subroutine. As a bonus, we obtain an equilibrium-finding algorithm with ~O (1= p T) convergence rate in the extensive-form game setting that does not rely on a sampling strategy with lower-bounded reach probabilities (which MCCFR assumes). We demonstrate experimentally that, in the black-box setting, our methods are able to provide nontrivial exploitability guarantees while expanding only a small fraction of the game tree. 
    more » « less
  3. null (Ed.)
    A robustness certificate is the minimum distance of a given input to the decision boundary of the classifier (or its lower bound). For {\it any} input perturbations with a magnitude smaller than the certificate value, the classification output will provably remain unchanged. Exactly computing the robustness certificates for neural networks is difficult since it requires solving a non-convex optimization. In this paper, we provide computationally-efficient robustness certificates for neural networks with differentiable activation functions in two steps. First, we show that if the eigenvalues of the Hessian of the network are bounded, we can compute a robustness certificate in the l2 norm efficiently using convex optimization. Second, we derive a computationally-efficient differentiable upper bound on the curvature of a deep network. We also use the curvature bound as a regularization term during the training of the network to boost its certified robustness. Putting these results together leads to our proposed {\bf C}urvature-based {\bf R}obustness {\bf C}ertificate (CRC) and {\bf C}urvature-based {\bf R}obust {\bf T}raining (CRT). Our numerical results show that CRT leads to significantly higher certified robust accuracy compared to interval-bound propagation (IBP) based training. We achieve certified robust accuracy 69.79\%, 57.78\% and 53.19\% while IBP-based methods achieve 44.96\%, 44.74\% and 44.66\% on 2,3 and 4 layer networks respectively on the MNIST-dataset. 
    more » « less
  4. Synopsis

    Public health researchers have long been aware of the importance of defining the human community associated with research on environmental health initiatives. However, the field community’s human components where applied ecology research is conducted, e.g. diverse participants and perspectives, are often overlooked in environmental problem solving. We outline a framework for elevating the human dimension in defining the field community in applied ecology research and for teaching diverse undergraduate students the skills needed to address Anthropocene environmental concerns. We promote broadening participation and incorporating cultural and racial perspectives in ecology research planning, implementation, and teaching. We use the environmental research problem of concern to identify the diverse human community groups potentially connected to the problem and guide the strategies for incorporating their perspectives in the proposed research project. Which human community, whether local, ethnic, or visiting public community, affects the resource management strategy, i.e. people protect what they love, can change the outcomes of applied ecological research, as well as promote development of a diverse environmental workforce. Broadening participation and perspectives means that the people asking the research questions are also part of the social ecological community processes who choose which questions to pursue to manage the natural resources of the community. Here, we promote research and teaching practices that consider the long-standing multicultural connections to nature to allow all students to pursue their love of nature and its beauty in a safe, comfortable, and mentoring setting. We integrate current human diversity, equity, and inclusion-focused pedagogical knowledge into the Ecological Society of America-endorsed 4DEE multidimensional curricular framework. We provide a faculty action guide to engage and train diverse students in ecological practices that meet the needs of today’s environmental problem-solving workforce.

     
    more » « less
  5. Powerful domain-independent planners have been developed to solve various types of planning problems. These planners often require a model of the acting agent's actions, given in some planning domain description language. Yet obtaining such an action model is a notoriously hard task. This task is even more challenging in mission-critical domains, where a trial-and-error approach to learning how to act is not an option. In such domains, the action model used to generate plans must be safe, in the sense that plans generated with it must be applicable and achieve their goals. Learning safe action models for planning has been recently explored for domains in which states are sufficiently described with Boolean variables. In this work, we go beyond this limitation and propose the NSAM algorithm. NSAM runs in time that is polynomial in the number of observations and, under certain conditions, is guaranteed to return safe action models. We analyze its worst-case sample complexity, which may be intractable for some domains. Empirically, however, NSAM can quickly learn a safe action model that can solve most problems in the domain. 
    more » « less