More Like this

1. Robot Failure Mode Prediction with Explainable Machine Learning

   https://doi.org/10.1109/CASE48305.2020.9216965  

   Alvanpour, Aneseh; Das, Sumit Kumar; Robinson, Christopher Kevin; Nasraoui, Olfa; Popa, Dan (August 2020, IEEE CASE 2020)

   The ability to determine whether a robot's grasp has a high chance of failing, before it actually does, can save significant time and avoid failures by planning for re-grasping or changing the strategy for that special case. Machine Learning (ML) offers one way to learn to predict grasp failure from historic data consisting of a robot's attempted grasps alongside labels of the success or failure. Unfortunately, most powerful ML models are black-box models that do not explain the reasons behind their predictions. In this paper, we investigate how ML can be used to predict robot grasp failure and study the tradeoff between accuracy and interpretability by comparing interpretable (white box) ML models that are inherently explainable with more accurate black box ML models that are inherently opaque. Our results show that one does not necessarily have to compromise accuracy for interpretability if we use an explanation generation method, such as Shapley Additive explanations (SHAP), to add explainability to the accurate predictions made by black box models. An explanation of a predicted fault can lead to an efficient choice of corrective action in the robot's design that can be taken to avoid future failures. 

   more » « less
2. Expanding Human Response to Automation Failures to Sociotechnical Systems

   https://doi.org/10.1177/15553434241229124  

   Cooke, Nancy J (January 2024, Journal of Cognitive Engineering and Decision Making)

   Skraaning and Jamieson raise some interesting issues related to the response of humans to automation failures and offer a taxonomy of failure types that broadens its definition. In this commentary a further attempt to broaden the scope of automation failures is made that places failures within a sociotechnical system of multiple humans and multiple machine components including automation. A suggestion of how one might understand the system’s response to automation failures is offered and the inclusion of autonomy is raised as another complication. 

   more » « less
3. Predictive and Adaptive Failure Mitigation to Avert Production Cloud VM Interruptions

   Levy, Sebastien; Yao, Randolph; Wu, Youjiang; Dang, Yingnong; Huang, Peng; Mu, Zheng; Zhao, Pu; Ramani, Tarun; Govindraju, Naga; Li, Xukun; et al (November 2020, Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation)

   When a failure occurs in production systems, the highest priority is to quickly mitigate it. Despite its importance, failure mitigation is done in a reactive and ad-hoc way: taking some fixed actions only after a severe symptom is observed. For cloud systems, such a strategy is inadequate. In this paper, we propose a preventive and adaptive failure mitigation service, Narya, that is integrated in a production cloud, Microsoft Azure's compute platform. Narya predicts imminent host failures based on multi-layer system signals and then decides smart mitigation actions. The goal is to avert VM failures. Narya's decision engine takes a novel online experimentation approach to continually explore the best mitigation action. Narya further enhances the adaptive decision capability through reinforcement learning. Narya has been running in production for 15 months. It on average reduces VM interruptions by 26% compared to the previous static strategy. 

   more » « less
4. Automating Failure Testing Research at Internet Scale

   https://doi.org/10.1145/2987550.2987555  

   Alvaro, Peter; Andrus, Kolton; Sanden, Chris; Rosenthal, Casey; Basiri, Ali; Hochstein, Lorin (January 2016, Proceedings of the Seventh ACM Symposium on Cloud Computing)

   Large-scale distributed systems must be built to anticipate and mitigate a variety of hardware and software failures. In order to build confidence that fault-tolerant systems are correctly implemented, Netflix (and similar enterprises) regularly run failure drills in which faults are deliberately injected in their production system. The combinatorial space of failure scenarios is too large to explore exhaustively. Existing failure testing approaches either randomly explore the space of potential failures randomly or exploit the "hunches" of domain experts to guide the search. Random strategies waste resources testing "uninteresting" faults, while programmer-guided approaches are only as good as human intuition and only scale with human effort. In this paper, we describe how we adapted and implemented a research prototype called lineage-driven fault injection (LDFI) to automate failure testing at Netflix. Along the way, we describe the challenges that arose adapting the LDFI model to the complex and dynamic realities of the Netflix architecture. We show how we implemented the adapted algorithm as a service atop the existing tracing and fault injection infrastructure, and present early results. 

   more » « less
5. On robot grasp learning using equivariant models

   https://doi.org/10.1007/s10514-023-10112-w  

   Zhu, Xupeng; Wang, Dian; Su, Guanang; Biza, Ondrej; Walters, Robin; Platt, Robert (July 2023, Autonomous Robots)

   Abstract Real-world grasp detection is challenging due to the stochasticity in grasp dynamics and the noise in hardware. Ideally, the system would adapt to the real world by training directly on physical systems. However, this is generally difficult due to the large amount of training data required by most grasp learning models. In this paper, we note that the planar grasp function is $$\textrm{SE}(2)$$ -equivariant and demonstrate that this structure can be used to constrain the neural network used during learning. This creates an inductive bias that can significantly improve the sample efficiency of grasp learning and enable end-to-end training from scratch on a physical robot with as few as 600 grasp attempts. We call this method Symmetric Grasp learning (SymGrasp) and show that it can learn to grasp “from scratch” in less that 1.5 h of physical robot time. This paper represents an expanded and revised version of the conference paper Zhu et al. (2022). 

   more » « less