More Like this

1. Making the Black Box More Transparent: Understanding the Physical Implications of Machine Learning

   https://doi.org/10.1175/BAMS-D-18-0195.1  

   McGovern, Amy; Lagerquist, Ryan; John Gagne, David; Jergensen, G. Eli; Elmore, Kimberly L.; Homeyer, Cameron R.; Smith, Travis (November 2019, Bulletin of the American Meteorological Society)

   This paper synthesizes multiple methods for machine learning (ML) model interpretation and visualization (MIV) focusing on meteorological applications. ML has recently exploded in popularity in many fields, including meteorology. Although ML has been successful in meteorology, it has not been as widely accepted, primarily due to the perception that ML models are “black boxes,” meaning the ML methods are thought to take inputs and provide outputs but not to yield physically interpretable information to the user. This paper introduces and demonstrates multiple MIV techniques for both traditional ML and deep learning, to enable meteorologists to understand what ML models have learned. We discuss permutation-based predictor importance, forward and backward selection, saliency maps, class-activation maps, backward optimization, and novelty detection. We apply these methods at multiple spatiotemporal scales to tornado, hail, winter precipitation type, and convective-storm mode. By analyzing such a wide variety of applications, we intend for this work to demystify the black box of ML, offer insight in applying MIV techniques, and serve as a MIV toolbox for meteorologists and other physical scientists. 

   more » « less
2. In-Training Explainability Frameworks: A Method to Make Black-Box Machine Learning Models More Explainable

   https://doi.org/10.1109/WI-IAT59888.2023.00036  

   Acun, Cagla; Nasraoui, Olfa (October 2023, IEEE)
3. Black-Box Garbled RAM

   Garg, Sanjam; Lu, Steve; Ostrovsky, Rafail (January 2015, Annual Symposium on Foundations of Computer Science)
4. Simple Black-box Adversarial Attacks

   Guo, Chuan; Gardner, Jacob; You, Yurong; Wilson, Andrew; Weinberger, Kilian (August 2019, Proceedings of Machine Learning Research)

   We propose an intriguingly simple method for the construction of adversarial images in the black-box setting. In constrast to the white-box scenario, constructing black-box adversarial im- ages has the additional constraint on query bud- get, and efficient attacks remain an open prob- lem to date. With only the mild assumption of continuous-valued confidence scores, our highly query-efficient algorithm utilizes the following simple iterative principle: we randomly sample a vector from a predefined orthonormal basis and either add or subtract it to the target image. De- spite its simplicity, the proposed method can be used for both untargeted and targeted attacks – resulting in previously unprecedented query effi- ciency in both settings. We demonstrate the effi- cacy and efficiency of our algorithm on several real world settings including the Google Cloud Vision API. We argue that our proposed algorithm should serve as a strong baseline for future black- box attacks, in particular because it is extremely fast and its implementation requires less than 20 lines of PyTorch code. 

   more » « less
5. On Black-Box Verifiable Outsourcing

   Agarwal, Amit; Alamati, Navid; Khurana, Dakshita; Raghuraman, Srinivasan; Rindal, Peter (November 2023, TCC 2023)

   We study verifiable outsourcing of computation in a model where the verifier has black-box access to the function being computed. We introduce the problem of oracle-aided batch verification of computation (OBVC) for a function class $$\mathcal{F}$$. This allows a verifier to efficiently verify the correctness of any $$f \in \mathcal{F}$$ evaluated on a batch of $$n$$ instances $$x_1, \ldots, x_n$$, while only making $$\lambda$$ calls to an oracle for $$f$$ (along with $$O(n \lambda)$$ calls to low-complexity helper oracles), for security parameter $$\lambda$$. We obtain the following positive and negative results: - We build OBVC protocols for the class of all functions that admit {\em random-self-reductions}. Some of our protocols rely on homomorphic encryption schemes. - We show that there cannot exist OBVC schemes for the class of all functions mapping $$\lambda$$-bit inputs to $$\lambda$$-bit outputs, for any $$n = \mathsf{poly}(\lambda)$$. 

   more » « less