More Like this

1. Learning From Strategic Agents: Accuracy, Improvement, and Causality, ICML

   Shavit, Yonadav ; Edelman, Benjamin ; Axelrod, Brian ( July 2020 , Proceedings of Machine Learning Research)

   In many predictive decision-making scenarios, such as credit scoring and academic testing, a decision-maker must construct a model that accounts for agents' incentives to ``game'' their features in order to receive better decisions. Whereas the strategic classification literature generally assumes that agents' outcomes are not causally dependent on their features (and thus strategic behavior is a form of lying), we join concurrent work in modeling agents' outcomes as a function of their changeable attributes. Our formulation is the first to incorporate a crucial phenomenon: when agents act to change observable features, they may as a side effect perturb unobserved features that causally affect their true outcomes. We consider three distinct desiderata for a decision-maker's model: accurately predicting agents' post-gaming outcomes (accuracy), incentivizing agents to improve these outcomes (improvement), and, in the linear setting, estimating the visible coefficients of the true causal model (causal precision). As our main contribution, we provide the first algorithms for learning accuracy-optimizing, improvement-optimizing, and causal-precision-optimizing linear regression models directly from data, without prior knowledge of agents' possible actions. These algorithms circumvent the hardness result of Miller et al. (2019) by allowing the decision maker to observe agents' responses to a sequence of decision rules, in effect inducing agents to perform causal interventions for free. 

   more » « less
2. Decreasing False-alarm Rates in CNN-based Solar Flare Prediction Using SDO/HMI Data

   https://doi.org/10.3847/1538-4365/ac5b0c  

   Deshmukh, Varad ; Flyer, Natasha ; van der Sande, Kiera ; Berger, Thomas ( May 2022 , The Astrophysical Journal Supplement Series)

   A hybrid two-stage machine-learning architecture that addresses the problem of excessive false positives (false alarms) in solar flare prediction systems is investigated. The first stage is a convolutional neural network (CNN) model based on the VGG-16 architecture that extracts features from a temporal stack of consecutive Solar Dynamics Observatory Helioseismic and Magnetic Imager magnetogram images to produce a flaring probability. The probability of flaring is added to a feature vector derived from the magnetograms to train an extremely randomized trees (ERT) model in the second stage to produce a binary deterministic prediction (flare/no-flare) in a 12 hr forecast window. To tune the hyperparameters of the architecture, a new evaluation metric is introduced: the “scaled True Skill Statistic.” It specifically addresses the large discrepancy between the true positive rate and the false positive rate in the highly unbalanced solar flare event training data sets. Through hyperparameter tuning to maximize this new metric, our two-stage architecture drastically reduces false positives by ≈48% without significantly affecting the true positives (reduction by ≈12%), when compared with predictions from the first-stage CNN alone. This, in turn, improves various traditional binary classification metrics sensitive to false positives, such as the precision, F1, and the Heidke Skill Score. The end result is a more robust 12 hr flare prediction system that could be combined with current operational flare-forecasting methods. Additionally, using the ERT-based feature-ranking mechanism, we show that the CNN output probability is highly ranked in terms of flare prediction relevance.

    

   more » « less
3. BGWAS: Bayesian variable selection in linear mixed models with nonlocal priors for genome-wide association studies

   https://doi.org/10.1186/s12859-023-05316-x  

   Williams, Jacob ; Xu, Shuangshuang ; Ferreira, Marco A. R. ( May 2023 , BMC Bioinformatics)

   Genome-wide association studies (GWAS) seek to identify single nucleotide polymorphisms (SNPs) that cause observed phenotypes. However, with highly correlated SNPs, correlated observations, and the number of SNPs being two orders of magnitude larger than the number of observations, GWAS procedures often suffer from high false positive rates.

   We propose BGWAS, a novel Bayesian variable selection method based on nonlocal priors for linear mixed models specifically tailored for genome-wide association studies. Our proposed method BGWAS uses a novel nonlocal prior for linear mixed models (LMMs). BGWAS has two steps: screening and model selection. The screening step scans through all the SNPs fitting one LMM for each SNP and then uses Bayesian false discovery control to select a set of candidate SNPs. After that, a model selection step searches through the space of LMMs that may have any number of SNPs from the candidate set. A simulation study shows that, when compared to popular GWAS procedures, BGWAS greatly reduces false positives while maintaining the same ability to detect true positive SNPs. We show the utility and flexibility of BGWAS with two case studies: a case study on salt stress in plants, and a case study on alcohol use disorder.

   BGWAS maintains and in some cases increases the recall of true SNPs while drastically lowering the number of false positives compared to popular SMA procedures.

    

   more » « less
4. Interpreting Deep Learning-based Vulnerability Detector Predictions Based on Heuristic Searching

   https://doi.org/10.1145/3429444  

   Zou, Deqing ; Zhu, Yawei ; Xu, Shouhuai ; Li, Zhen ; Jin, Hai ; Ye, Hengkai ( March 2021 , ACM Transactions on Software Engineering and Methodology)

   null (Ed.)

   Detecting software vulnerabilities is an important problem and a recent development in tackling the problem is the use of deep learning models to detect software vulnerabilities. While effective, it is hard to explain why a deep learning model predicts a piece of code as vulnerable or not because of the black-box nature of deep learning models. Indeed, the interpretability of deep learning models is a daunting open problem. In this article, we make a significant step toward tackling the interpretability of deep learning model in vulnerability detection. Specifically, we introduce a high-fidelity explanation framework, which aims to identify a small number of tokens that make significant contributions to a detector’s prediction with respect to an example. Systematic experiments show that the framework indeed has a higher fidelity than existing methods, especially when features are not independent of each other (which often occurs in the real world). In particular, the framework can produce some vulnerability rules that can be understood by domain experts for accepting a detector’s outputs (i.e., true positives) or rejecting a detector’s outputs (i.e., false-positives and false-negatives). We also discuss limitations of the present study, which indicate interesting open problems for future research. 

   more » « less
5. Droplet encapsulation improves accuracy of immune cell cytokine capture assays

   https://doi.org/10.1039/c9lc01261c  

   Yuan, Yuan ; Brouchon, Julie ; Calvo-Calle, J. Mauricio ; Xia, Jing ; Sun, Li ; Zhang, Xu ; Clayton, Kiera L. ; Ye, Fangfu ; Weitz, David A. ; Heyman, John A. ( April 2020 , Lab on a Chip)

   null (Ed.)

   Quantification of cell-secreted molecules, e.g. , cytokines, is fundamental to the characterization of immune responses. Cytokine capture assays that use engineered antibodies to anchor the secreted molecules to the secreting cells are widely used to characterize immune responses because they allow both sensitive identification and recovery of viable responding cells. However, if the cytokines diffuse away from the secreting cells, non-secreting cells will also be identified as responding cells. Here we encapsulate immune cells in microfluidic droplets and perform in-droplet cytokine capture assays to limit the diffusion of the secreted cytokines. We use microfluidic devices to rapidly encapsulate single natural killer NK-92 MI cells and their target K562 cells into microfluidic droplets. We perform in-droplet IFN-γ capture assays and demonstrate that NK-92 MI cells recognize target cells within droplets and become activated to secrete IFN-γ. Droplet encapsulation prevents diffusion of secreted products to neighboring cells and dramatically reduces both false positives and false negatives, relative to assays performed without droplets. In a sample containing 1% true positives, encapsulation reduces, from 94% to 2%, the number of true-positive cells appearing as negatives; in a sample containing 50% true positives, the number of non-stimulated cells appearing as positives is reduced from 98% to 1%. After cells are released from the droplets, secreted cytokine remains captured onto secreting immune cells, enabling FACS-isolation of populations highly enriched for activated effector immune cells. Droplet encapsulation can be used to reduce background and improve detection of any single-cell secretion assay. 

   more » « less