More Like this

1. Benchmarking Deep Learning Interpretability in Time Series Predictions

   Ismail, Aya ; Gunady, Mohamed ; Bravo, Héctor ; Feizi, Soheil ( January 2020 , Advances in Neural Information Processing Systems Foundation (NeurIPS))

   null (Ed.)

   Saliency methods are used extensively to highlight the importance of input features in model predictions. These methods are mostly used in vision and language tasks, and their applications to time series data is relatively unexplored. In this paper, we set out to extensively compare the performance of various saliency-based interpretability methods across diverse neural architectures, including Recurrent Neural Network, Temporal Convolutional Networks, and Transformers in a new benchmark of synthetic time series data. We propose and report multiple metrics to empirically evaluate the performance of saliency methods for detecting feature importance over time using both precision (i.e., whether identified features contain meaningful signals) and recall (i.e., the number of features with signal identified as important). Through several experiments, we show that (i) in general, network architectures and saliency methods fail to reliably and accurately identify feature importance over time in time series data, (ii) this failure is mainly due to the conflation of time and feature domains, and (iii) the quality of saliency maps can be improved substantially by using our proposed two-step temporal saliency rescaling (TSR) approach that first calculates the importance of each time step before calculating the importance of each feature at a time step. 

   more » « less
2. Identifying Flare-indicative Photospheric Magnetic Field Parameters from Multivariate Time-series Data of Solar Active Regions

   https://doi.org/10.3847/1538-4365/ad21e4  

   Alshammari, Khaznah ; Hamdi, Shah Muhammad ; Filali Boubrahimi, Soukaina ( March 2024 , The Astrophysical Journal Supplement Series)

   Photospheric magnetic field parameters are frequently used to analyze and predict solar events. Observation of these parameters over time, i.e., representing solar events by multivariate time-series (MVTS) data, can determine relationships between magnetic field states in active regions and extreme solar events, e.g., solar flares. We can improve our understanding of these events by selecting the most relevant parameters that give the highest predictive performance. In this study, we propose a two-step incremental feature selection method for MVTS data using a deep-learning model based on long short-term memory (LSTM) networks. First, each MVTS feature (magnetic field parameter) is evaluated individually by a univariate sequence classifier utilizing an LSTM network. Then, the top performing features are combined to produce input for an LSTM-based multivariate sequence classifier. Finally, we tested the discrimination ability of the selected features by training downstream classifiers, e.g., Minimally Random Convolutional Kernel Transform and support vector machine. We performed our experiments using a benchmark data set for flare prediction known as Space Weather Analytics for Solar Flares. We compared our proposed method with three other baseline feature selection methods and demonstrated that our method selects more discriminatory features compared to other methods. Due to the imbalanced nature of the data, primarily caused by the rarity of minority flare classes (e.g., the X and M classes), we used the true skill statistic as the evaluation metric. Finally, we reported the set of photospheric magnetic field parameters that give the highest discrimination performance in predicting flare classes.

    

   more » « less
3. Network-based drug sensitivity prediction

   https://doi.org/10.1186/s12920-020-00829-3  

   Ahmed, Khandakar Tanvir ; Park, Sunho ; Jiang, Qibing ; Yeu, Yunku ; Hwang, TaeHyun ; Zhang, Wei ( December 2020 , BMC Medical Genomics)

   null (Ed.)

   Abstract Background Drug sensitivity prediction and drug responsive biomarker selection on high-throughput genomic data is a critical step in drug discovery. Many computational methods have been developed to serve this purpose including several deep neural network models. However, the modular relations among genomic features have been largely ignored in these methods. To overcome this limitation, the role of the gene co-expression network on drug sensitivity prediction is investigated in this study. Methods In this paper, we first introduce a network-based method to identify representative features for drug response prediction by using the gene co-expression network. Then, two graph-based neural network models are proposed and both models integrate gene network information directly into neural network for outcome prediction. Next, we present a large-scale comparative study among the proposed network-based methods, canonical prediction algorithms (i.e., Elastic Net, Random Forest, Partial Least Squares Regression, and Support Vector Regression), and deep neural network models for drug sensitivity prediction. All the source code and processed datasets in this study are available at https://github.com/compbiolabucf/drug-sensitivity-prediction . Results In the comparison of different feature selection methods and prediction methods on a non-small cell lung cancer (NSCLC) cell line RNA-seq gene expression dataset with 50 different drug treatments, we found that (1) the network-based feature selection method improves the prediction performance compared to Pearson correlation coefficients; (2) Random Forest outperforms all the other canonical prediction algorithms and deep neural network models; (3) the proposed graph-based neural network models show better prediction performance compared to deep neural network model; (4) the prediction performance is drug dependent and it may relate to the drug’s mechanism of action. Conclusions Network-based feature selection method and prediction models improve the performance of the drug response prediction. The relations between the genomic features are more robust and stable compared to the correlation between each individual genomic feature and the drug response in high dimension and low sample size genomic datasets. 

   more » « less
4. A New Attention Mechanism to Classify Multivariate Time Series

   https://doi.org/10.24963/ijcai.2020/277  

   Hao, Yifan ; Cao, Huiping ( July 2020 , Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence)

   Classifying multivariate time series (MTS), which record the values of multiple variables over a continuous period of time, has gained a lot of attention. However, existing techniques suffer from two major issues. First, the long-range dependencies of the time-series sequences are not well captured. Second, the interactions of multiple variables are generally not represented in features. To address these aforementioned issues, we propose a novel Cross Attention Stabilized Fully Convolutional Neural Network (CA-SFCN) to classify MTS data. First, we introduce a temporal attention mechanism to extract long- and short-term memories across all time steps. Second, variable attention is designed to select relevant variables at each time step. CA-SFCN is compared with 16 approaches using 14 different MTS datasets. The extensive experimental results show that the CA-SFCN outperforms state-of-the-art classification methods, and the cross attention mechanism achieves better performance than other attention mechanisms.

    

   more » « less
5. Comparison of neural network architectures for feature extraction from binary black hole merger waveforms

   https://doi.org/10.1088/2632-2153/ad2972  

   Gramaxo Freitas, Osvaldo ; Calderón Bustillo, Juan ; Font, José A. ; Nunes, Solange ; Onofre, Antonio ; Torres-Forné, Alejandro ( March 2024 , Machine Learning: Science and Technology)

   We evaluate several neural-network architectures, both convolutional and recurrent, for gravitational-wave time-series feature extraction by performing point parameter estimation on noisy waveforms from binary-black-hole mergers. We build datasets of 100 000 elements for each of four different waveform models (or approximants) in order to test how approximant choice affects feature extraction. Our choices includeSEOBNRv4PandIMRPhenomPv3, which contain only the dominant quadrupole emission mode, alongsideIMRPhenomPv3HMandNRHybSur3dq8, which also account for high-order modes. Each dataset element is injected into detector noise corresponding to the third observing run of the LIGO-Virgo-KAGRA (LVK) collaboration. We identify the temporal convolutional network architecture as the overall best performer in terms of training and validation losses and absence of overfitting to data. Comparison of results between datasets shows that the choice of waveform approximant for the creation of a dataset conditions the feature extraction ability of a trained network. Hence, care should be taken when building a dataset for the training of neural networks, as certain approximants may result in better network convergence of evaluation metrics. However, this performance does not necessarily translate to data which is more faithful to numerical relativity simulations. We also apply this network on actual signals from LVK runs, finding that its feature-extracting performance can be effective on real data.

    

   more » « less