Search for: All records

Existing work in continual learning (CL) focuses on mitigating catastrophic forgetting, i.e., model performance deterioration on past tasks when learning a new task. However, the training efficiency of a CL system is under-investigated, which limits the real-world application of CL systems under resource-limited scenarios. In this work, we propose a novel framework called Sparse Continual Learning(SparCL), which is the first study that leverages sparsity to enable cost-effective continual learning on edge devices. SparCL achieves both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity. Specifically, we propose task-aware dynamic masking (TDM) to learn a sparse network throughout the entire CL process, dynamic data removal (DDR) to remove less informative training data, and dynamic gradient masking (DGM) to sparsify the gradient updates. Each of them not only improves efficiency, but also further mitigates catastrophic forgetting. SparCL consistently improves the training efficiency of existing state-of-the-art (SOTA) CL methods by at most 23X less training FLOPs, and, surprisingly, further improves the SOTA accuracy by at most 1.7%. SparCL also outperforms competitive baselines obtained from adapting SOTA sparse training methods to the CL setting in both efficiency and accuracy. We also evaluate the effectiveness of SparCL on a real mobile phone, further indicating the practical potential of our method. 

The El Nino Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to describe a complex phenomenon and translating DL-based ENSO predictions to river flow predictions. Here we show that eXplainable DL (XDL) methods, based on saliency maps, can extract interpretable predictive information contained in global SST and discover novel SST information regions and dependence structures relevant for river flows which, in tandem with climate network constructions, enable improved predictive understanding. Our results reveal additional information content in global SST beyond ENSO indices, develop new understanding of how SSTs influence river flows, and generate improved river flow predictions with uncertainties. Observations, reanalysis data, and earth system model simulations are used to demonstrate the value of the XDL-CN based methods for future interannual and decadal scale climate projections. 

Abstract Degeneracy in biological systems refers to a many-to-one mapping between physical structures and their functional (including psychological) outcomes. Despite the ubiquity of the phenomenon, traditional analytical tools for modeling degeneracy in neuroscience are extremely limited. In this study, we generated synthetic datasets to describe three situations of degeneracy in fMRI data to demonstrate the limitations of the current univariate approach. We describe a novel computational approach for the analysis referred to as neural topographic factor analysis (NTFA). NTFA is designed to capture variations in neural activity across task conditions and participants. The advantage of this discovery-oriented approach is to reveal whether and how experimental trials and participants cluster into task conditions and participant groups. We applied NTFA on simulated data, revealing the appropriate degeneracy assumption in all three situations and demonstrating NTFA’s utility in uncovering degeneracy. Lastly, we discussed the importance of testing degeneracy in fMRI data and the implications of applying NTFA to do so. 

Rapid and accurate detection of the pathogens, such as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) for COVID-19, is critical for mitigating the COVID-19 pandemic. Current state-of-the-art pathogen tests for COVID-19 diagnosis are done in a liquid medium and take 10–30 min for rapid antigen tests and hours to days for polymerase chain reaction (PCR) tests. Herein we report novel accurate pathogen sensors, a new test method, and machine-learning algorithms for a breathalyzer platform for fast detection of SARS-CoV-2 virion particles in the aerosol in 30 s. The pathogen sensors are based on a functionalized molecularly-imprinted polymer, with the template molecules being the receptor binding domain spike proteins for different variants of SARS-CoV-2. Sensors are tested in the air and exposed for 10 s to the aerosols of various types of pathogens, including wild-type, D614G, alpha, delta, and omicron variant SARS-CoV-2, BSA (Bovine serum albumin), Middle East respiratory syndrome–related coronavirus (MERS-CoV), influenza, and wastewater samples from local sewage. Our low-cost, fast-responsive pathogen sensors yield accuracy above 99% with a limit-of-detection (LOD) better than 1 copy/μL for detecting the SARS-CoV-2 virus from the aerosol. The machine-learning algorithm supporting these sensors can accurately detect the pathogens, thereby enabling a new and unique breathalyzer platform for rapid COVID-19 tests with unprecedented speeds.