More Like this

1. Federated learning with matched averaging

   Wang, Hongyi; Yurochkin, Mikhail; Sun, Yuekai; Papailiopoulos, Dimitris; Khazaeni, Yasaman (April 2020, International Conference on Learning Representations (ICLR))

   Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose the Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, but also reduces the overall communication burden. 

   more » « less
2. Federated Learning with Matched Averaging

   Wang, Hongyi; Yurochkin, Mikhail; Sun, Yuekai; Papailiopoulos, Dimitris; Khazaeni, Yasaman (April 2020, International Conference on Learning Representations)

   Federated learning allows edge devices to collaboratively learn a shared model while keeping the training data on device, decoupling the ability to do model training from the need to store the data in the cloud. We propose the Federated matched averaging (FedMA) algorithm designed for federated learning of modern neural network architectures e.g. convolutional neural networks (CNNs) and LSTMs. FedMA constructs the shared global model in a layer-wise manner by matching and averaging hidden elements (i.e. channels for convolution layers; hidden states for LSTM; neurons for fully connected layers) with similar feature extraction signatures. Our experiments indicate that FedMA not only outperforms popular state-of-the-art federated learning algorithms on deep CNN and LSTM architectures trained on real world datasets, but also reduces the overall communication burden. 

   more » « less
3. Exploring a global interpretation mechanism for deep learning networks when predicting sepsis

   https://doi.org/10.1038/s41598-023-30091-3  

   Strickler, Ethan A.; Thomas, Joshua; Thomas, Johnson P.; Benjamin, Bruce; Shamsuddin, Rittika (December 2023, Scientific Reports)

   Abstract The purpose of this study is to identify additional clinical features for sepsis detection through the use of a novel mechanism for interpreting black-box machine learning models trained and to provide a suitable evaluation for the mechanism. We use the publicly available dataset from the 2019 PhysioNet Challenge. It has around 40,000 Intensive Care Unit (ICU) patients with 40 physiological variables. Using Long Short-Term Memory (LSTM) as the representative black-box machine learning model, we adapted the Multi-set Classifier to globally interpret the black-box model for concepts it learned about sepsis. To identify relevant features, the result is compared against: (i) features used by a computational sepsis expert, (ii) clinical features from clinical collaborators, (iii) academic features from literature, and (iv) significant features from statistical hypothesis testing. Random Forest was found to be the computational sepsis expert because it had high accuracies for solving both the detection and early detection, and a high degree of overlap with clinical and literature features. Using the proposed interpretation mechanism and the dataset, we identified 17 features that the LSTM used for sepsis classification, 11 of which overlaps with the top 20 features from the Random Forest model, 10 with academic features and 5 with clinical features. Clinical opinion suggests, 3 LSTM features have strong correlation with some clinical features that were not identified by the mechanism. We also found that age, chloride ion concentration, pH and oxygen saturation should be investigated further for connection with developing sepsis. Interpretation mechanisms can bolster the incorporation of state-of-the-art machine learning models into clinical decision support systems, and might help clinicians to address the issue of early sepsis detection. The promising results from this study warrants further investigation into creation of new and improvement of existing interpretation mechanisms for black-box models, and into clinical features that are currently not used in clinical assessment of sepsis. 

   more » « less
4. Multi-Shared-Task Self-Supervised CNN-LSTM for Monitoring Free-Body Movement UPDRS-III Using Wearable Sensors

   https://doi.org/10.3390/bioengineering11070689  

   Shuqair, Mustafa; Jimenez-Shahed, Joohi; Ghoraani, Behnaz (July 2024, Bioengineering)

   The Unified Parkinson’s Disease Rating Scale (UPDRS) is used to recognize patients with Parkinson’s disease (PD) and rate its severity. The rating is crucial for disease progression monitoring and treatment adjustment. This study aims to advance the capabilities of PD management by developing an innovative framework that integrates deep learning with wearable sensor technology to enhance the precision of UPDRS assessments. We introduce a series of deep learning models to estimate UPDRS Part III scores, utilizing motion data from wearable sensors. Our approach leverages a novel Multi-shared-task Self-supervised Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) framework that processes raw gyroscope signals and their spectrogram representations. This technique aims to refine the estimation accuracy of PD severity during naturalistic human activities. Utilizing 526 min of data from 24 PD patients engaged in everyday activities, our methodology demonstrates a strong correlation of 0.89 between estimated and clinically assessed UPDRS-III scores. This model outperforms the benchmark set by single and multichannel CNN, LSTM, and CNN-LSTM models and establishes a new standard in UPDRS-III score estimation for free-body movements compared to recent state-of-the-art methods. These results signify a substantial step forward in bioengineering applications for PD monitoring, providing a robust framework for reliable and continuous assessment of PD symptoms in daily living settings. 

   more » « less
5. Effect of Sampling Rate on ECG Deep Learning Classifiers

   https://doi.org/10.1109/ICHI64645.2025.00043  

   Dakshit, Sristi; Dakshit, Sagnik; Prabhakaran, Balakrishnan (June 2025, IEEE)

   With recent advances in Deep Learning (DL) models, the healthcare domain has seen an increased adoption of neural networks for clinical diagnosis, monitoring, and prediction. Deep Learning models have been developed for various tasks using 1D (one-dimensional) time-series signals. Time-series healthcare data, typically collected through sensors, have specific structures and characteristics such as frequency and amplitude. The nature of these features, including varying sampling rates that depend on the instruments used for sensing, poses challenges in handling them. Electrocardiograms (ECG), a class of 1D time-series signals representing the electrical activity of the heart, have been used to develop heart condition classification decision support systems. The sampling rate of these signals, influenced by different ECG instruments as well as their calibrations, can greatly impact the learning functions of deep learning models and subsequently, their decision outcomes. This hinders the development and deployment of generalized, DL-based ECG classifiers that can work with data from a variety of ECG instruments, particularly when the sampling rate of the training data remains unknown to users. Moreover, DL models are not designed to recognize the sampling rate of the testing data on which they are being deployed, further complicating their effective application across diverse clinical settings. In this study, we investigated the effect of different sampling rates of time-series ECG signals on DL-based ECG classifiers. To the best of our knowledge, this is the first work to understand how varying sampling rates affect the performance of DL-based models for classifying 1D time-series ECG signals. Through our comprehensive experiments, we showed that accuracy can drop by as much as 20% when the training and testing sampling rates are different. We provide visual explanations to understand the differences in learned model features through activation maps when the sampling rates for training and testing data are different. We also investigated potential strategies to address the challenges posed by different sampling rates: (i) transfer learning, (ii) resampling, and (iii) training a DL model using ECG data at different sampling rates. 

   more » « less