An official website of the United States government
Abstract Hybrid Knowledge‐Guided Machine Learning (KGML) models, which are deep learning models that utilize scientific theory and process‐based model simulations, have shown improved performance over their process‐based counterparts for the simulation of water temperature and hydrodynamics. We highlight the modular compositional learning (MCL) methodology as a novel design choice for the development of hybrid KGML models in which the model is decomposed into modular sub‐components that can be process‐based models and/or deep learning models. We develop a hybrid MCL model that integrates a deep learning model into a modularized, process‐based model. To achieve this, we first train individual deep learning models with the output of the process‐based models. In a second step, we fine‐tune one deep learning model with observed field data. In this study, we replaced process‐based calculations of vertical diffusive transport with deep learning. Finally, this fine‐tuned deep learning model is integrated into the process‐based model, creating the hybrid MCL model with improved overall projections for water temperature dynamics compared to the original process‐based model. We further compare the performance of the hybrid MCL model with the process‐based model and two alternative deep learning models and highlight how the hybrid MCL model has the best performance for projecting water temperature, Schmidt stability, buoyancy frequency, and depths of different isotherms. Modular compositional learning can be applied to existing modularized, process‐based model structures to make the projections more robust and improve model performance by letting deep learning estimate uncertain process calculations.
more »
« less
This content will become publicly available on January 1, 2026
Benchmarking study of deep generative models for inverse polymer design
This benchmark study evaluates deep learning-based molecular generative models on various polymer datasets. Selected models were further refined with reinforcement learning to generate hypothetical heat-resistant polymers.
more »
« less
- Award ID(s):
- 2332276
- PAR ID:
- 10571090
- Publisher / Repository:
- RSC
- Date Published:
- Journal Name:
- Digital Discovery
- ISSN:
- 2635-098X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This work compares various existing rough-wall models on a large collection of rough surfaces with different characteristics and studies the potential of these models in accommodating new datasets. We consider three empirical roughness correlations, two physics-based models, and one data-driven machine-learning model on 68 rough surfaces inside and outside the Roughness Database1. Results show that correlation-type models and machine-learning models do not extrapolate outside the dataset against which they are calibrated or trained. In contrast, the physics-based sheltering model performs well in extrapolation. Recalibrating a roughness correlation against a large dataset proves unfruitful. However, retraining a machine learning model yields good results. We do not pursue further retraining and recalibrating of a physics-based model, as it requires new physical insights. Overall, our findings suggest that a universal rough-wall model is yet to be found. The capability of extrapolation will likely come from incorporating physics. Data, on the other hand, benefits machine learning models.more » « less
-
Abstract Backpropagation is widely used to train artificial neural networks, but its relationship to synaptic plasticity in the brain is unknown. Some biological models of backpropagation rely on feedback projections that are symmetric with feedforward connections, but experiments do not corroborate the existence of such symmetric backward connectivity. Random feedback alignment offers an alternative model in which errors are propagated backward through fixed, random backward connections. This approach successfully trains shallow models, but learns slowly and does not perform well with deeper models or online learning. In this study, we develop a meta-learning approach to discover interpretable, biologically plausible plasticity rules that improve online learning performance with fixed random feedback connections. The resulting plasticity rules show improved online training of deep models in the low data regime. Our results highlight the potential of meta-learning to discover effective, interpretable learning rules satisfying biological constraints.more » « less
-
The growing number of AI-driven applications in mobile devices has led to solutions that integrate deep learning models with the available edge-cloud resources. Due to multiple benefits such as reduction in on-device energy consumption, improved latency, improved network usage, and certain privacy improvements, split learning, where deep learning models are split away from the mobile device and computed in a distributed manner, has become an extensively explored topic. Incorporating compression-aware methods (where learning adapts to compression level of the communicated data) has made split learning even more advantageous. This method could even offer a viable alternative to traditional methods, such as federated learning techniques. In this work, we develop an adaptive compression-aware split learning method (“deprune”) to improve and train deep learning models so that they are much more network-efficient, which would make them ideal to deploy in weaker devices with the help of edge-cloud resources. This method is also extended (“prune”) to very quickly train deep learning models through a transfer learning approach, which tradesoff little accuracy for much more network-efficient inference abilities. We show that the “deprune” method can reduce network usage by 4× when compared with a split-learning approach (that does not use our method) without loss of accuracy, while also improving accuracy over compression-aware split-learning by up to 4 percent. Lastly, we show that the “prune” method can reduce the training time for certain models by up to 6× without affecting the accuracy when compared against a compression-aware split-learning approach.more » « less
-
BackgroundClinical prediction models suffer from performance drift as the patient population shifts over time. There is a great need for model updating approaches or modeling frameworks that can effectively use the old and new data. ObjectiveBased on the paradigm of transfer learning, we aimed to develop a novel modeling framework that transfers old knowledge to the new environment for prediction tasks, and contributes to performance drift correction. MethodsThe proposed predictive modeling framework maintains a logistic regression–based stacking ensemble of 2 gradient boosting machine (GBM) models representing old and new knowledge learned from old and new data, respectively (referred to as transfer learning gradient boosting machine [TransferGBM]). The ensemble learning procedure can dynamically balance the old and new knowledge. Using 2010-2017 electronic health record data on a retrospective cohort of 141,696 patients, we validated TransferGBM for hospital-acquired acute kidney injury prediction. ResultsThe baseline models (ie, transported models) that were trained on 2010 and 2011 data showed significant performance drift in the temporal validation with 2012-2017 data. Refitting these models using updated samples resulted in performance gains in nearly all cases. The proposed TransferGBM model succeeded in achieving uniformly better performance than the refitted models. ConclusionsUnder the scenario of population shift, incorporating new knowledge while preserving old knowledge is essential for maintaining stable performance. Transfer learning combined with stacking ensemble learning can help achieve a balance of old and new knowledge in a flexible and adaptive way, even in the case of insufficient new data.more » « less
