An official website of the United States government
Abstract MotivationMillions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. ResultsWe developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementationThe source code of TransFun is available at https://github.com/jianlin-cheng/TransFun.
more »
« less
Accelerate microstructure evolution simulation using graph neural networks with adaptive spatiotemporal resolution
Abstract Surrogate models driven by sizeable datasets and scientific machine-learning methods have emerged as an attractive microstructure simulation tool with the potential to deliver predictive microstructure evolution dynamics with huge savings in computational costs. Taking 2D and 3D grain growth simulations as an example, we present a completely overhauled computational framework based on graph neural networks with not only excellent agreement to both the ground truth phase-field methods and theoretical predictions, but enhanced accuracy and efficiency compared to previous works based on convolutional neural networks. These improvements can be attributed to the graph representation, both improved predictive power and a more flexible data structure amenable to adaptive mesh refinement. As the simulated microstructures coarsen, our method can adaptively adopt remeshed grids and larger timesteps to achieve further speedup. The data-to-model pipeline with training procedures together with the source codes are provided.
more »
« less
- Award ID(s):
- 1929949
- PAR ID:
- 10504733
- Publisher / Repository:
- IOP Publishing
- Date Published:
- Journal Name:
- Machine Learning: Science and Technology
- Volume:
- 5
- Issue:
- 2
- ISSN:
- 2632-2153
- Format(s):
- Medium: X Size: Article No. 025027
- Size(s):
- Article No. 025027
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Neural networks are often represented as graphs of connections between neurons. However, despite their wide use, there is currently little understanding of the relationship between the graph structure of the neural network and its predictive performance. Here we systematically investigate how does the graph structure of neural networks affect their predictive performance. To this end, we develop a novel graph-based representation of neural networks called relational graph, where layers of neural network computation correspond to rounds of message exchange along the graph structure. Using this representation we show that: (1) a “sweet spot” of relational graphs leads to neural networks with significantly improved predictive performance; (2) neural network’s performance is approximately a smooth function of the clustering coefficient and average path length of its relational graph; (3) our findings are consistent across many different tasks and datasets; (4) the sweet spot can be identified efficiently; (5) topperforming neural networks have graph structure surprisingly similar to those of real biological neural networks. Our work opens new directions for the design of neural architectures and the understanding on neural networks in general.more » « less
-
Abstract Modern data mining methods have demonstrated effectiveness in comprehending and predicting materials properties. An essential component in the process of materials discovery is to know which material(s) will possess desirable properties. For many materials properties, performing experiments and density functional theory computations are costly and time-consuming. Hence, it is challenging to build accurate predictive models for such properties using conventional data mining methods due to the small amount of available data. Here we present a framework for materials property prediction tasks using structure information that leverages graph neural network-based architecture along with deep-transfer-learning techniques to drastically improve the model’s predictive ability on diverse materials (3D/2D, inorganic/organic, computational/experimental) data. We evaluated the proposed framework in cross-property and cross-materials class scenarios using 115 datasets to find that transfer learning models outperform the models trained from scratch in 104 cases, i.e., ≈90%, with additional benefits in performance for extrapolation problems. We believe the proposed framework can be widely useful in accelerating materials discovery in materials science.more » « less
-
Deep learning has emerged as a promising paradigm to give access to highly accurate predictions of molecular and material properties. A common short-coming shared by current approaches, however, is that neural networks only give point estimates of their predictions and do not come with predictive uncertainties associated with these estimates. Existing uncertainty quantification efforts have primarily leveraged the standard deviation of predictions across an ensemble of independently trained neural networks. This incurs a large computational overhead in both training and prediction, resulting in order-of-magnitude more expensive predictions. Here, we propose a method to estimate the predictive uncertainty based on a single neural network without the need for an ensemble. This allows us to obtain uncertainty estimates with virtually no additional computational overhead over standard training and inference. We demonstrate that the quality of the uncertainty estimates matches those obtained from deep ensembles. We further examine the uncertainty estimates of our methods and deep ensembles across the configuration space of our test system and compare the uncertainties to the potential energy surface. Finally, we study the efficacy of the method in an active learning setting and find the results to match an ensemble-based strategy at order-of-magnitude reduced computational cost.more » « less
-
Abstract The deformation behavior of Ti-6Al-4V titanium alloy is significantly influenced by slip localized within crystallographic slip bands. Experimental observations reveal that intense slip bands in Ti-6Al-4V form at strains well below the macroscopic yield strain and may serially propagate across grain boundaries, resulting in long-range localization that percolates through the microstructure. These connected, localized slip bands serve as potential sites for crack initiation. Although slip localization in Ti-6Al-4V is known to be influenced by various factors, an investigation of optimal microstructures that limit localization remains lacking. In this work, we develop a novel strategy that integrates an explicit slip band crystal plasticity technique, graph networks, and neural network models to identify Ti-6Al-4V microstructures that reduce the propensity for strain localization. Simulations are conducted on a dataset of 3D polycrystals, each represented as a graph to account for grain neighborhood and connectivity. The results are then used to train neural network surrogate models that accurately predict localization-based properties of a polycrystal, given its microstructure. These properties include the ratio of slip accumulated in the band to that in the matrix, fraction of total applied strain accommodated by slip bands, and spatial connectivity of slip bands throughout the microstructure. The initial dataset is enriched by synthetic data generated by the surrogate models, and a grid search optimization is subsequently performed to find optimal microstructures. Describing a 3D polycrystal with only a few features and a combination of graph and neural network models offer robustness compared to the alternative approaches without compromising accuracy. We show that while each material property is optimized through a unique microstructure solution, elongated grain shape emerges as a recurring feature among all optimal microstructures. This finding suggests that designing microstructures with elongated grains could potentially mitigate strain localization without compromising strength.more » « less
