More Like this

1. In the twilight zone of protein sequence homology: do protein language models learn protein structure?

   https://doi.org/10.1093/bioadv/vbae119  

   Kabir, Anowarul; Moldwin, Asher; Bromberg, Yana; Shehu, Amarda (January 2024, Bioinformatics Advances)

   Gogovi, Gideon (Ed.)

   Abstract MotivationProtein language models based on the transformer architecture are increasingly improving performance on protein prediction tasks, including secondary structure, subcellular localization, and more. Despite being trained only on protein sequences, protein language models appear to implicitly learn protein structure. This paper investigates whether sequence representations learned by protein language models encode structural information and to what extent. ResultsWe address this by evaluating protein language models on remote homology prediction, where identifying remote homologs from sequence information alone requires structural knowledge, especially in the “twilight zone” of very low sequence identity. Through rigorous testing at progressively lower sequence identities, we profile the performance of protein language models ranging from millions to billions of parameters in a zero-shot setting. Our findings indicate that while transformer-based protein language models outperform traditional sequence alignment methods, they still struggle in the twilight zone. This suggests that current protein language models have not sufficiently learned protein structure to address remote homology prediction when sequence signals are weak. Availability and implementationWe believe this opens the way for further research both on remote homology prediction and on the broader goal of learning sequence- and structure-rich representations of protein molecules. All code, data, and models are made publicly available. 

   more » « less
2. Heterogeneous entity representation for medicinal synergy prediction

   https://doi.org/10.1093/bioinformatics/btae750  

   Wu, Jiawei; Wen, Jun; Yan, Mingyuan; Dong, Anqi; Gao, Shuai; Wang, Ren; Chen, Can (December 2024, Bioinformatics)

   Lu, Zhiyong (Ed.)

   Abstract MotivationForecasting the synergistic effects of drug combinations facilitates drug discovery and development, especially regarding cancer therapeutics. While numerous computational methods have emerged, most of them fall short in fully modeling the relationships among clinical entities including drugs, cell lines, and diseases, which hampers their ability to generalize to drug combinations involving unseen drugs. These relationships are complex and multidimensional, requiring sophisticated modeling to capture nuanced interplay that can significantly influence therapeutic efficacy. ResultsWe present a novel deep hypergraph learning method named Heterogeneous Entity Representation for MEdicinal Synergy (HERMES) prediction to predict the synergistic effects of anti-cancer drugs. Heterogeneous data sources, including drug chemical structures, gene expression profiles, and disease clinical semantics, are integrated into hypergraph neural networks equipped with a gated residual mechanism to enhance high-order relationship modeling. HERMES demonstrates state-of-the-art performance on two benchmark datasets, significantly outperforming existing methods in predicting the synergistic effects of drug combinations, particularly in cases involving unseen drugs. Availability and implementationThe source code is available at https://github.com/Christina327/HERMES. 

   more » « less
3. Improving protein function prediction by learning and integrating representations of protein sequences and function labels

   https://doi.org/10.1093/bioadv/vbae120  

   Boadu, Frimpong; Cheng, Jianlin (August 2024, Bioinformatics Advances)

   Zhu, Shanfeng (Ed.)

   Abstract MotivationAs fewer than 1% of proteins have protein function information determined experimentally, computationally predicting the function of proteins is critical for obtaining functional information for most proteins and has been a major challenge in protein bioinformatics. Despite the significant progress made in protein function prediction by the community in the last decade, the general accuracy of protein function prediction is still not high, particularly for rare function terms associated with few proteins in the protein function annotation database such as the UniProt. ResultsWe introduce TransFew, a new transformer model, to learn the representations of both protein sequences and function labels [Gene Ontology (GO) terms] to predict the function of proteins. TransFew leverages a large pre-trained protein language model (ESM2-t48) to learn function-relevant representations of proteins from raw protein sequences and uses a biological natural language model (BioBert) and a graph convolutional neural network-based autoencoder to generate semantic representations of GO terms from their textual definition and hierarchical relationships, which are combined together to predict protein function via the cross-attention. Integrating the protein sequence and label representations not only enhances overall function prediction accuracy, but delivers a robust performance of predicting rare function terms with limited annotations by facilitating annotation transfer between GO terms. Availability and implementationhttps://github.com/BioinfoMachineLearning/TransFew. 

   more » « less
4. Combining protein sequences and structures with transformers and equivariant graph neural networks to predict protein function

   https://doi.org/10.1093/bioinformatics/btad208  

   Boadu, Frimpong; Cao, Hongyuan; Cheng, Jianlin (June 2023, Bioinformatics)

   Abstract MotivationMillions of protein sequences have been generated by numerous genome and transcriptome sequencing projects. However, experimentally determining the function of the proteins is still a time consuming, low-throughput, and expensive process, leading to a large protein sequence-function gap. Therefore, it is important to develop computational methods to accurately predict protein function to fill the gap. Even though many methods have been developed to use protein sequences as input to predict function, much fewer methods leverage protein structures in protein function prediction because there was lack of accurate protein structures for most proteins until recently. ResultsWe developed TransFun—a method using a transformer-based protein language model and 3D-equivariant graph neural networks to distill information from both protein sequences and structures to predict protein function. It extracts feature embeddings from protein sequences using a pre-trained protein language model (ESM) via transfer learning and combines them with 3D structures of proteins predicted by AlphaFold2 through equivariant graph neural networks. Benchmarked on the CAFA3 test dataset and a new test dataset, TransFun outperforms several state-of-the-art methods, indicating that the language model and 3D-equivariant graph neural networks are effective methods to leverage protein sequences and structures to improve protein function prediction. Combining TransFun predictions and sequence similarity-based predictions can further increase prediction accuracy. Availability and implementationThe source code of TransFun is available at https://github.com/jianlin-cheng/TransFun. 

   more » « less
5. DeepPurpose: a deep learning library for drug–target interaction prediction

   https://doi.org/10.1093/bioinformatics/btaa1005  

   Huang, Kexin; Fu, Tianfan; Glass, Lucas M; Zitnik, Marinka; Xiao, Cao; Sun, Jimeng (December 2020, Bioinformatics)

   Wren, Jonathan (Ed.)

   Abstract Summary Accurate prediction of drug–target interactions (DTI) is crucial for drug discovery. Recently, deep learning (DL) models for show promising performance for DTI prediction. However, these models can be difficult to use for both computer scientists entering the biomedical field and bioinformaticians with limited DL experience. We present DeepPurpose, a comprehensive and easy-to-use DL library for DTI prediction. DeepPurpose supports training of customized DTI prediction models by implementing 15 compound and protein encoders and over 50 neural architectures, along with providing many other useful features. We demonstrate state-of-the-art performance of DeepPurpose on several benchmark datasets. Availability and implementation https://github.com/kexinhuang12345/DeepPurpose. Supplementary information Supplementary data are available at Bioinformatics online. 

   more » « less