Search for: All records

Abstract The phenotype of an organism is shaped by gene expression within developing tissues. This shaping relates the evolution of gene expression to phenotypic evolution, through divergence in gene expression and consequent phenotype. Rates of phenotypic evolution receive extensive attention. However, the degree to which divergence in the phenotype of gene expression is subject to heterogeneous rates of evolution across developmental stages has not previously been assessed. Here, we analyzed the evolution of the expression of single-copy orthologs within 9 species of Sordariomycetes Fungi, across 9 developmental stages within asexual spore germination and sexual reproduction. Rates of gene expression evolution exhibited high variation both within and among developmental stages. Furthermore, rates of gene expression evolution were correlated with nonsynonymous to synonymous substitution rates (dN/dS), suggesting that gene sequence evolution and expression evolution are indirectly or directly driven by common evolutionary forces. Functional pathway analyses demonstrate that rates of gene expression evolution are higher in labile pathways such as carbon metabolism, and lower in conserved pathways such as those involved in cell cycle and molecular signaling. Lastly, the expression of genes in the meiosis pathway evolved at a slower rate only across the stages where meiosis took place, suggesting that stage-specific low rates of expression evolution implicate high relevance of the genes to developmental operations occurring between those stages. 

In opportunistic human pathogenic fungi, changes in gene expression play a crucial role in the progression of growth stages from early spore germination through host infection. Comparative transcriptomics between diverse fungal pathogens and non-pathogens provided insights into regulatory mechanisms behind the initiation of infectious processes. We examined the gene expression patterns of 3,845 single-copy orthologous genes (SCOGs) across five phylogenetically distinct species, including the opportunistic human pathogens Fusarium oxysporum, Aspergillus fumigatus, and A. nidulans, and nonpathogenic species Neurospora crassa and Trichoderma asperelloides, at four sequential stages of spore germination. Ancestral status of gene expression was inferred for nodes along the phylogeny. By comparing expression patterns of the SCOGs with their most recent common ancestor (MRCA), we identified genes that exhibit divergent levels of expression during spore germination when comparing fungal pathogens to non-pathogens. We focused on genes related to the MAPK pathway, nitrogen metabolism, asexual development, G-protein signaling, and conidial-wall integrity. Notably, orthologs of the transcription activator abaA, a known central regulator of conidiation, exhibited significant divergence in gene expression in F. oxysporum. This dramatic expression change in abaA was accompanied by structural modifications of phialides in F. oxysporum, and revealed how these changes impact development of offspring, formation of aerial hyphae, spore production, and pathogenicity. Our research provides insights into ecological adaptations observed during the divergence of these species, specifically highlighting how divergence in gene expression during spore germination contributes to their ability to thrive in distinct environments. 

Autoregressive Large Language Models (LLMs) have achieved impressive performance in language tasks but face two significant bottlenecks: (1) quadratic complexity in the attention module as the number of tokens increases, and (2) limited efficiency due to the sequential processing nature of autoregressive LLMs during generation. While linear attention and speculative decoding offer potential solutions, their applicability and synergistic potential for enhancing autoregressive LLMs remain uncertain. We conduct the first comprehensive study on the efficacy of existing linear attention methods for autoregressive LLMs, integrating them with speculative decoding. We introduce an augmentation technique for linear attention that ensures compatibility with speculative decoding, enabling more efficient training and serving of LLMs. Extensive experiments and ablation studies involving seven existing linear attention models and five encoder/decoder-based LLMs consistently validate the effectiveness of our augmented linearized LLMs. Notably, our approach achieves up to a 6.67 reduction in perplexity on the LLaMA model and up to a 2× speedup during generation compared to prior linear attention methods. Codes and models are available at https://github.com/GATECH-EIC/Linearized-LLM. 

Autoregressive Large Language Models (LLMs) have achieved impressive performance in language tasks but face two significant bottlenecks: (1) quadratic complexity in the attention module as the number of tokens increases, and (2) limited efficiency due to the sequential processing nature of autoregressive LLMs during generation. While linear attention and speculative decoding offer potential solutions, their applicability and synergistic potential for enhancing autoregressive LLMs remain uncertain. We conduct the first comprehensive study on the efficacy of existing linear attention methods for autoregressive LLMs, integrating them with speculative decoding. We introduce an augmentation technique for linear attention that ensures compatibility with speculative decoding, enabling more efficient training and serving of LLMs. Extensive experiments and ablation studies involving seven existing linear attention models and five encoder/decoder-based LLMs consistently validate the effectiveness of our augmented linearized LLMs. Notably, our approach achieves up to a 6.67 reduction in perplexity on the LLaMA model and up to a 2× speedup during generation compared to prior linear attention methods. Codes and models are available at https://github.com/GATECH-EIC/Linearized-LLM. 

The deployment of Deep Learning Recommendation Models (DLRMs) involves the parallelization of extra-large embedding tables (EMTs) on multiple GPUs. Existing works overlook the input-dependent behavior of EMTs and parallelize them in a coarse-grained manner, resulting in unbalanced workload distribution and inter-GPU communication. To this end, we propose OPER, an algorithm-system co-design with OPtimality-guided Embedding table parallelization for large-scale Recommendation model training and inference. The core idea of OPER is to explore the connection between DLRM inputs and the efficiency of distributed EMTs, aiming to provide a near-optimal parallelization strategy for EMTs. Specifically, we conduct an in-depth analysis of various types of EMTs parallelism and propose a heuristic search algorithm to efficiently approximate an empirically near-optimal EMT parallelization. Furthermore, we implement a distributed shared memory-based system, which supports the lightweight but complex computation and communication pattern of fine-grained EMT parallelization, effectively converting theoretical improvements into real speedups. Extensive evaluation shows that OPER achieves 2.3× and 4.0× speedup on average in training and inference, respectively, over state-of-the-art DLRM frameworks.