An official website of the United States government
Histopathology images capture tissue morphology, while spatial transcriptomics (ST) provides spatially resolved gene expression, offering complementary molecular insights. However, acquiring ST data is costly and time-consuming, limiting its practical use. To address this, we propose HAGE (Hierarchical Alignment Gene-Enhanced), a framework that enhances pathology representation learning by predicting gene expression directly from histological images and integrating molecular context into the pathology model. HAGE leverages gene-type embeddings, which encode relationships among genes, guiding the model in learning biologically meaningful expression patterns. To further improve alignment between histology and gene expression, we introduce a hierarchical clustering strategy that groups image patches based on molecular and visual similarity, capturing both local and global dependencies. HAGE consistently outperforms existing methods across six datasets. In particular, on the HER2+ breast cancer cohort, it significantly improves the Pearson correlation coefficient by 8.0% and achieves substantial reductions in mean squared error and mean absolute error by 18.1% and 38.0%, respectively. Beyond gene expression prediction, HAGE improves downstream tasks, such as patch-level cancer classification and whole-slide image diagnostics, demonstrating its broader applicability. To the best of our knowledge, HAGE is the first framework to integrate gene co-expression as prior knowledge into a pathology image encoder via a cross-attention mechanism, enabling more biologically informed and accurate pathology representations. https://github.com/uta-smile/gene_expression.
more »
« less
This content will become publicly available on September 9, 2026
Impact of Data Quality on Deep Learning Prediction of Spatial Transcriptomics from Histology Images
Abstract Spatial transcriptomics technologies enable high-throughput quantification of gene expression at specific locations across tissue sections, facilitating insights into the spatial organization of biological processes. However, high costs associated with these technologies have motivated the development of deep learning methods to predict spatial gene expression from inexpensive hematoxylin and eosin-stained histology images. While most efforts have focused on modifying model architectures to boost predictive performance, the influence of training data quality remains largely unexplored. Here, we investigate how variation in molecular and image data quality stemming from differences in imaging (Xenium) versus sequencing (Visium) spatial transcriptomics technologies impact deep learning-based gene expression prediction from histology images. To delineate the aspects of data quality that impact predictive performance, we conductedin silicoablation experiments, which showed that increased sparsity and noise in molecular data degraded predictive performance, whilein silicorescue experiments via imputation provided only limited improvements that failed to generalize beyond the test set. Likewise, reduced image resolution can degrade predictive performance and further impacts model interpretability. Overall, our results underscore how improving data quality offers an orthogonal strategy to tuning model architecture in enhancing predictive modeling using spatial transcriptomics and emphasize the need for careful consideration of technological limitations that directly impact data quality when developing predictive methodologies.
more »
« less
- Award ID(s):
- 2124230
- PAR ID:
- 10652371
- Publisher / Repository:
- bioRxiv
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Current clustering analysis of spatial transcriptomics data primarily relies on molecular information and fails to fully exploit the morphological features present in histology images, leading to compromised accuracy and interpretability. To overcome these limitations, we have developed a multi-stage statistical method called iIMPACT. It identifies and defines histology-based spatial domains based on AI-reconstructed histology images and spatial context of gene expression measurements, and detects domain-specific differentially expressed genes. Through multiple case studies, we demonstrate iIMPACT outperforms existing methods in accuracy and interpretability and provides insights into the cellular spatial organization and landscape of functional genes within spatial transcriptomics data.more » « less
-
Abstract Spatially resolved transcriptomics technologies enable the measurement of transcriptome information while retaining the spatial context at the regional, cellular or sub-cellular level. While previous computational methods have relied on gene expression information alone for clustering single-cell populations, more recent methods have begun to leverage spatial location and histology information to improve cell clustering and cell-type identification. In this study, using seven semi-synthetic datasets with real spatial locations, simulated gene expression and histology images as well as ground truth cell-type labels, we evaluate 15 clustering methods based on clustering accuracy, robustness to data variation and input parameters, computational efficiency, and software usability. Our analysis demonstrates that even though incorporating the additional spatial and histology information leads to increased accuracy in some datasets, it does not consistently improve clustering compared with using only gene expression data. Our results indicate that for the clustering of spatial transcriptomics data, there are still opportunities to enhance the overall accuracy and robustness by improving information extraction and feature selection from spatial and histology data.more » « less
-
Abstract MotivationSpatial transcriptomics technologies, which generate a spatial map of gene activity, can deepen the understanding of tissue architecture and its molecular underpinnings in health and disease. However, the high cost makes these technologies difficult to use in practice. Histological images co-registered with targeted tissues are more affordable and routinely generated in many research and clinical studies. Hence, predicting spatial gene expression from the morphological clues embedded in tissue histological images provides a scalable alternative approach to decoding tissue complexity. ResultsHere, we present a graph neural network based framework to predict the spatial expression of highly expressed genes from tissue histological images. Extensive experiments on two separate breast cancer data cohorts demonstrate that our method improves the prediction performance compared to the state-of-the-art, and that our model can be used to better delineate spatial domains of biological interest. Availability and implementationhttps://github.com/song0309/asGNN/more » « less
-
Abstract Spatial transcriptomics is a powerful and widely used approach for profiling the gene expression landscape across a tissue with emerging applications in molecular medicine and tumor diagnostics. Recent spatial transcriptomics experiments utilize slides containing thousands of spots with spot-specific barcodes that bind RNA. Ideally, unique molecular identifiers (UMIs) at a spot measure spot-specific expression, but this is often not the case in practice due to bleed from nearby spots, an artifact we refer to as spot swapping. To improve the power and precision of downstream analyses in spatial transcriptomics experiments, we propose SpotClean, a probabilistic model that adjusts for spot swapping to provide more accurate estimates of gene-specific UMI counts. SpotClean provides substantial improvements in marker gene analyses and in clustering, especially when tissue regions are not easily separated. As demonstrated in multiple studies of cancer, SpotClean improves tumor versus normal tissue delineation and improves tumor burden estimation thus increasing the potential for clinical and diagnostic applications of spatial transcriptomics technologies.more » « less
