More Like this

1. Batch normalization followed by merging is powerful for phenotype prediction integrating multiple heterogeneous studies

   https://doi.org/10.1371/journal.pcbi.1010608  

   Gao, Yilin; Sun, Fengzhu (October 2023, PLOS Computational Biology)

   Roy, Sushmita (Ed.)

   Heterogeneity in different genomic studies compromises the performance of machine learning models in cross-study phenotype predictions. Overcoming heterogeneity when incorporating different studies in terms of phenotype prediction is a challenging and critical step for developing machine learning algorithms with reproducible prediction performance on independent datasets. We investigated the best approaches to integrate different studies of the same type of omics data under a variety of different heterogeneities. We developed a comprehensive workflow to simulate a variety of different types of heterogeneity and evaluate the performances of different integration methods together with batch normalization by using ComBat. We also demonstrated the results through realistic applications on six colorectal cancer (CRC) metagenomic studies and six tuberculosis (TB) gene expression studies, respectively. We showed that heterogeneity in different genomic studies can markedly negatively impact the machine learning classifier’s reproducibility. ComBat normalization improved the prediction performance of machine learning classifier when heterogeneous populations are present, and could successfully remove batch effects within the same population. We also showed that the machine learning classifier’s prediction accuracy can be markedly decreased as the underlying disease model became more different in training and test populations. Comparing different merging and integration methods, we found that merging and integration methods can outperform each other in different scenarios. In the realistic applications, we observed that the prediction accuracy improved when applying ComBat normalization with merging or integration methods in both CRC and TB studies. We illustrated that batch normalization is essential for mitigating both population differences of different studies and batch effects. We also showed that both merging strategy and integration methods can achieve good performances when combined with batch normalization. In addition, we explored the potential of boosting phenotype prediction performance by rank aggregation methods and showed that rank aggregation methods had similar performance as other ensemble learning approaches. 

   more » « less
2. scDisInFact: disentangled learning for integration and prediction of multi-batch multi-condition single-cell RNA-sequencing data

   https://doi.org/10.1038/s41467-024-45227-w  

   Zhang, Ziqi; Zhao, Xinye; Bindra, Mehak; Qiu, Peng; Zhang, Xiuwei (December 2024, Nature Communications)

   Abstract Single-cell RNA-sequencing (scRNA-seq) has been widely used for disease studies, where sample batches are collected from donors under different conditions including demographic groups, disease stages, and drug treatments. It is worth noting that the differences among sample batches in such a study are a mixture of technical confounders caused by batch effect and biological variations caused by condition effect. However, current batch effect removal methods often eliminate both technical batch effect and meaningful condition effect, while perturbation prediction methods solely focus on condition effect, resulting in inaccurate gene expression predictions due to unaccounted batch effect. Here we introduce scDisInFact, a deep learning framework that models both batch effect and condition effect in scRNA-seq data. scDisInFact learns latent factors that disentangle condition effect from batch effect, enabling it to simultaneously perform three tasks: batch effect removal, condition-associated key gene detection, and perturbation prediction. We evaluate scDisInFact on both simulated and real datasets, and compare its performance with baseline methods for each task. Our results demonstrate that scDisInFact outperforms existing methods that focus on individual tasks, providing a more comprehensive and accurate approach for integrating and predicting multi-batch multi-condition single-cell RNA-sequencing data. 

   more » « less
3. ComBat-seq: batch effect adjustment for RNA-seq count data

   https://doi.org/10.1093/nargab/lqaa078  

   Zhang, Yuqing; Parmigiani, Giovanni; Johnson, W Evan (September 2020, Nucleic acids research)

   null (Ed.)

   The benefit of integrating batches of genomic data to increase statistical power is often hindered by batch effects, or unwanted variation in data caused by differences in technical factors across batches. It is therefore critical to effectively address batch effects in genomic data to overcome these challenges. Many existing methods for batch effects adjustment assume the data follow a continuous, bell-shaped Gaussian distribution. However in RNA-seq studies the data are typically skewed, over-dispersed counts, so this assumption is not appropriate and may lead to erroneous results. Negative binomial regression models have been used previously to better capture the properties of counts. We developed a batch correction method, ComBat-seq, using a negative binomial regression model that retains the integer nature of count data in RNA-seq studies, making the batch adjusted data compatible with common differential expression software packages that require integer counts. We show in realistic simulations that the ComBat-seq adjusted data results in better statistical power and control of false positives in differential expression compared to data adjusted by the other available methods. We further demonstrated in a real data example that ComBat-seq successfully removes batch effects and recovers the biological signal in the data. 

   more » « less
4. A random batch method for efficient ensemble forecasts of multiscale turbulent systems

   https://doi.org/10.1063/5.0129127  

   Qi, Di; Liu, Jian-Guo (February 2023, Chaos: An Interdisciplinary Journal of Nonlinear Science)

   A new efficient ensemble prediction strategy is developed for a multiscale turbulent model framework with emphasis on the nonlinear interactions between large and small-scale variables. The high computational cost in running large ensemble simulations of high-dimensional equations is effectively avoided by adopting a random batch decomposition of the wide spectrum of the fluctuation states, which is a characteristic feature of the multiscale turbulent systems. The time update of each ensemble sample is then only subject to a small portion of the small-scale fluctuation modes in one batch, while the true model dynamics with multiscale coupling is respected by frequent random resampling of the batches at each time updating step. We investigate both theoretical and numerical properties of the proposed method. First, the convergence of statistical errors in the random batch model approximation is shown rigorously independent of the sample size and full dimension of the system. Next, the forecast skill of the computational algorithm is tested on two representative models of turbulent flows exhibiting many key statistical phenomena with a direct link to realistic turbulent systems. The random batch method displays robust performance in capturing a series of crucial statistical features with general interests, including highly non-Gaussian fat-tailed probability distributions and intermittent bursts of instability, while requires a much lower computational cost than the direct ensemble approach. The efficient random batch method also facilitates the development of new strategies in uncertainty quantification and data assimilation for a wide variety of general complex turbulent systems in science and engineering. 

   more » « less
5. Microbiome Data Integration via Shared Dictionary Learning

   https://doi.org/10.1101/2024.10.04.616752  

   Yuan, Bo; Wang, Shulei (October 2024, bioRxiv)

   Abstract Data integration is a powerful tool for facilitating a comprehensive and generalizable understanding of microbial communities and their association with outcomes of interest. However, integrating data sets from different studies remains a challenging problem because of severe batch effects, unobserved confounding variables, and high heterogeneity across data sets. We propose a new data integration method called MetaDICT, which initially estimates the batch effects by weighting methods in causal inference literature and then refines the estimation via a novel shared dictionary learning. Compared with existing methods, MetaDICT can better avoid the overcorrection of batch effects and preserve biological variation when there exist unobserved confounding variables or data sets are highly heterogeneous across studies. Furthermore, MetaDICT can generate comparable embedding at both taxa and sample levels that can be used to unravel the hidden structure of the integrated data and improve the integrative analysis. Applications to synthetic and real microbiome data sets demonstrate the robustness and effectiveness of MetaDICT in integrative analysis. Using MetaDICT, we characterize microbial interaction, identify generalizable microbial signatures, and enhance the accuracy of disease prediction in an integrative analysis of colorectal cancer metagenomics studies. 

   more » « less