More Like this

1. Online control of the false coverage rate and false sign rate

   Weinstein, Asaf; Ramdas, Aaditya (July 2020, Proceedings of Machine Learning Research)

   The reproducibility debate has caused a renewed interest in changing how one reports uncertainty, from 𝑝-value for testing a null hypothesis to a confidence interval (CI) for the corresponding parameter. When CIs for multiple selected parameters are being reported, the analog of the false discovery rate (FDR) is the false coverage rate (FCR), which is the expected ratio of number of reported CIs failing to cover their respective parameters to the total number of reported CIs. Here, we consider the general problem of FCR control in the online setting, where one encounters an infinite sequence of fixed unknown parameters ordered by time. We propose a novel solution to the problem which only requires the scientist to be able to construct marginal CIs. As special cases, our framework yields algorithms for online FDR control and online sign-classification procedures that control the false sign rate (FSR). All of our methodology applies equally well to prediction intervals, having particular implications for selective conformal inference. 

   more » « less
2. Power of knockoff: The impact of ranking algorithm, augmented design, and symmetric statistic

   Ke, Zheng Tracy; Liu, Jun S; Ma, Yucong (January 2024, Journal of machine learning research)

   The knockoff filter is a recent false discovery rate (FDR) control method for high-dimensional linear models. We point out that knockoff has three key components: ranking algorithm, augmented design, and symmetric statistic, and each component admits multiple choices. By considering various combinations of the three components, we obtain a collection of variants of knockoff. All these variants guarantee finite-sample FDR control, and our goal is to compare their power. We assume a Rare and Weak signal model on regression coeffi- cients and compare the power of different variants of knockoff by deriving explicit formulas of false positive rate and false negative rate. Our results provide new insights on how to improve power when controlling FDR at a targeted level. We also compare the power of knockoff with its propotype - a method that uses the same ranking algorithm but has access to an ideal threshold. The comparison reveals the additional price one pays by finding a data-driven threshold to control FDR. 

   more » « less
3. Secondary-Structure-Dependent Cooperation and Interference Between Peptides of Different Chain Lengths in Antifreeze Activity: Insights from Molecular Dynamics Simulations

   https://doi.org/10.3390/foods15122228  

   Yuan, Yuan; Smith, Micholas; Wang, Tong (June 2026, Foods)

   Ice recrystallization inhibition (IRI) activity of peptides is influenced by both peptide length and secondary structure; however, whether combinations of peptides with different lengths exhibit cooperative or antagonistic effects remains poorly understood. Using molecular dynamics simulations, this study investigated how secondary structure and chain-length heterogeneity jointly affect the IRI activity of peptide-pair mixtures. For systems containing only β-sheet-rich peptides, mixtures of different chain lengths consistently reduced ice content relative to the corresponding single-peptide systems, suggesting cooperative enhancement of IRI activity. In contrast, individual α-helical peptides showed strong inhibition of ice growth, but this effect was diminished after they were mixed into peptide pairs. Structural analyses suggested that the improved performance of β-sheet mixtures was associated less with the simple preservation of native β-sheet structure than with mixing-induced changes in peptide–peptide coupling and surface exposure. By contrast, helix-containing mixtures retained more of their original local structure in some cases, but this structural retention was not accompanied by improved ice-growth suppression after mixing. Together, these findings suggest that peptide length effects on IRI are not universally synergistic but depend strongly on secondary-structure compatibility. 

   more » « less
4. APIR: Aggregating Universal Proteomics Database Search Algorithms for Peptide Identification with FDR Control

   https://doi.org/10.1093/gpbjnl/qzae042  

   Chen, Yiling Elaine; Ge, Xinzhou; Woyshner, Kyla; McDermott, MeiLu; Manousopoulou, Antigoni; Ficarro, Scott B; Marto, Jarrod A; Li, Kexin; Wang, Leo David; Li, Jingyi Jessica (April 2024, Genomics, Proteomics & Bioinformatics)

   Fu, Yan (Ed.)

   Abstract Advances in mass spectrometry (MS) have enabled high-throughput analysis of proteomes in biological systems. The state-of-the-art MS data analysis relies on database search algorithms to quantify proteins by identifying peptide–spectrum matches (PSMs), which convert mass spectra to peptide sequences. Different database search algorithms use distinct search strategies and thus may identify unique PSMs. However, no existing approaches can aggregate all user-specified database search algorithms with a guaranteed increase in the number of identified peptides and a control on the false discovery rate (FDR). To fill in this gap, we proposed a statistical framework, Aggregation of Peptide Identification Results (APIR), that is universally compatible with all database search algorithms. Notably, under an FDR threshold, APIR is guaranteed to identify at least as many, if not more, peptides as individual database search algorithms do. Evaluation of APIR on a complex proteomics standard dataset showed that APIR outpowers individual database search algorithms and empirically controls the FDR. Real data studies showed that APIR can identify disease-related proteins and post-translational modifications missed by some individual database search algorithms. The APIR framework is easily extendable to aggregating discoveries made by multiple algorithms in other high-throughput biomedical data analysis, e.g., differential gene expression analysis on RNA sequencing data. The APIR R package is available at https://github.com/yiling0210/APIR. 

   more » « less
5. Leveraging conformal prediction to annotate enzyme function space with limited false positives

   https://doi.org/10.1371/journal.pcbi.1012135  

   Ding, Kerr; Luo, Jiaqi; Luo, Yunan (May 2024, PLOS Computational Biology)

   Mura, Cameron (Ed.)

   Machine learning (ML) is increasingly being used to guide biological discovery in biomedicine such as prioritizing promising small molecules in drug discovery. In those applications, ML models are used to predict the properties of biological systems, and researchers use these predictions to prioritize candidates as new biological hypotheses for downstream experimental validations. However, when applied to unseen situations, these models can be overconfident and produce a large number of false positives. One solution to address this issue is to quantify the model’s prediction uncertainty and provide a set of hypotheses with a controlled false discovery rate (FDR) pre-specified by researchers. We propose CPEC, an ML framework for FDR-controlled biological discovery. We demonstrate its effectiveness using enzyme function annotation as a case study, simulating the discovery process of identifying the functions of less-characterized enzymes. CPEC integrates a deep learning model with a statistical tool known as conformal prediction, providing accurate and FDR-controlled function predictions for a given protein enzyme. Conformal prediction provides rigorous statistical guarantees to the predictive model and ensures that the expected FDR will not exceed a user-specified level with high probability. Evaluation experiments show that CPEC achieves reliable FDR control, better or comparable prediction performance at a lower FDR than existing methods, and accurate predictions for enzymes under-represented in the training data. We expect CPEC to be a useful tool for biological discovery applications where a high yield rate in validation experiments is desired but the experimental budget is limited. 

   more » « less