Title: Assessment of false discovery rate control in tandem mass spectrometry analysis using entrapment
Abstract A critical challenge in mass spectrometry proteomics is accurately assessing error control, especially given that software tools employ distinct methods for reporting errors. Many tools are closed-source and poorly documented, leading to inconsistent validation strategies. Here we identify three prevalent methods for validating false discovery rate (FDR) control: one invalid, one providing only a lower bound, and one valid but under-powered. The result is that the proteomics community has limited insight into actual FDR control effectiveness, especially for data-independent acquisition (DIA) analyses. We propose a theoretical framework for entrapment experiments, allowing us to rigorously characterize different approaches. Moreover, we introduce a more powerful evaluation method and apply it alongside existing techniques to assess existing tools. We first validate our analysis in the better-understood data-dependent acquisition setup, and then, we analyze DIA data, where we find that no DIA search tool consistently controls the FDR, with particularly poor performance on single-cell datasets.  more » « less
Award ID(s):
2245300
PAR ID:
10684863
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Springer Nature
Date Published:
Journal Name:
Nature Methods
Volume:
22
Issue:
7
ISSN:
1548-7091
Page Range / eLocation ID:
1454 to 1463
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Data-independent acquisition (DIA)-based mass spectrometry is becoming an increasingly popular mass spectrometry acquisition strategy for carrying out quantitative proteomics experiments. Most of the popular DIA search engines make use of in silico generated spectral libraries. However, the generation of high-quality spectral libraries for DIA data analysis remains a challenge, particularly because most such libraries are generated directly from data-dependent acquisition (DDA) data or are from in silico prediction using models trained on DDA data. In this study, we introduce Carafe, a tool that generates high-quality experiment-specific in silico spectral libraries by training deep learning models directly on DIA data. We demonstrate the performance of Carafe on a wide range of DIA datasets, where we observe improved fragment ion intensity prediction and peptide detection relative to existing pretrained DDA models. To make Carafe more accessible to the community, we integrate Carafe into the widely used Skyline tool. 
    more » « less
  2. Fu, Yan (Ed.)
    Abstract Advances in mass spectrometry (MS) have enabled high-throughput analysis of proteomes in biological systems. The state-of-the-art MS data analysis relies on database search algorithms to quantify proteins by identifying peptide–spectrum matches (PSMs), which convert mass spectra to peptide sequences. Different database search algorithms use distinct search strategies and thus may identify unique PSMs. However, no existing approaches can aggregate all user-specified database search algorithms with a guaranteed increase in the number of identified peptides and a control on the false discovery rate (FDR). To fill in this gap, we proposed a statistical framework, Aggregation of Peptide Identification Results (APIR), that is universally compatible with all database search algorithms. Notably, under an FDR threshold, APIR is guaranteed to identify at least as many, if not more, peptides as individual database search algorithms do. Evaluation of APIR on a complex proteomics standard dataset showed that APIR outpowers individual database search algorithms and empirically controls the FDR. Real data studies showed that APIR can identify disease-related proteins and post-translational modifications missed by some individual database search algorithms. The APIR framework is easily extendable to aggregating discoveries made by multiple algorithms in other high-throughput biomedical data analysis, e.g., differential gene expression analysis on RNA sequencing data. The APIR R package is available at https://github.com/yiling0210/APIR. 
    more » « less
  3. Shotgun proteomics has been widely used to identify histone marks. Conventional database search methods rely on the “target-decoy” strategy to calculate the false discovery rate (FDR) and distinguish true peptide-spectrum matches (PSMs) from false ones. This strategy has a caveat of inaccurate FDR caused by the small data size of histone marks. To address this challenge, we developed a tailored database search strategy, named “Comprehensive Histone Mark Analysis (CHiMA).” Instead of target-decoy–based FDR, this method uses “50% matched fragment ions” as the key criterion to identify high-confidence PSMs. CHiMA identified twice as many histone modification sites as the conventional method in benchmark datasets. Reanalysis of our previous proteomics data using CHiMA led to the identification of 113 new histone marks for four types of lysine acylations, almost doubling the number of previously reported marks. This tool not only offers a valuable approach for identifying histone modifications but also greatly expands the repertoire of histone marks. 
    more » « less
  4. null (Ed.)
    Interactions of organisms with their environment are complex and regulation at different levels of biological organization from genotype to phenotype is often non-linear. While studies of transcriptome regulation are now common for many species, corresponding quantitative studies of environmental effects on proteomes are needed. Here we report the generation of a data-independent acquisition (DIA) assay library that enables simultaneous targeted proteomics of thousands of O. niloticus kidney proteins using a label- and gel-free workflow that is well suited for ecologically relevant field samples. Transcript and protein abundance differences in kidneys of tilapia acclimated to freshwater and brackish water (25 g/kg) were correlated for 2114 unique genes. A high degree of non-linearity in salinity-dependent regulation of transcriptomes and proteomes was revealed, demonstrating the complementary nature of the DIA assay library approach and suggesting that the regulation of O. niloticus renal function by environmental salinity relies heavily on post-transcriptional mechanisms. In addition to significance testing, the application of functional enrichment analyses using STRING and KEGG to DIA assay datasets identified myo-inositol metabolism, antioxidant and xenobiotic functions, and signaling mechanisms as key elements controlled by salinity in tilapia kidneys. In conclusion, this study presents an innovative approach for targeted quantitative proteomics used to identify proteins and biological processes that are regulated non-linearly at mRNA and protein levels during a change of environmental salinity. 
    more » « less
  5. null (Ed.)
    Interactions of organisms with their environment are complex and regulation at different levels of biological organization from genotype to phenotype is often non-linear. While studies of transcriptome regulation are now common for many species, corresponding quantitative studies of environmental effects on proteomes are needed. Here we report the generation of a data-independent acquisition (DIA) assay library that enables simultaneous targeted proteomics of thousands of O. niloticus kidney proteins using a label- and gel-free workflow that is well suited for ecologically relevant field samples. Transcript and protein abundance differences in kidneys of tilapia acclimated to freshwater and brackish water (25 g/kg) were correlated for 2114 unique genes. A high degree of non-linearity in salinity-dependent regulation of transcriptomes and proteomes was revealed, demonstrating the complementary nature of the DIA assay library approach and suggesting that the regulation of O. niloticus renal function by environmental salinity relies heavily on post-transcriptional mechanisms. In addition to significance testing, the application of functional enrichment analyses using STRING and KEGG to DIA assay datasets identified myo-inositol metabolism, antioxidant and xenobiotic functions, and signaling mechanisms as key elements controlled by salinity in tilapia kidneys. In conclusion, this study presents an innovative approach for targeted quantitative proteomics used to identify proteins and biological processes that are regulated non-linearly at mRNA and protein levels during a change of environmental salinity. 
    more » « less