skip to main content


The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, May 23 until 2:00 AM ET on Friday, May 24 due to maintenance. We apologize for the inconvenience.

This content will become publicly available on December 1, 2024

Title: Soil organic carbon models need independent time-series validation for reliable prediction
Abstract Numerical models are crucial to understand and/or predict past and future soil organic carbon dynamics. For those models aiming at prediction, validation is a critical step to gain confidence in projections. With a comprehensive review of ~250 models, we assess how models are validated depending on their objectives and features, discuss how validation of predictive models can be improved. We find a critical lack of independent validation using observed time series. Conducting such validations should be a priority to improve the model reliability. Approximately 60% of the models we analysed are not designed for predictions, but rather for conceptual understanding of soil processes. These models provide important insights by identifying key processes and alternative formalisms that can be relevant for predictive models. We argue that combining independent validation based on observed time series and improved information flow between predictive and conceptual models will increase reliability in predictions.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more » ; ; ; ; ; ; « less
Date Published:
Journal Name:
Communications Earth & Environment
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Aim

    Species distribution models (SDMs) are widely used to make predictions on how species distributions may change as a response to climatic change. To assess the reliability of those predictions, they need to be critically validated with respect to what they are used for. While ecologists are typically interested in how and where distributions will change, we argue that SDMs have seldom been evaluated in terms of their capacity to predict such change. Instead, typical retrospective validation methods estimate model's ability to predict to only one static time in future. Here, we apply two validation methods, one that predicts and evaluates a static pattern, while the other measures change and compare their estimates of predictive performance.




    We applied a joint SDM to model the distributions of 120 bird species in four model validation settings. We trained models with a dataset from 1975 to 1999 and predicted species' future occurrence and abundance in two ways: for one static time period (2013–2016, ‘static validation’) and for a change between two time periods (difference between 1996–1999 and 2013–2016, ‘change validation’). We then measured predictive performance using correlation between predicted and observed values. We also related predictive performance to species traits.


    Even though static validation method evaluated predictive performance as good, change method indicated very poor performance. Predictive performance was not strongly related to any trait.

    Main Conclusions

    Static validation method might overestimate predictive performance by not revealing the model's inability to predict change events. If species' distributions remain mostly stable, then even an unfit model can predict the near future well due to temporal autocorrelation. We urge caution when working with forecasts of changes in spatial patterns of species occupancy or abundance, even for SDMs that are based on time series datasets unless they are critically validated for forecasting such change.

    more » « less
  2. Abstract. The terrestrial carbon cycle plays a critical role in modulating the interactions of climate with the Earth system, but different models often make vastly different predictions of its behavior. Efforts to reduce model uncertainty have commonly focused on model structure, namely by introducing additional processes and increasing structural complexity. However, the extent to which increased structural complexity can directly improve predictive skill is unclear. While adding processes may improve realism, the resulting models are often encumbered by a greater number of poorly determined or over-generalized parameters. To guide efficient model development, here we map the theoretical relationship between model complexity and predictive skill. To do so, we developed 16 structurally distinct carbon cycle models spanning an axis of complexity and incorporated them into a model–data fusion system. We calibrated each model at six globally distributed eddy covariance sites with long observation time series and under 42 data scenarios that resulted in different degrees of parameter uncertainty. For each combination of site, data scenario, and model, we then predicted net ecosystem exchange (NEE) and leaf area index (LAI) for validation against independent local site data. Though the maximum model complexity we evaluated is lower than most traditional terrestrial biosphere models, the complexity range we explored provides universal insight into the inter-relationship between structural uncertainty, parametric uncertainty, and model forecast skill. Specifically, increased complexity only improves forecast skill if parameters are adequately informed (e.g., when NEE observations are used for calibration). Otherwise, increased complexity can degrade skill and an intermediate-complexity model is optimal. This finding remains consistent regardless of whether NEE or LAI is predicted. Our COMPLexity EXperiment (COMPLEX) highlights the importance of robust observation-based parameterization for land surface modeling and suggests that data characterizing net carbon fluxes will be key to improving decadal predictions of high-dimensional terrestrial biosphere models. 
    more » « less
  3. Phenotypic evaluation and efficient utilization of germplasm collections can be time-intensive, laborious, and expensive. However, with the plummeting costs of next-generation sequencing and the addition of genomic selection to the plant breeder’s toolbox, we now can more efficiently tap the genetic diversity within large germplasm collections. In this study, we applied and evaluated genomic prediction’s potential to a set of 482 pea ( Pisum sativum L.) accessions—genotyped with 30,600 single nucleotide polymorphic (SNP) markers and phenotyped for seed yield and yield-related components—for enhancing selection of accessions from the USDA Pea Germplasm Collection. Genomic prediction models and several factors affecting predictive ability were evaluated in a series of cross-validation schemes across complex traits. Different genomic prediction models gave similar results, with predictive ability across traits ranging from 0.23 to 0.60, with no model working best across all traits. Increasing the training population size improved the predictive ability of most traits, including seed yield. Predictive abilities increased and reached a plateau with increasing number of markers presumably due to extensive linkage disequilibrium in the pea genome. Accounting for population structure effects did not significantly boost predictive ability, but we observed a slight improvement in seed yield. By applying the best genomic prediction model (e.g., RR-BLUP), we then examined the distribution of genotyped but nonphenotyped accessions and the reliability of genomic estimated breeding values (GEBV). The distribution of GEBV suggested that none of the nonphenotyped accessions were expected to perform outside the range of the phenotyped accessions. Desirable breeding values with higher reliability can be used to identify and screen favorable germplasm accessions. Expanding the training set and incorporating additional orthogonal information (e.g., transcriptomics, metabolomics, physiological traits, etc.) into the genomic prediction framework can enhance prediction accuracy. 
    more » « less
  4. Abstract

    Proximity to roads is one of the main determinants of deforestation in the Amazon basin. Determining the construction year of roads (CYR) is critical to improve the understanding of the drivers of road construction and to enable predictions of the expansion of the road network and its consequent impact on ecosystems. While recent artificial intelligence approaches have been successfully used for road extraction, they have typically relied on high spatial‐resolution imagery, precluding their adoption for the determination of CYR for older roads. In this article, we developed a new approach to automate the process of determining CYR that relies on the approximate position of the current road network and a time‐series of the proportion of exposed soil based on the multidecadal remote sensing imagery from the Landsat program. Starting with these inputs, our methodology relies on the Least Cost Path algorithm to co‐register the road network and on a Before‐After Control‐Impact design to circumvent the inherent image‐to‐image variability in the estimated amount of exposed soil. We demonstrate this approach for a 357 000 km2area around the Transamazon highway (BR‐230) in the Brazilian Amazon, encompassing 36 240 road segments. The reliability of this approach is assessed by comparing the estimated CYR using our approach to the observed CYR based on a time‐series of Landsat images. This exercise reveals a close correspondence between the estimated and observed CYR (). Finally, we show how these data can be used to assess the effectiveness of protected areas (PAs) in reducing the yearly rate of road construction and thus their vulnerability to future degradation. In particular, we find that integral protection PAs in this region were generally more effective in reducing the expansion of the road network when compared to sustainable use PAs.

    more » « less
  5. Abstract

    An estimated 1700 Pg of carbon is frozen in the Arctic permafrost and the fate of this carbon is unclear because of the complex interaction of biophysical, ecological and biogeochemical processes that govern the Arctic carbon budget. Two key processes determining the region’s long-term carbon budget are: (a) carbon uptake through increased plant growth, and (b) carbon release through increased heterotrophic respiration (HR) due to warmer soils. Previous predictions for how these two opposing carbon fluxes may change in the future have varied greatly, indicating that improved understanding of these processes and their feedbacks is critical for advancing our predictive ability for the fate of Arctic peatlands. In this study, we implement and analyze a vertically-resolved model of peatland soil carbon into a cohort-based terrestrial biosphere model to improve our understanding of how on-going changes in climate are altering the Arctic carbon budget. A key feature of the formulation is that accumulation of peat within the soil column modifies its texture, hydraulic conductivity, and thermal conductivity, which, in turn influences resulting rates of HR within the soil column. Analysis of the model at three eddy covariance tower sites in the Alaskan tundra shows that the vertically-resolved soil column formulation accurately captures the zero-curtain phenomenon, in which the temperature of soil layers remain at or near 0 °C during fall freezeback due to the release of latent heat, is critical to capturing observed patterns of wintertime respiration. We find that significant declines in net ecosystem productivity (NEP) occur starting in 2013 and that these declines are driven by increased HR arising from increased precipitation and warming. Sensitivity analyses indicate that the cumulative NEP over the decade responds strongly to the estimated soil carbon stock and more weakly to vegetation abundance at the beginning of the simulation.

    more » « less