skip to main content

Title: Upscaling soil organic carbon measurements at the continental scale using multivariate clustering analysis and machine learning

Data Description:

To improve SOC estimation in the United States, we upscaled site-based SOC measurements to the continental scale using multivariate geographic clustering (MGC) approach coupled with machine learning models. First, we used the MGC approach to segment the United States at 30 arc second resolution based on principal component information from environmental covariates (gNATSGO soil properties, WorldClim bioclimatic variables, MODIS biological variables, and physiographic variables) to 20 SOC regions. We then trained separate random forest model ensembles for each of the SOC regions identified using environmental covariates and soil profile measurements from the International Soil Carbon Network (ISCN) and an Alaska soil profile data. We estimated United States SOC for 0-30 cm and 0-100 cm depths were 52.6 + 3.2 and 108.3 + 8.2 Pg C, respectively.

Files in collection (32):

Collection contains 22 soil properties geospatial rasters, 4 soil SOC geospatial rasters, 2 ISCN site SOC observations csv files, and 4 R scripts

gNATSGO TIF files:

├── available_water_storage_30arc_30cm_us.tif                   [30 cm depth soil available water storage]
├── available_water_storage_30arc_100cm_us.tif                 [100 cm depth soil available water storage]
├── caco3_30arc_30cm_us.tif                                                 [30 cm depth soil CaCO3 content]
├── caco3_30arc_100cm_us.tif                                               [100 cm depth soil CaCO3 content]
├── cec_30arc_30cm_us.tif                                                     [30 cm depth soil cation exchange capacity]
├── cec_30arc_100cm_us.tif                                                   [100 cm depth soil cation exchange capacity]
├── clay_30arc_30cm_us.tif                                                     [30 cm depth soil clay content]
├── clay_30arc_100cm_us.tif                                                   [100 cm depth soil clay content]
├── depthWT_30arc_us.tif                                                        [depth to water table]
├── kfactor_30arc_30cm_us.tif                                                 [30 cm depth soil erosion factor]
├── kfactor_30arc_100cm_us.tif                                               [100 cm depth soil erosion factor]
├── ph_30arc_100cm_us.tif                                                      [100 cm depth soil pH]
├── ph_30arc_100cm_us.tif                                                      [30 cm depth soil pH]
├── pondingFre_30arc_us.tif                                                     [ponding frequency]
├── sand_30arc_30cm_us.tif                                                    [30 cm depth soil sand content]
├── sand_30arc_100cm_us.tif                                                  [100 cm depth soil sand content]
├── silt_30arc_30cm_us.tif                                                        [30 cm depth soil silt content]
├── silt_30arc_100cm_us.tif                                                      [100 cm depth soil silt content]
├── water_content_30arc_30cm_us.tif                                      [30 cm depth soil water content]
└── water_content_30arc_100cm_us.tif                                   [100 cm depth soil water content]

SOC TIF files:

├──30cm SOC mean.tif                             [30 cm depth soil SOC]
├──100cm SOC mean.tif                           [100 cm depth soil SOC]
├──30cm SOC CV.tif                                 [30 cm depth soil SOC coefficient of variation]
└──100cm SOC CV.tif                              [100 cm depth soil SOC coefficient of variation]

site observations csv files:

ISCN_rmNRCS_addNCSS_30cm.csv       30cm ISCN sites SOC replaced NRCS sites with NCSS centroid removed data

ISCN_rmNRCS_addNCSS_100cm.csv       100cm ISCN sites SOC replaced NRCS sites with NCSS centroid removed data

Data format:

Geospatial files are provided in Geotiff format in Lat/Lon WGS84 EPSG: 4326 projection at 30 arc second resolution.

Geospatial projection

GEOGCS["GCS_WGS_1984", DATUM["D_WGS_1984", SPHEROID["WGS_1984",6378137,298.257223563]], PRIMEM["Greenwich",0], UNIT["Degree",0.017453292519943295]] (base) [jbk@theseus ltar_regionalization]$ g.proj -w GEOGCS["wgs84", DATUM["WGS_1984", SPHEROID["WGS_1984",6378137,298.257223563]], PRIMEM["Greenwich",0], UNIT["degree",0.0174532925199433]]


more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Date Published:
Subject(s) / Keyword(s):
["the United States SOC","US soil properties","Gridded National Soil Survey Geographic Database","gNATSGO","International Soil Carbon Network (ISCN)"]
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Estimates of soil organic carbon (SOC) stocks are essential for many environmental applications. However, significant inconsistencies exist in SOC stock estimates for the U.S. across current SOC maps. We propose a framework that combines unsupervised multivariate geographic clustering (MGC) and supervised Random Forests regression, improving SOC maps by capturing heterogeneous relationships with SOC drivers. We first used MGC to divide the U.S. into 20 SOC regions based on the similarity of covariates (soil biogeochemical, bioclimatic, biological, and physiographic variables). Subsequently, separate Random Forests models were trained for each SOC region, utilizing environmental covariates and SOC observations. Our estimated SOC stocks for the U.S. (52.6 ± 3.2 Pg for 0–30 cm and 108.3 ± 8.2 Pg for 0–100 cm depth) were within the range estimated by existing products like Harmonized World Soil Database, HWSD (46.7 Pg for 0–30 cm and 90.7 Pg for 0–100 cm depth) and SoilGrids 2.0 (45.7 Pg for 0–30 cm and 133.0 Pg for 0–100 cm depth). However, independent validation with soil profile data from the National Ecological Observatory Network showed that our approach (R2 = 0.51) outperformed the estimates obtained from Harmonized World Soil Database (R2 = 0.23) and SoilGrids 2.0 (R2 = 0.39) for the topsoil (0–30 cm). Uncertainty analysis (e.g., low representativeness and high coefficients of variation) identified regions requiring more measurements, such as Alaska and the deserts of the U.S. Southwest. Our approach effectively captures the heterogeneous relationships between widely available predictors and the current SOC baseline across regions, offering reliable SOC estimates at 1 km resolution for benchmarking Earth system models.

    more » « less
  2. Soil nitrogen (N) is an important driver of plant productivity and ecosystem functioning; consequently, it is critical to understand its spatial variability from local-to-global scales. Here we provide a quantitative assessment of the three-dimensional spatial distribution of soil N across the conterminous United States (CONUS) using a digital soil mapping (DSM) approach. We used a random forest-regression kriging algorithm to predict soil N concentrations and associated uncertainty across six soil depths (0-5, 5-15, 15-30, 30-60, 60-100, 100-200 cm) at 5 km spatial grids. Across CONUS, there is a strong spatial dependence of soil N, where soil N concentrations decrease but uncertainty increases with soil depth. Soil N was higher in Pacific Northwest, Northeast, and Great Lakes National Ecological Observatory Network (NEON) ecoclimatic domains. Model uncertainty was higher in Atlantic Neotropical, Southern Rockies/Colorado Plateau and Southeast NEON domains. We also compared our soil N predictions with satellite-derived gross primary production (GPP) and forest biomass from the National Biomass and Carbon Dataset. Finally, we used uncertainty information to propose optimized locations for designing future soil surveys and found that the Atlantic Neotropical, Pacific Northwest, Pacific Southwest, and Appalachian/Cumberland Plateau NEON domains may require larger survey efforts. We highlight the need to increase knowledge of biophysical factors regulating soil processes at deeper depths to better characterize the three-dimensional space of soils. Our results provide a national benchmark regarding the spatial variability and uncertainty of soil N and reveal areas in need of a better representation.

    This dataset includes all covariates used for modeling soil Nitrogen, the training data, and the modeling output. The output represents raster files at 5km resolution of soil N at different depths and associated model uncertainty.

    Main reference:

    Smith EM, Guevara M, Tarin T, Pouyat R, Vargas R. Spatial variability and uncertainty of soil nitrogen across the conterminous United States (in review). Ecosphere.

    more » « less
  3. Abstract

    The “4 per 1,000” initiative was launched at the 21st Conference of the Parties (COP21) stimulating a long‐standing debate on the potential of no‐till (NT) to promote soil C sequestration. Previous reviews found little or no soil organic C (SOC) accrual in NT soils as compared with full inversion tillage when soils are sampled deeper than 30 cm. Here, we present the results of a global meta‐analysis of studies assessing SOC and total N (TN) storage and dynamics in NT and tilled soils from the most important agricultural regions of the world. Overall, our results show that NT soils stored 6.7 ± 1.9 Mg C ha–1and 1.1 ± 0.4 Mg N ha–1more than tilled soils (0‐to‐100‐cm depth) with an average of 16 yr of NT, in contrast with previous findings. However, C sequestration (+4.7 ± 1.9 Mg C ha–1in the 0‐to‐60‐cm depth with an average of 11 yr of NT) depended on the association of NT with increased crop frequency and the inclusion of legumes cover crops. Single‐cropping systems lack the necessary C inputs to offset SOC losses in the soil profile (below 30‐cm depth). However, double‐cropping systems decreased soil TN that may constrain future C sequestration. The use of legumes alleviated TN loss and supported soil C sequestration. Briefly, our findings indicate that NT can avoid SOC losses from tilled soils, partially offsetting CO2emissions from agriculture. Moreover, NT with agricultural intensification can promote soil C sequestration, thus contributing to soil quality, food security, and adaptation to climate change.

    more » « less
  4. This dataset includes multiple fields: (i) files for monthly and annual fields for the max curl line and the zero curl line at 0.1 degree longitudinal resolutions; (ii) files for monthly and annual GS path obtained from Altimetry and originally processed by Andres (2016) at 0.1 degree longitudinal resolution. The maximum curl line (MCL) and the zero curl line (ZCL) calculations are briefly described here and are based on the original wind data (at 1.25 x 1.25 degree) provided by the Japanese reanalysis (JRA-55; Kobayashi et al., 2015) and available at (Gifford et al. 2023). For details see Gifford, 2023. 

    The wind stress curl (WSC) fields used for the MCL and ZCL calculations extend from 80W to 45W and 30N to 45N at the 1.25 by 1.25-degree resolution.  The MCL is defined as the maximum WSC values greater than zero within the domain per 1.25 degree longitude. As such, it is a function of longitude and is not a constant WSC value unlike the zero contour. High wind stress curl values that occurred near the coast were not included within this calculation. After MCL at the 1.25 resolution was obtained the line was smoothed with a gaussian smoothing and interpolated on to a 0.1 longitudinal resolution. The smoothed MCL lines at 0.1 degree resolution are provided in separate files for monthly and annual averages (2 files). Similarly, 2 other files (monthly and annual) are provided for the ZCL.    

    Like the MCL, the ZCL is a line derived from 1.25 degree longitude throughout the domain under the condition that it's the line of zero WSC. The ZCL is constant at 0 and does not vary spatially like the MCL. If there are more than one location of zero curl for a given longitude the first location south of the MCL is selected. Similar to the MCL, the ZCL was smoothed with a gaussian smoothing and interpolated on to a 0.1 longitudinal resolution.   

    The above files span the years from 1980 through 2019. So, the monthly files have 480 months starting January 1980, and the annual files have 40 years of data. The files are organized with each row being a new time step and each column being a different longitude. Therefore, the monthly MCL and ZCL files are each 480 x 351 for the 0.1 resolution data. Similarly, the annual files are 40 x 351 for the 0.1 degree resolution data.  

    Note that the monthly MCLs and ZCLs are obtained from the monthly wind-stress curl fields. The annual MCLs and ZCLs are obtained from the annual wind-stress curl fields.

    Since the monthly curl fields preserves more atmospheric mesoscales than the annual curl fields, the 12-month average of the monthly MCLs and ZCLs will not match with the annual MCLs and ZCLs derived from the annual curl field.  The annual MCLs and ZCLs provided here are obtained from the annual curl fields and representative metrics of the wind forcing on an annual time-scale. 

    Furthermore, the monthly Gulf Stream axis path (25 cm isoheight from Altimeter, reprocessed by Andres (2016) technique) from 1993 through 2019 have been made available here. A total of 324 monthly paths of the Gulf Stream are tabulated. In addition, the annual GS paths for these 27 years (1993-2019) of altimetry era have been put together for ease of use. The monthly Gulf Stream paths have been resampled and reprocessed for uniqueness at every 0.1 degree longitude from 75W to 50W and smoothed with a 100 km (10 point) running average via matlab. The uniqueness has been achieved by using Consolidator algorithm (D’Errico, 2023). 

    Each monthly or annual GS path has 251 points between 75W to 50W at 0.1 degree resolution.  

    Please contact for any queries. {"references": ["Andres, M., 2016. On the recent destabilization of the Gulf Stream path downstream of Cape Hatteras. Geophysical Research Letters, 43(18), 9836-9842.", "D'Errico, J., 2023. Consolidator ( 8354-consolidator), MATLAB Central File Exchange. Retrieved June 17, 2023.", "Gifford, Ian. H., 2023. The Synchronicity of the Gulf Stream Free Jet and the Wind Induced Cyclonic Vorticity Pool. MS Thesis, University of Massachusetts Dartmouth. 75pp.", "Gifford, Ian, H., Avijit Gangopadhyay, Magdalena Andres, Glen Gawarkiewicz, Hilde Oliver, Adrienne Silver, 2023. Wind Stress, Wind Stress Curl, and Upwelling Velocities in the Northwest Atlantic (80-45W, 30-45N) during 1980-2019,", "Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H. and Miyaoka, K., 2015. The JRA-55 reanalysis: General specifications and basic characteristics.\u202fJournal of the Meteorological Society of Japan. Ser. II,\u202f93(1), pp.5-48. Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H. and Miyaoka, K., 2015. The JRA-55 reanalysis: General specifications and basic characteristics.\u202fJournal of the Meteorological Society of Japan. Ser. II,\u202f93(1), pp.5-48."]} 
    more » « less
  5. Site description. This data package consists of data obtained from sampling surface soil (the 0-7.6 cm depth profile) in black mangrove (Avicennia germinans) dominated forest and black needlerush (Juncus roemerianus) saltmarsh along the Gulf of Mexico coastline in peninsular west-central Florida, USA. This location has a subtropical climate with mean daily temperatures ranging from 15.4 °C in January to 27.8 °C in August, and annual precipitation of 1336 mm. Precipitation falls as rain primarily between June and September. Tides are semi-diurnal, with 0.57 m median amplitudes during the year preceding sampling (U.S. NOAA National Ocean Service, Clearwater Beach, Florida, station 8726724). Sea-level rise is 4.0 ± 0.6 mm per year (1973-2020 trend, mean ± 95 % confidence interval, NOAA NOS Clearwater Beach station). The A. germinans mangrove zone is either adjacent to water or fringed on the seaward side by a narrow band of red mangrove (Rhizophora mangle). A near-monoculture of J. roemerianus is often adjacent to and immediately landward of the A. germinans zone. The transition from the mangrove to the J. roemerianus zone is variable in our study area. An abrupt edge between closed-canopy mangrove and J. roemerianus monoculture may extend for up to several hundred meters in some locations, while other stretches of ecotone present a gradual transition where smaller, widely spaced trees are interspersed into the herbaceous marsh. Juncus roemerianus then extends landward to a high marsh patchwork of succulent halophytes (including Salicornia bigellovi, Sesuvium sp., and Batis maritima), scattered dwarf mangrove, and salt pans, followed in turn by upland vegetation that includes Pinus sp. and Serenoa repens. Field design and sample collection. We established three study sites spaced at approximately 5 km intervals along the western coastline of the central Florida peninsula. The sites consisted of the Salt Springs (28.3298°, -82.7274°), Energy Marine Center (28.2903°, -82.7278°), and Green Key (28.2530°, -82.7496°) sites on the Gulf of Mexico coastline in Pasco County, Florida, USA. At each site, we established three plot pairs, each consisting of one saltmarsh plot and one mangrove plot. Plots were 50 m^2 in size. Plots pairs within a site were separated by 230-1070 m, and the mangrove and saltmarsh plots composing a pair were 70-170 m apart. All plot pairs consisted of directly adjacent patches of mangrove forest and J. roemerianus saltmarsh, with the mangrove forests exhibiting a closed canopy and a tree architecture (height 4-6 m, crown width 1.5-3 m). Mangrove plots were located at approximately the midpoint between the seaward edge (water-mangrove interface) and landward edge (mangrove-marsh interface) of the mangrove zone. Saltmarsh plots were located 20-25 m away from any mangrove trees and into the J. roemerianus zone (i.e., landward from the mangrove-marsh interface). Plot pairs were coarsely similar in geomorphic setting, as all were located on the Gulf of Mexico coastline, rather than within major sheltering formations like Tampa Bay, and all plot pairs fit the tide-dominated domain of the Woodroffe classification (Woodroffe, 2002, "Coasts: Form, Process and Evolution", Cambridge University Press), given their conspicuous semi-diurnal tides. There was nevertheless some geomorphic variation, as some plot pairs were directly open to the Gulf of Mexico while others sat behind keys and spits or along small tidal creeks. Our use of a plot-pair approach is intended to control for this geomorphic variation. Plot center elevations (cm above mean sea level, NAVD 88) were estimated by overlaying the plot locations determined with a global positioning system (Garmin GPS 60, Olathe, KS, USA) on a LiDAR-derived bare-earth digital elevation model (Dewberry, Inc., 2019). The digital elevation model had a vertical accuracy of ± 10 cm (95 % CI) and a horizontal accuracy of ± 116 cm (95 % CI). Soil samples were collected via coring at low tide in June 2011. From each plot, we collected a composite soil sample consisting of three discrete 5.1 cm diameter soil cores taken at equidistant points to 7.6 cm depth. Cores were taken by tapping a sleeve into the soil until its top was flush with the soil surface, sliding a hand under the core, and lifting it up. Cores were then capped and transferred on ice to our laboratory at the University of South Florida (Tampa, Florida, USA), where they were combined in plastic zipper bags, and homogenized by hand into plot-level composite samples on the day they were collected. A damp soil subsample was immediately taken from each composite sample to initiate 1 y incubations for determination of active C and N (see below). The remainder of each composite sample was then placed in a drying oven (60 °C) for 1 week with frequent mixing of the soil to prevent aggregation and liberate water. Organic wetland soils are sometimes dried at 70 °C, however high drying temperatures can volatilize non-water liquids and oxidize and decompose organic matter, so 50 °C is also a common drying temperature for organic soils (Gardner 1986, "Methods of Soil Analysis: Part 1", Soil Science Society of America); we accordingly chose 60 °C as a compromise between sufficient water removal and avoidance of non-water mass loss. Bulk density was determined as soil dry mass per core volume (adding back the dry mass equivalent of the damp subsample removed prior to drying). Dried subsamples were obtained for determination of soil organic matter (SOM), mineral texture composition, and extractable and total carbon (C) and nitrogen (N) within the following week. Sample analyses. A dried subsample was apportioned from each composite sample to determine SOM as mass loss on ignition at 550 °C for 4 h. After organic matter was removed from soil via ignition, mineral particle size composition was determined using a combination of wet sieving and density separation in 49 mM (3 %) sodium hexametaphosphate ((NaPO_3)_6) following procedures in Kettler et al. (2001, Soil Science Society of America Journal 65, 849-852). The percentage of dry soil mass composed of silt and clay particles (hereafter, fines) was calculated as the mass lost from dispersed mineral soil after sieving (0.053 mm mesh sieve). Fines could have been slightly underestimated if any clay particles were burned off during the preceding ignition of soil. An additional subsample was taken from each composite sample to determine extractable N and organic C concentrations via 0.5 M potassium sulfate (K_2SO_4) extractions. We combined soil and extractant (ratio of 1 g dry soil:5 mL extractant) in plastic bottles, reciprocally shook the slurry for 1 h at 120 rpm, and then gravity filtered it through Fisher G6 (1.6 μm pore size) glass fiber filters, followed by colorimetric detection of nitrite (NO_2^-) + nitrate (NO_3^-) and ammonium (NH_4^+) in the filtrate (Hood Nowotny et al., 2010,Soil Science Society of America Journal 74, 1018-1027) using a microplate spectrophotometer (Biotek Epoch, Winooski, VT, USA). Filtrate was also analyzed for dissolved organic C (referred to hereafter as extractable organic C) and total dissolved N via combustion and oxidation followed by detection of the evolved CO_2 and N oxide gases on a Formacs HT TOC/TN analyzer (Skalar, Breda, The Netherlands). Extractable organic N was then computed as total dissolved N in filtrate minus extractable mineral N (itself the sum of extractable NH_4-N and NO_2-N + NO_3-N). We determined soil total C and N from dried, milled subsamples subjected to elemental analysis (ECS 4010, Costech, Inc., Valencia, CA, USA) at the University of South Florida Stable Isotope Laboratory. Median concentration of inorganic C in unvegetated surface soil at our sites is 0.5 % of soil mass (Anderson, 2019, Univ. of South Florida M.S. thesis via methods in Wang et al., 2011, Environmental Monitoring and Assessment 174, 241-257). Inorganic C concentrations are likely even lower in our samples from under vegetation, where organic matter would dilute the contribution of inorganic C to soil mass. Nevertheless, the presence of a small inorganic C pool in our soils may be counted in the total C values we report. Extractable organic C is necessarily of organic C origin given the method (sparging with HCl) used in detection. Active C and N represent the fractions of organic C and N that are mineralizable by soil microorganisms under aerobic conditions in long-term soil incubations. To quantify active C and N, 60 g of field-moist soil were apportioned from each composite sample, placed in a filtration apparatus, and incubated in the dark at 25 °C and field capacity moisture for 365 d (as in Lewis et al., 2014, Ecosphere 5, art59). Moisture levels were maintained by frequently weighing incubated soil and wetting them up to target mass. Daily CO_2 flux was quantified on 29 occasions at 0.5-3 week intervals during the incubation period (with shorter intervals earlier in the incubation), and these per day flux rates were integrated over the 365 d period to compute an estimate of active C. Observations of per day flux were made by sealing samples overnight in airtight chambers fitted with septa and quantifying headspace CO_2 accumulation by injecting headspace samples (obtained through the septa via needle and syringe) into an infrared gas analyzer (PP Systems EGM 4, Amesbury, MA, USA). To estimate active N, each incubated sample was leached with a C and N free, 35 psu solution containing micronutrients (Nadelhoffer, 1990, Soil Science Society of America Journal 54, 411-415) on 19 occasions at increasing 1-6 week intervals during the 365 d incubation, and then extracted in 0.5 M K_2SO_4 at the end of the incubation in order to remove any residual mineral N. Active N was then quantified as the total mass of mineral N leached and extracted. Mineral N in leached and extracted solutions was detected as NH_4-N and NO_2-N + NO_3-N via colorimetry as above. This incubation technique precludes new C and N inputs and persistently leaches mineral N, forcing microorganisms to meet demand by mineralizing existing pools, and thereby directly assays the potential activity of soil organic C and N pools present at the time of soil sampling. Because this analysis commences with disrupting soil physical structure, it is biased toward higher estimates of active fractions. Calculations. Non-mobile C and N fractions were computed as total C and N concentrations minus the extractable and active fractions of each element. This data package reports surface-soil constituents (moisture, fines, SOM, and C and N pools and fractions) in both gravimetric units (mass constituent / mass soil) and areal units (mass constituent / soil surface area integrated through 7.6 cm soil depth, the depth of sampling). Areal concentrations were computed as X × D × 7.6, where X is the gravimetric concentration of a soil constituent, D is soil bulk density (g dry soil / cm^3), and 7.6 is the sampling depth in cm. 
    more » « less