With the advances of satellite remote sensing techniques, we are receiving huge amount of satellite observation data for the Earth. While the data greatly helps Earth scientists on their research, conduct- ing data processing and analytics from the data is getting more and more time consuming and complicated. One common data processing task is to aggregate satellite observation data from original pixel level to latitude-longitude grid level to easily obtain global information and work with global climate models. This paper focuses on how to best aggregate NASA MODIS satellite data products from pixel level to grid level in a distributed environment and provision the aggregation capa- bility as a service for Earth scientists to use easily. We propose three different approaches of parallel data aggregation and employ three par- allel platforms (Spark, Dask and MPI) to implement the approaches. We run extensive experiments based on these parallel approaches and platforms on a local cluster to benchmark their differences in execution performance and discuss key factors to achieve good speedup. We also study how to make the provisioned service adaptable to different service libraries and protocols via a unified framework.
more »
« less
Efficient and Flexible Aggregation and Distribution of MODIS Atmospheric Products Based on Climate Analytics as a Service Framework
MODIS (Moderate Resolution Imaging Spectroradiometer) is a key instrument onboard NASA’s Terra (launched in 1999) and Aqua (launched in 2002) satellite missions as part of the more extensive Earth Observation System (EOS). By measuring the reflection and emission by the Earth-Atmosphere system in 36 spectral bands from the visible to thermal infrared with near-daily global coverage and high-spatial-resolution (250 m ~ 1 km at nadir), MODIS is playing a vital role in developing validated, global, interactive Earth system models. MODIS products are processed into three levels, i.e., Level-1 (L1), Level-2 (L2) and Level-3 (L3). To shift the current static and “one-size-fits-all” data provision method of MODIS products, in this paper, we propose a service-oriented flexible and efficient MODIS aggregation framework. Using this framework, users only need to get aggregated MODIS L3 data based on their unique requirements and the aggregation can run in parallel to achieve a speedup. The experiments show that our aggregation results are almost identical to the current MODIS L3 products and our parallel execution with 8 computing nodes can work 88.63 times faster than a serial code execution on a single node.
more »
« less
- NSF-PAR ID:
- 10301881
- Date Published:
- Journal Name:
- Remote Sensing
- Volume:
- 13
- Issue:
- 17
- ISSN:
- 2072-4292
- Page Range / eLocation ID:
- 3541
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Surface albedo is a fundamental radiative parameter as it controls the Earth’s energy budget and directly affects the Earth’s climate. Satellite observations have long been used to capture the temporal and spatial variations of surface albedo because of their continuous global coverage. However, space-based albedo products are often affected by errors in the atmospheric correction, multi-angular bi-directional reflectance distribution function (BRDF) modelling, as well as spectral conversions. To validate space-based albedo products, an in situ tower albedometer is often used to provide continuous “ground truth” measurements of surface albedo over an extended area. Since space-based albedo and tower-measured albedo are produced at different spatial scales, they can be directly compared only for specific homogeneous land surfaces. However, most land surfaces are inherently heterogeneous with surface properties that vary over a wide range of spatial scales. In this work, tower-measured albedo products, including both directional hemispherical reflectance (DHR) and bi-hemispherical reflectance (BHR), are upscaled to coarse satellite spatial resolutions using a new method. This strategy uses high-resolution satellite derived surface albedos to fill the gaps between the albedometer’s field-of-view (FoV) and coarse satellite scales. The high-resolution surface albedo is generated from a combination of surface reflectance retrieved from high-resolution Earth Observation (HR-EO) data and moderate resolution imaging spectroradiometer (MODIS) BRDF climatology over a larger area. We implemented a recently developed atmospheric correction method, the Sensor Invariant Atmospheric Correction (SIAC), to retrieve surface reflectance from HR-EO (e.g., Sentinel-2 and Landsat-8) top-of-atmosphere (TOA) reflectance measurements. This SIAC processing provides an estimated uncertainty for the retrieved surface spectral reflectance at the HR-EO pixel level and shows excellent agreement with the standard Landsat 8 Surface Reflectance Code (LaSRC) in retrieving Landsat-8 surface reflectance. Atmospheric correction of Sentinel-2 data is vastly improved by SIAC when compared against the use of in situ AErosol RObotic NETwork (AERONET) data. Based on this, we can trace the uncertainty of tower-measured albedo during its propagation through high-resolution EO measurements up to coarse satellite scales. These upscaled albedo products can then be compared with space-based albedo products over heterogeneous land surfaces. In this study, both tower-measured albedo and upscaled albedo products are examined at Ground Based Observation for Validation (GbOV) stations (https://land.copernicus.eu/global/gbov/), and used to compare with satellite observations, including Copernicus Global Land Service (CGLS) based on ProbaV and VEGETATION 2 data, MODIS and multi-angle imaging spectroradiometer (MISR).more » « less
-
Abstract. Land surface temperature (LST) is one of the most important and widely used parameters for studying land surface processes. Moderate ResolutionImaging Spectroradiometer (MODIS) LST products (e.g., MOD11A1 and MYD11A1) can provide this information with moderate spatiotemporal resolution withglobal coverage. However, the applications of these data are hampered because of missing values caused by factors such as cloud contamination,indicating the necessity to produce a seamless global MODIS-like LST dataset, which is still not available. In this study, we used a spatiotemporalgap-filling framework to generate a seamless global 1 km daily (mid-daytime and mid-nighttime) MODIS-like LST dataset from 2003 to 2020based on standard MODIS LST products. The method includes two steps: (1) data pre-processing and (2) spatiotemporal fitting. In the datapre-processing, we filtered pixels with low data quality and filled gaps using the observed LST at another three time points of the same day. In thespatiotemporal fitting, first we fitted the temporal trend (overall mean) of observations based on the day of year (independent variable) in eachpixel using the smoothing spline function. Then we spatiotemporally interpolated residuals between observations and overall mean values for eachday. Finally, we estimated missing values of LST by adding the overall mean and interpolated residuals. The results show that the missing values inthe original MODIS LST were effectively and efficiently filled with reduced computational cost, and there is no obvious block effect caused by largeareas of missing values, especially near the boundary of tiles, which might exist in other seamless LST datasets. The cross-validation withdifferent missing rates at the global scale indicates that the gap-filled LST data have high accuracies with the average root mean squared error(RMSE) of 1.88 and 1.33∘, respectively, for mid-daytime (13:30) and mid-nighttime (01:30). The seamless global daily (mid-daytime andmid-nighttime) LST dataset at a 1 km spatial resolution is of great use in global studies of urban systems, climate research and modeling,and terrestrial ecosystem studies. The data are available at Iowa State University's DataShare at https://doi.org/10.25380/iastate.c.5078492 (T. Zhanget al., 2021).more » « less
-
more » « less
Abstract Accurate and timely inland waterbody extent and location data are foundational information to support a variety of hydrological applications and water resources management. Recently, the Cyclone Global Navigation Satellite System (CYGNSS) has emerged as a promising tool for delineating inland water due to distinct surface reflectivity characteristics over dry versus wet land which are observable by CYGNSS’s eight microsatellites with passive bistatic radars that acquire reflected L-band signals from the Global Positioning System (GPS) (i.e., signals of opportunity). This study conducts a baseline 1-km comparison of water masks for the contiguous United States between latitudes of 24°N-37°N for 2019 using three Earth observation systems: CYGNSS (i.e., our baseline water mask data), the Moderate Resolution Imaging Spectroradiometer (MODIS) (i.e., land water mask data), and the Landsat Global Surface Water product (i.e., Pekel data). Spatial performance of the 1-km comparison water mask was assessed using confusion matrix statistics and optical high-resolution commercial satellite imagery. When a mosaic of binary thresholds for 8 sub-basins for CYGNSS data were employed, confusion matrix statistics were improved such as up to a 34% increase in F1-score. Further, a performance metric of ratio of inland water to catchment area showed that inland water area estimates from CYGNSS, MODIS, and Landsat were within 2.3% of each other regardless of the sub-basin observed. Overall, this study provides valuable insight into the spatial similarities and discrepancies of inland water masks derived from optical (visible) versus radar (Global Navigation Satellite System Reflectometry, GNSS-R) based satellite Earth observations.
-
more » « less
Agricultural irrigation has significant potential for altering local climate by reducing soil albedo, increasing evapotranspiration, and enabling greater leaf area. Numerous studies using regional or global climate models have demonstrated the cooling effects of irrigation on mean and extreme temperature, especially over regions where irrigation is extensive. However, these model‐based results have not been well validated due to the limitations of observational data sets. In this study, multiple satellite‐based products, including the Moderate Resolution Imaging Spectroradiometer (MODIS) and Soil Moisture Active Passive (SMAP) data sets, are used to isolate and quantify the local impacts of irrigation on surface climate over irrigated regions, which are derived from the Global Map of Irrigation Areas (GMIA). The relationships among soil moisture, albedo, evapotranspiration, and surface temperature are explored. Strong evaporative cooling by irrigation lowers daytime surface temperature over arid and semi‐arid regions, such as California's Central Valley, the Great Plains, central Asia, and northwestern India. However, the cooling effects are less evident in areas of eastern China and the Lower Mississippi River Basin despite extensive irrigation over these regions. Results are also compared with irrigation experiments using the Community Earth System Model (CESM) to assess the model's ability to represent land–atmosphere interactions in regards to irrigation. CESM greatly underestimates the surface temperature response to irrigation. The comparison between the offline and coupled simulations suggests that the irrigation‐induced cooling can be regulated by the interactions between land surface and atmosphere, and amplified signals are found over the “hot spot” regions. Meanwhile, model resolution can also influence the magnitude of the local cooling by irrigation.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/rs13173541
