skip to main content


Title: Attention-augmented Spatio-Temporal Segmentation for Land Cover Mapping
The availability of massive earth observing satellite data provides huge opportunities for land use and land cover mapping. However, such mapping effort is challenging due to the existence of various land cover classes, noisy data, and the lack of proper labels. Also, each land cover class typically has its own unique temporal pattern and can be identified only during certain periods. In this article, we introduce a novel architecture that incorporates the UNet structure with a Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to better identify the unique temporal patterns of each land cover class. We compare our method with other state-of-the-art methods both quantitatively and qualitatively on two real-world datasets which involve multiple land cover classes. We also visualize the attention weights to study its effectiveness in mitigating noise and in identifying discriminative time periods of different classes. The code and dataset used in this work are made publicly available for reproducibility.  more » « less
Award ID(s):
1838159
NSF-PAR ID:
10346459
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
2021 IEEE International Conference on Big Data (Big Data)
Page Range / eLocation ID:
1399 to 1408
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    The availability of massive earth observing satellite data provides huge opportunities for land use and land cover mapping. However, such mapping effort is challenging due to the existence of various land cover classes, noisy data, and the lack of proper labels. Also, each land cover class typically has its own unique temporal pattern and can be identified only during certain periods. In this article, we introduce a novel architecture that incorporates the UNet structure with Bidirectional LSTM and Attention mechanism to jointly exploit the spatial and temporal nature of satellite data and to better identify the unique temporal patterns of each land cover class. We compare our method with other state-of-the-art methods both quantitatively and qualitatively on two real-world datasets which involve multiple land cover classes. We also visualize the attention weights to study its effectiveness in mitigating noise and in identifying discriminative time periods of different classes. The code and dataset used in this work are made publicly available for reproducibility. 
    more » « less
  2. Expansion of large-scale tree plantations for commodity crop and timber production is a leading cause of tropical deforestation. While automated detection of plantations across large spatial scales and with high temporal resolution is critical to inform policies to reduce deforestation, such mapping is technically challenging. Thus, most available plantation maps rely on visual inspection of imagery, and many of them are limited to small areas for specific years. Here, we present an automated approach, which we call Plantation Analysis by Learning from Multiple Classes (PALM), for mapping plantations on an annual basis using satellite remote sensing data. Due to the heterogeneity of land cover classes, PALM utilizes ensemble learning to simultaneously incorporate training samples from multiple land cover classes over different years. After the ensemble learning, we further improve the performance by post-processing using a Hidden Markov Model. We implement the proposed automated approach using MODIS data in Sumatra and Indonesian Borneo (Kalimantan). To validate the classification, we compare plantations detected using our approach with existing datasets developed through visual interpretation. Based on random sampling and comparison with high-resolution images, the user’s accuracy and producer’s accuracy of our generated map are around 85% and 80% in our study region. 
    more » « less
  3. This dataset incorporates Mexico City related essential data files associated with Beth Tellman's dissertation: Mapping and Modeling Illicit and Clandestine Drivers of Land Use Change: Urban Expansion in Mexico City and Deforestation in Central America. It contains spatio-temporal datasets covering three domains; i) urban expansion from 1992-2015, ii) district and section electoral records for 6 elections from 2000-2015, iii) land titling (regularization) data for informal settlements from 1997-2012 on private and ejido land. The urban expansion data includes 30m resolution urban land cover for 1992 and 2013 (methods published in Goldblatt et al 2018), and a shapefile of digitized urban informal expansion in conservation land from 2000-2015 using the Worldview-2 satellite. The electoral records include shapefiles with the geospatial boundaries of electoral districts and sections for each election, and .csv files of the number of votes per party for mayoral, delegate, and legislature candidates. The private land titling data includes the approximate (in coordinates) location and date of titles given by the city government (DGRT) extracted from public records (Diario Oficial) from 1997-2012. The titling data on ejido land includes a shapefile of georeferenced polygons taken from photos in the CORETT office or ejido land that has been expropriated by the government, and including an accompany .csv from the National Agrarian Registry detailing the date and reason for expropriation from 1987-2007. Further details are provided in the dissertation and subsequent article publication (Tellman et al 2021). The Mexico City portion of these data were generated via a National Science Foundation sponsored project (No. 1657773, DDRI: Mapping and Modeling Clandestine Drivers of Urban Expansion in Mexico City). The project P.I. is Beth Tellman with collaborators at ASU (B.L Turner II and Hallie Eakin). Other collaborators include the National Autonomous University of Mexico (UNAM), at the Institute of Geography via Dr. Armando Peralta Higuera, who provided support for two students, Juan Alberto Guerra Moreno and Kimberly Mendez Gomez for validating the Landsat urbanization algorithm. Fidel Serrano-Candela, at the UNAM Laboratory of the National Laboratory for Sustainability Sciences (LANCIS) also provided support for urbanization algorithm development and validation, and Rodrigo Garcia Herrera, who provided support for hosting data at LANCIS (at: http://patung.lancis.ecologia.unam.mx/tellman/). Additional collaborators include Enrique Castelán, who provided support for the informal urbanization data from SEDEMA (Ministry of the Environmental for Mexico City). Electoral, land titling, and land zoning data were digitized with support from Juana Martinez, Natalia Hernandez, Alexia Macario Sanchez, Enrique Ruiz Durazo, in collaboration with Felipe de Alba, at CESOP (Center of Social Studies and Public Opinion, at the Mexican Legislative Assembly). The data include geospatial time series data regarding changes in urban land cover, digitized electoral results, land titling, land zoning, and public housing. Additional funding for this work was provided by NSF under Grant No. 1414052, CNH: The Dynamics of Multiscalar Adaptation in Megacities (PI H. Eakin), and the NSF-CONACYT GROW fellowship NSF No. 026257-001 and CONACYT number 291303 (PI Bojórquez). References: Tellman, B., Eakin, H., Janssen, M.A., Alba, F. De, Ii, B.L.T., 2021. The Role of Institutional Entrepreneurs and Informal Land Transactions in Mexico City’s Urban Expansion. World Dev. 140, 1–44. https://doi.org/10.1016/j.worlddev.2020.105374 Goldblatt, R., Stuhlmacher, M.F., Tellman, B., Clinton, N., Hanson, G., Georgescu, M., Wang, C., Serrano-Candela, F., Khandelwal, A.K., Cheng, W.-H., Balling, R.C., 2018. Using Landsat and nighttime lights for supervised pixel-based image classification of urban land cover. Remote Sens. Environ. 205, 253–275. https://doi.org/10.1016/j.rse.2017.11.026 
    more » « less
  4. Mapping crop types and land cover in smallholder farming systems in sub-Saharan Africa remains a challenge due to data costs, high cloud cover, and poor temporal resolution of satellite data. With improvement in satellite technology and image processing techniques, there is a potential for integrating data from sensors with different spectral characteristics and temporal resolutions to effectively map crop types and land cover. In our Malawi study area, it is common that there are no cloud-free images available for the entire crop growth season. The goal of this experiment is to produce detailed crop type and land cover maps in agricultural landscapes using the Sentinel-1 (S-1) radar data, Sentinel-2 (S-2) optical data, S-2 and PlanetScope data fusion, and S-1 C2 matrix and S-1 H/α polarimetric decomposition. We evaluated the ability to combine these data to map crop types and land cover in two smallholder farming locations. The random forest algorithm, trained with crop and land cover type data collected in the field, complemented with samples digitized from Google Earth Pro and DigitalGlobe, was used for the classification experiments. The results show that the S-2 and PlanetScope fused image + S-1 covariance (C2) matrix + H/α polarimetric decomposition (an entropy-based decomposition method) fusion outperformed all other image combinations, producing higher overall accuracies (OAs) (>85%) and Kappa coefficients (>0.80). These OAs represent a 13.53% and 11.7% improvement on the Sentinel-2-only (OAs < 80%) experiment for Thimalala and Edundu, respectively. The experiment also provided accurate insights into the distribution of crop and land cover types in the area. The findings suggest that in cloud-dense and resource-poor locations, fusing high temporal resolution radar data with available optical data presents an opportunity for operational mapping of crop types and land cover to support food security and environmental management decision-making. 
    more » « less
  5. null (Ed.)
    Abstract. The empirical attribution of hydrologic change presents a unique data availability challenge in terms of establishing baseline prior conditions, as one cannot go back in time to retrospectively collect the necessary data. Although global remote sensing data can alleviate this challenge, most satellite missions are too recent to capture changes that happened long ago enough to provide sufficient observations for adequate statistical inference. In that context, the 4 decades of continuous global high-resolution monitoring enabled by the Landsat missions are an unrivaled source of information. However, constructing a time series of land cover observation across Landsat missions remains a significant challenge because cloud masking and inconsistent image quality complicate the automatized interpretation of optical imagery. Focusing on the monitoring of lake water extent, we present an automatized gap-filling approach to infer the class (wet or dry) of pixels masked by clouds or sensing errors. The classification outcome of unmasked pixels is compiled across images taken on different dates to estimate the inundation frequency of each pixel, based on the assumption that different pixels are masked at different times. The inundation frequency is then used to infer the inundation status of masked pixels on individual images through supervised classification. Applied to a variety of global lakes with substantial long term or seasonal fluctuations, the approach successfully captured water extent variations obtained from in situ gauges (where applicable), or from other Landsat missions during overlapping time periods. Although sensitive to classification errors in the input imagery, the gap-filling algorithm is straightforward to implement on Google's Earth Engine platform and stands as a scalable approach to reliably monitor, and ultimately attribute, historical changes in water bodies. 
    more » « less