More Like this

1. Transferability of the Deep Learning Mask R-CNN Model for Automated Mapping of Ice-Wedge Polygons in High-Resolution Satellite and UAV Images

   https://doi.org/10.3390/rs12071085  

   Zhang, Weixing; Liljedahl, Anna K.; Kanevskiy, Mikhail; Epstein, Howard E.; Jones, Benjamin M.; Jorgenson, M. Torre; Kent, Kelcy (April 2020, Remote Sensing)

   State-of-the-art deep learning technology has been successfully applied to relatively small selected areas of very high spatial resolution (0.15 and 0.25 m) optical aerial imagery acquired by a fixed-wing aircraft to automatically characterize ice-wedge polygons (IWPs) in the Arctic tundra. However, any mapping of IWPs at regional to continental scales requires images acquired on different sensor platforms (particularly satellite) and a refined understanding of the performance stability of the method across sensor platforms through reliable evaluation assessments. In this study, we examined the transferability of a deep learning Mask Region-Based Convolutional Neural Network (R-CNN) model for mapping IWPs in satellite remote sensing imagery (~0.5 m) covering 272 km2 and unmanned aerial vehicle (UAV) (0.02 m) imagery covering 0.32 km2. Multi-spectral images were obtained from the WorldView-2 satellite sensor and pan-sharpened to ~0.5 m, and a 20 mp CMOS sensor camera onboard a UAV, respectively. The training dataset included 25,489 and 6022 manually delineated IWPs from satellite and fixed-wing aircraft aerial imagery near the Arctic Coastal Plain, northern Alaska. Quantitative assessments showed that individual IWPs were correctly detected at up to 72% and 70%, and delineated at up to 73% and 68% F1 score accuracy levels for satellite and UAV images, respectively. Expert-based qualitative assessments showed that IWPs were correctly detected at good (40–60%) and excellent (80–100%) accuracy levels for satellite and UAV images, respectively, and delineated at excellent (80–100%) level for both images. We found that (1) regardless of spatial resolution and spectral bands, the deep learning Mask R-CNN model effectively mapped IWPs in both remote sensing satellite and UAV images; (2) the model achieved a better accuracy in detection with finer image resolution, such as UAV imagery, yet a better accuracy in delineation with coarser image resolution, such as satellite imagery; (3) increasing the number of training data with different resolutions between the training and actual application imagery does not necessarily result in better performance of the Mask R-CNN in IWPs mapping; (4) and overall, the model underestimates the total number of IWPs particularly in terms of disjoint/incomplete IWPs. 

   more » « less
2. An integrated cyberGIS and machine learning framework for fine-scale prediction of Urban Heat Island using satellite remote sensing and urban sensor network data

   https://doi.org/10.1007/s44212-022-00002-4  

   Lyu, Fangzheng; Wang, Shaohua; Han, Su Yeon; Catlett, Charlie; Wang, Shaowen (September 2022, Urban Informatics)

   Abstract Due to climate change and rapid urbanization, Urban Heat Island (UHI), featuring significantly higher temperature in metropolitan areas than surrounding areas, has caused negative impacts on urban communities. Temporal granularity is often limited in UHI studies based on satellite remote sensing data that typically has multi-day frequency coverage of a particular urban area. This low temporal frequency has restricted the development of models for predicting UHI. To resolve this limitation, this study has developed a cyber-based geographic information science and systems (cyberGIS) framework encompassing multiple machine learning models for predicting UHI with high-frequency urban sensor network data combined with remote sensing data focused on Chicago, Illinois, from 2018 to 2020. Enabled by rapid advances in urban sensor network technologies and high-performance computing, this framework is designed to predict UHI in Chicago with fine spatiotemporal granularity based on environmental data collected with the Array of Things (AoT) urban sensor network and Landsat-8 remote sensing imagery. Our computational experiments revealed that a random forest regression (RFR) model outperforms other models with the prediction accuracy of 0.45 degree Celsius in 2020 and 0.8 degree Celsius in 2018 and 2019 with mean absolute error as the evaluation metric. Humidity, distance to geographic center, and PM_2.5concentration are identified as important factors contributing to the model performance. Furthermore, we estimate UHI in Chicago with 10-min temporal frequency and 1-km spatial resolution on the hottest day in 2018. It is demonstrated that the RFR model can accurately predict UHI at fine spatiotemporal scales with high-frequency urban sensor network data integrated with satellite remote sensing data. 

   more » « less
3. Auto-CM: Unsupervised Deep Learning for Satellite Imagery Composition and Cloud Masking Using Spatio-Temporal Dynamics

   https://doi.org/10.1609/aaai.v37i12.26704  

   Xie, Yiqun; Li, Zhili; Bao, Han; Jia, Xiaowei; Xu, Dongkuan; Zhou, Xun; Skakun, Sergii (June 2023, Proceedings of the AAAI Conference on Artificial Intelligence)

   Cloud masking is both a fundamental and a critical task in the vast majority of Earth observation problems across social sectors, including agriculture, energy, water, etc. The sheer volume of satellite imagery to be processed has fast-climbed to a scale (e.g., >10 PBs/year) that is prohibitive for manual processing. Meanwhile, generating reliable cloud masks and image composite is increasingly challenging due to the continued distribution-shifts in the imagery collected by existing sensors and the ever-growing variety of sensors and platforms. Moreover, labeled samples are scarce and geographically limited compared to the needs in real large-scale applications. In related work, traditional remote sensing methods are often physics-based and rely on special spectral signatures from multi- or hyper-spectral bands, which are often not available in data collected by many -- and especially more recent -- high-resolution platforms. Machine learning and deep learning based methods, on the other hand, often require large volumes of up-to-date training data to be reliable and generalizable over space. We propose an autonomous image composition and masking (Auto-CM) framework to learn to solve the fundamental tasks in a label-free manner, by leveraging different dynamics of events in both geographic domains and time-series. Our experiments show that Auto-CM outperforms existing methods on a wide-range of data with different satellite platforms, geographic regions and bands. 

   more » « less
4. High-sensitivity nanophotonic sensors with passive trapping of analyte molecules in hot spots

   https://doi.org/10.1038/s41377-020-00449-7  

   Miao, Xianglong; Yan, Lingyue; Wu, Yun; Liu, Peter Q. (January 2021, Light: Science & Applications)

   Abstract Nanophotonic resonators can confine light to deep-subwavelength volumes with highly enhanced near-field intensity and therefore are widely used for surface-enhanced infrared absorption spectroscopy in various molecular sensing applications. The enhanced signal is mainly contributed by molecules in photonic hot spots, which are regions of a nanophotonic structure with high-field intensity. Therefore, delivery of the majority of, if not all, analyte molecules to hot spots is crucial for fully utilizing the sensing capability of an optical sensor. However, for most optical sensors, simple and straightforward methods of introducing an aqueous analyte to the device, such as applying droplets or spin-coating, cannot achieve targeted delivery of analyte molecules to hot spots. Instead, analyte molecules are usually distributed across the entire device surface, so the majority of the molecules do not experience enhanced field intensity. Here, we present a nanophotonic sensor design with passive molecule trapping functionality. When an analyte solution droplet is introduced to the sensor surface and gradually evaporates, the device structure can effectively trap most precipitated analyte molecules in its hot spots, significantly enhancing the sensor spectral response and sensitivity performance. Specifically, our sensors produce a reflection change of a few percentage points in response to trace amounts of the amino-acid proline or glucose precipitate with a picogram-level mass, which is significantly less than the mass of a molecular monolayer covering the same measurement area. The demonstrated strategy for designing optical sensor structures may also be applied to sensing nano-particles such as exosomes, viruses, and quantum dots. 

   more » « less
5. Multispectral Band-Aware Generation of Satellite Images across Domains Using Generative Adversarial Networks and Contrastive Learning

   https://doi.org/10.3390/rs16071154  

   Mahara, Arpan; Rishe, Naphtali (April 2024, Remote Sensing)

   Generative models have recently gained popularity in remote sensing, offering substantial benefits for interpreting and utilizing satellite imagery across diverse applications such as climate monitoring, urban planning, and wildfire detection. These models are particularly adept at addressing the challenges posed by satellite images, which often exhibit domain variability due to seasonal changes, sensor characteristics, and, especially, variations in spectral bands. Such variability can significantly impact model performance across various tasks. In response to these challenges, our work introduces an adaptive approach that harnesses the capabilities of generative adversarial networks (GANs), augmented with contrastive learning, to generate target domain images that account for multispectral band variations effectively. By maximizing mutual information between corresponding patches and leveraging the power of GANs, our model aims to generate realistic-looking images across different multispectral domains. We present a comparative analysis of our model against other well-established generative models, demonstrating its efficacy in generating high-quality satellite images while effectively managing domain variations inherent to multispectral diversity. 

   more » « less