An official website of the United States government
Abstract Accurate cloud type identification and coverage analysis are crucial in understanding the Earth’s radiative budget. Traditional computer vision methods rely on low-level visual features of clouds for estimating cloud coverage or sky conditions. Several handcrafted approaches have been proposed; however, scope for improvement still exists. Newer deep neural networks (DNNs) have demonstrated superior performance for cloud segmentation and categorization. These methods, however, need expert engineering intervention in the preprocessing steps—in the traditional methods—or human assistance in assigning cloud or clear sky labels to a pixel for training DNNs. Such human mediation imposes considerable time and labor costs. We present the application of a new self-supervised learning approach to autonomously extract relevant features from sky images captured by ground-based cameras, for the classification and segmentation of clouds. We evaluate a joint embedding architecture that uses self-knowledge distillation plus regularization. We use two datasets to demonstrate the network’s ability to classify and segment sky images—one with ∼ 85,000 images collected from our ground-based camera and another with 400 labeled images from the WSISEG database. We find that this approach can discriminate full-sky images based on cloud coverage, diurnal variation, and cloud base height. Furthermore, it semantically segments the cloud areas without labels. The approach shows competitive performance in all tested tasks,suggesting a new alternative for cloud characterization.
more »
« less
Cloud Region Segmentation from All Sky Images using Double K-Means Clustering
The segmentation of sky images into regions of cloud and clear sky allows atmospheric scientists to determine the fraction of cloud cover and the distribution of cloud without resorting to subjective estimates by a human observer. This is a challenging problem because cloud boundaries and cirroform cloud regions are often semi-transparent and indistinct. In this study, we propose a lightweight, unsupervised methodology to identify cloud regions in ground-based hemispherical sky images. Our method offers a fast and adaptive approach without the necessity of fixed thresholds by utilizing K-means clustering on transformed pixel values. We present the results of our method for two data sets and compare them with three different methods in the literature.
more »
« less
- PAR ID:
- 10433836
- Date Published:
- Journal Name:
- 2022 IEEE International Symposium on Multimedia (ISM)
- Page Range / eLocation ID:
- 261 to 262
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Cloud cover estimation from images taken by sky-facing cameras can be an important input for analyzing current weather conditions and estimating photovoltaic power generation. The constant change in position, shape, and density of clouds, however, makes the development of a robust computational method for cloud cover estimation challenging. Accurately determining the edge of clouds and hence the separation between clouds and clear sky is difficult and often impossible. Toward determining cloud cover for estimating photovoltaic output, we propose using machine learning methods for cloud segmentation. We compare several methods including a classical regression model, deep learning methods, and boosting methods that combine results from the other machine learning models. To train each of the machine learning models with various sky conditions, we supplemented the existing Singapore whole sky imaging segmentation database with hazy and overcast images collected by a camera-equipped Waggle sensor node. We found that the U-Net architecture, one of the deep neural networks we utilized, segmented cloud pixels most accurately. However, the accuracy of segmenting cloud pixels did not guarantee high accuracy of estimating solar irradiance. We confirmed that the cloud cover ratio is directly related to solar irradiance. Additionally, we confirmed that solar irradiance and solar power output are closely related; hence, by predicting solar irradiance, we can estimate solar power output. This study demonstrates that sky-facing cameras with machine learning methods can be used to estimate solar power output. This ground-based approach provides an inexpensive way to understand solar irradiance and estimate production from photovoltaic solar facilities.more » « less
-
Hyperspectral cameras collect detailed spectral information at each image pixel, contributing to the identification of image features. The rich spectral content of hyperspectral imagery has led to its application in diverse fields of study. This study focused on cloud classification using a dataset of hyperspectral sky images captured by a Resonon PIKA XC2 camera. The camera records images using 462 spectral bands, ranging from 400 to 1000 nm, with a spectral resolution of 1.9 nm. Our preliminary/unlabeled dataset comprised 33 parent hyperspectral images (HSI), each a substantial unlabeled image measuring 4402-by-1600 pixels. With the meteorological expertise within our team, we manually labeled pixels by extracting 10 to 20 sample patches from each parent image, each patch consisting of a 50-by-50 pixel field. This process yielded a collection of 444 patches, each categorically labeled into one of seven cloud and sky condition categories. To embed the inherent data structure while classifying individual pixels, we introduced an innovative technique to boost classification accuracy by incorporating patch-specific information into each pixel’s feature vector. The posterior probabilities generated by these classifiers, which capture the unique attributes of each patch, were subsequently concatenated with the pixel’s original spectral data to form an augmented feature vector. We then applied a final classifier to map the augmented vectors to the seven cloud/sky categories. The results compared favorably to the baseline model devoid of patch-origin embedding, showing that incorporating the spatial context along with the spectral information inherent in hyperspectral images enhances the classification accuracy in hyperspectral cloud classification. The dataset is available on IEEE DataPort.more » « less
-
Abstract. In humid tropical regions, irregular illumination and cloud shadows can complicate near-surface optical remote sensing. This could lead to costly and repetitive surveys to maintain geographical and spectral consistency. This could have a significant impact on the regular monitoring of forest ecosystems. A novel correction method using deep learning is presented here to address the issue in high-resolution canopy images. Our method involves training a deep learning model on one or a few well-illuminated/homogeneous reference images augmented with artificially generated cloud shadows. This training enables the model to predict illumination and cloud shadow patterns in any image and ultimately mitigate these effects. Using images captured by multispectral and RGB cameras, we evaluated the method across multiple sensors and conditions. These included nadir-view images from two sensors mounted on a drone and tower-mounted RGB Phenocams. The technique effectively corrects uneven illumination in near-infrared and true-color RGB images, including non-forested areas. This improvement was also evident in more consistent normalized difference vegetation index (NDVI) patterns in areas affected by uneven illumination. By comparing corrected RGB images to the original in a binary classification task, we evaluated the method's accuracy and Kappa values. Our goal was to detect non-photosynthetic vegetation (NPV) in a mosaic. The overall accuracy and Kappa were both significantly improved in corrected images, with a 2.5% and 1.1% increase, respectively. Moreover, the method can be generalized across sensors and conditions. Further work should focus on refining the technique and exploring its applicability to satellite imagery and beyond.more » « less
-
A novel hyperspectral image classification algorithm is proposed and demonstrated on benchmark hyperspectral images. We also introduce a hyperspectral sky imaging dataset that we are collecting for detecting the amount and type of cloudiness. The algorithm designed to be applied to such systems could improve the spatial and temporal resolution of cloud information vital to understanding Earth’s climate. We discuss the nature of our HSI-Cloud dataset being collected and an algorithm we propose for processing the dataset using a categorical-boosting method. The proposed method utilizes multiple clusterings to augment the dataset and achieves higher pixel classification accuracy. Creating categorical features via clustering enriches the data representation and improves boosting ensembles. For the experimental datasets used in this paper, gradient boosting methods performed favorably to the benchmark algorithms.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ISM55400.2022.00058
