An official website of the United States government
Abstract Classification of snowflakes based on their geometric shape, degree of riming, and melt/dry state can improve understanding, characterization, and quantification of other geometrical, microphysical, and scattering properties of ice particles. For example, classification provides essential ground-truth data for interpreting polarimetric radar signatures of snow while validating and advancing radar-based quantitative precipitation estimation. High-resolution photographs of snowflakes obtained by emerging multicamera instruments are well suited for snowflake classification, which, coupled with recent machine learning techniques based on convolutional neural networks (CNNs), enable methods for accurate and fast automatic classification of snowflakes using images. Given that the appearance of a snowflake generally changes significantly with viewing angle, this work proposes and presents a novel multiview snowflake classification methodology based on the high-resolution photographs of frozen hydrometeors in free-fall from multiple views collected by the multicamera instruments. The approach employs machine/deep learning algorithms leveraging multiangle camera systems and enhanced supervised CNN-based techniques to achieve precise classification of snowflakes based on their geometrical categories and accurate and reliable estimates of specific snowflake properties, such as riming degree and melt/dry state. This represents the first multiview snowflake classification framework that takes full advantage of multiview camera systems. Presented multiview classification results show record accuracies of 98.57%, 98.22%, and 95.83% for geometric classes, riming degree, and melt/dry state, respectively. Significance StatementThis work proposes and presents a novel multiview snowflake classification methodology leveraging recent developments in machine learning, multiview classification, and multiangle multicamera instruments for acquiring high-resolution photographs of frozen hydrometeors in free-fall from multiple views. The results for multiview classification show record accuracies for snowflake geometric classification, riming degree estimation, and melt/dry state estimation, respectively, significantly outperforming other classification models in each of the same categories. Automatic multiview machine learning–based winter hydrometeor classification enhances understanding, characterization, and quantification of geometrical, microphysical, and scattering properties of ice and snow hydrometeors. These improvements are essential for quantitative precipitation estimation algorithms and for microphysical parameterizations employed in numerical winter-weather forecast models and regional climate projections, with impacts on economy, safety, and everyday life.
more »
« less
Advanced Deep Learning-Based Supervised Classification of Multi-Angle Snowflake Camera Images
Abstract We present improvements over our previous approach to automatic winter hydrometeor classification by means of convolutional neural networks (CNNs), using more data and improved training techniques to achieve higher accuracy on a more complicated dataset than we had previously demonstrated. As an advancement of our previous proof-of-concept study, this work demonstrates broader usefulness of deep CNNs by using a substantially larger and more diverse dataset, which we make publicly available, from many more snow events. We describe the collection, processing, and sorting of this dataset of over 25,000 high-quality multiple-angle snowflake camera (MASC) image chips split nearly evenly between five geometric classes: aggregate, columnar crystal, planar crystal, graupel, and small particle. Raw images were collected over 32 snowfall events between November 2014 and May 2016 near Greeley, Colorado and were processed with an automated cropping and normalization algorithm to yield 224x224 pixel images containing possible hydrometeors. From the bulk set of over 8,400,000 extracted images, a smaller dataset of 14,793 images was sorted by image quality and recognizability (Q&R) using manual inspection. A presorting network trained on the Q&R dataset was applied to all 8,400,000+ images to automatically collect a subset of 283,351 good snowflake images. Roughly 5,000 representative examples were then collected from this subset manually for each of the five geometric classes. With a higher emphasis on in-class variety than our previous work, the final dataset yields trained networks that better capture the imperfect cases and diverse forms that occur within the broad categories studied to achieve an accuracy of 96.2% on a vastly more challenging dataset.
more »
« less
- Award ID(s):
- 2029806
- PAR ID:
- 10298847
- Date Published:
- Journal Name:
- Journal of Atmospheric and Oceanic Technology
- ISSN:
- 0739-0572
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The proliferation of Artificial Intelligence (AI) models such as Generative Adversarial Networks (GANs) has shown impressive success in image synthesis. Artificial GAN-based synthesized images have been widely spread over the Internet with the advancement in generating naturalistic and photo-realistic images. This might have the ability to improve content and media; however, it also constitutes a threat with regard to legitimacy, authenticity, and security. Moreover, implementing an automated system that is able to detect and recognize GAN-generated images is significant for image synthesis models as an evaluation tool, regardless of the input modality. To this end, we propose a framework for reliably detecting AI-generated images from real ones through Convolutional Neural Networks (CNNs). First, GAN-generated images were collected based on different tasks and different architectures to help with the generalization. Then, transfer learning was applied. Finally, several Class Activation Maps (CAM) were integrated to determine the discriminative regions that guided the classification model in its decision. Our approach achieved 100% on our dataset, i.e., Real or Synthetic Images (RSI), and a superior performance on other datasets and configurations in terms of its accuracy. Hence, it can be used as an evaluation tool in image generation. Our best detector was a pre-trained EfficientNetB4 fine-tuned on our dataset with a batch size of 64 and an initial learning rate of 0.001 for 20 epochs. Adam was used as an optimizer, and learning rate reduction along with data augmentation were incorporated.more » « less
-
We present a new annotated microscopic cellular image dataset to improve the effectiveness of machine learning methods for cellular image analysis. Cell counting is an important step in cell analysis. Typically, domain experts manually count cells in a microscopic image. Automated cell counting can potentially eliminate this tedious, time-consuming process. However, a good, labeled dataset is required for training an accurate machine learning model. Our dataset includes microscopic images of cells, and for each image, the cell count and the location of individual cells. The data were collected as part of an ongoing study investigating the potential of electrical stimulation to modulate stem cell differentiation and possible applications for neural repair. Compared to existing publicly available datasets, our dataset has more images of cells stained with more variety of antibodies (protein components of immune responses against invaders) typically used for cell analysis. The experimental results on this dataset indicate that none of the five existing models under this study are able to achieve sufficiently accurate count to replace the manual methods. The dataset is available at https://figshare.com/articles/dataset/Dataset/21970604.more » « less
-
Robust Mask R-CNN (Mask Regional Convolutional Neural Network) methods are proposed and tested for automatic detection of cracks on structures or their components that may be damaged during extreme events, such as earthquakes. We curated a new dataset with 2,021 labeled images for training and validation and aimed to find end-to-end deep neural networks for crack detection in the field. With data augmentation and parameters fine-tuning, Path Aggregation Network (PANet) with spatial attention mechanisms and High- resolution Network (HRNet) are introduced into Mask R-CNNs. The tests on three public datasets with low- or high-resolution images demonstrate that the proposed methods can achieve a big improvement over alternative networks, so the proposed method may be sufficient for crack detection for a variety of scales in real applications.more » « less
-
SE3Lig: SE(3)-equivariant CNNs for the reconstruction of cofactors and ligands in protein structuresProtein structure prediction algorithms such as AlphaFold2 and ESMFold have dramatically increased the availability of high-quality models of protein structures. Because these algorithms predict only the structure of the protein itself, there is a growing need for methods that can rapidly screen protein structures for ligands. Previous work on similar tasks has shown promise but is lacking scope in the classes of atoms predicted and can benefit from the recent architectural developments in convolutional neural networks (CNNs). In this work, we introduce SE3Lig, a model for semantic in-painting of small molecules in protein structures. Specifically, we report SE(3)-equivariant CNNs trained to predict the atomic densities of common classes of cofactors (hemes, flavins, etc.) and the water molecules and inorganic ions in their vicinity. While the models are trained on high-resolution crystal structures of enzymes, they perform well on structures predicted by AlphaFold2, which suggests that the algorithm correctly represents cofactor-binding cavities.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1175/JTECH-D-20-0189.1
