Abstract Underwater imaging enables nondestructive plankton sampling at frequencies, durations, and resolutions unattainable by traditional methods. These systems necessitate automated processes to identify organisms efficiently. Early underwater image processing used a standard approach: binarizing images to segment targets, then integrating deep learning models for classification. While intuitive, this infrastructure has limitations in handling high concentrations of biotic and abiotic particles, rapid changes in dominant taxa, and highly variable target sizes. To address these challenges, we introduce a new framework that starts with a scene classifier to capture large within‐image variation, such as disparities in the layout of particles and dominant taxa. After scene classification, scene‐specific Mask regional convolutional neural network (Mask R‐CNN) models are trained to separate target objects into different groups. The procedure allows information to be extracted from different image types, while minimizing potential bias for commonly occurring features. Using in situ coastal plankton images, we compared the scene‐specific models to the Mask R‐CNN model encompassing all scene categories as a single full model. Results showed that the scene‐specific approach outperformed the full model by achieving a 20% accuracy improvement in complex noisy images. The full model yielded counts that were up to 78% lower than those enumerated by the scene‐specific model for some small‐sized plankton groups. We further tested the framework on images from a benthic video camera and an imaging sonar system with good results. The integration of scene classification, which groups similar images together, can improve the accuracy of detection and classification for complex marine biological images.
more »
« less
A microfluidic approach for label-free identification of small-sized microplastics in seawater
Abstract Marine microplastics are emerging as a growing environmental concern due to their potential harm to marine biota. The substantial variations in their physical and chemical properties pose a significant challenge when it comes to sampling and characterizing small-sized microplastics. In this study, we introduce a novel microfluidic approach that simplifies the trapping and identification process of microplastics in surface seawater, eliminating the need for labeling. We examine various models, including support vector machine, random forest, convolutional neural network (CNN), and residual neural network (ResNet34), to assess their performance in identifying 11 common plastics. Our findings reveal that the CNN method outperforms the other models, achieving an impressive accuracy of 93% and a mean area under the curve of 98 ± 0.02%. Furthermore, we demonstrate that miniaturized devices can effectively trap and identify microplastics smaller than 50 µm. Overall, this proposed approach facilitates efficient sampling and identification of small-sized microplastics, potentially contributing to crucial long-term monitoring and treatment efforts.
more »
« less
- Award ID(s):
- 1655221
- PAR ID:
- 10437622
- Date Published:
- Journal Name:
- Scientific Reports
- Volume:
- 13
- Issue:
- 1
- ISSN:
- 2045-2322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Pollen identification is necessary for several subfields of geology, ecology, and evolutionary biology. However, the existing methods for pollen identification are laborious, time-consuming, and require highly skilled scientists. Therefore, there is a pressing need for an automated and accurate system for pollen identification, which can be beneficial for both basic research and applied issues such as identifying airborne allergens. In this study, we propose a deep learning (DL) approach to classify pollen grains in the Great Basin Desert, Nevada, USA. Our dataset consisted of 10,000 images of 40 pollen species. To mitigate the limitations imposed by the small volume of our training dataset, we conducted an in-depth comparative analysis of numerous pre-trained Convolutional Neural Network (CNN) architectures utilizing transfer learning methodologies. Simultaneously, we developed and incorporated an innovative CNN model, serving to augment our exploration and optimization of data modeling strategies. We applied different architectures of well-known pre-trained deep CNN models, including AlexNet, VGG-16, MobileNet-V2, ResNet (18, 34, and 50, 101), ResNeSt (50, 101), SE-ResNeXt, and Vision Transformer (ViT), to uncover the most promising modeling approach for the classification of pollen grains in the Great Basin. To evaluate the performance of the pre-trained deep CNN models, we measured accuracy, precision, F1-Score, and recall. Our results showed that the ResNeSt-110 model achieved the best performance, with an accuracy of 97.24%, precision of 97.89%, F1-Score of 96.86%, and recall of 97.13%. Our results also revealed that transfer learning models can deliver better and faster image classification results compared to traditional CNN models built from scratch. The proposed method can potentially benefit various fields that rely on efficient pollen identification. This study demonstrates that DL approaches can improve the accuracy and efficiency of pollen identification, and it provides a foundation for further research in the field.more » « less
-
Mass spectrometry imaging (MSI) is widely used for the label-free molecular mapping of biological samples. The identification of co-localized molecules in MSI data is crucial to the understanding of biochemical pathways. One of key challenges in molecular colocalization is that complex MSI data are too large for manual annotation but too small for training deep neural networks. Herein, we introduce a self-supervised clustering approach based on contrastive learning, which shows an excellent performance in clustering of MSI data. We train a deep convolutional neural network (CNN) using MSI data from a single experiment without manual annotations to effectively learn high-level spatial features from ion images and classify them based on molecular colocalizations. We demonstrate that contrastive learning generates ion image representations that form well-resolved clusters. Subsequent self-labeling is used to fine-tune both the CNN encoder and linear classifier based on confidently classified ion images. This new approach enables autonomous and high-throughput identification of co-localized species in MSI data, which will dramatically expand the application of spatial lipidomics, metabolomics, and proteomics in biological research.more » « less
-
Golpira, Hemin (Ed.)The paper proposes an approach for fast small signal stability assessment on a short data window using deep learning algorithms. This paper shows that the proposed deep convolutional neural networks (CNNs)-based assessment approach is faster than traditional methods (i.e. Prony’s method). The evaluated CNNs are fully convolutional network (FCN), CNN with sub-sampling steps performed through max pooling (Time LeNet), time CNN, fully convolutional network with attention mechanism (Encoder), and CNN with a shortcut residual connection (ResNet). The proposed approach is validated on different synthetic measurement data sets generated from the IEEE 9-bus system that is used as a reference, and further applied to a 769-bus system representing a region in the U. S. Eastern Interconnection. We show that precision and recall are more informative metrics than accuracy for the reliability of the stability assessment process using the proposed methodology. In addition, the method’s efficiency is compared to classical Prony method.more » « less
-
An accurate molecular identification of plastic waste is important in increasing the efficacy of automatic plastic sorting in recycling. However, identification of real-world plastic waste, according to their resin identification code, remains challenging due to the lack of techniques that can provide high molecular selectivity. In this study, a standoff photothermal spectroscopy technique, utilizing a microcantilever, was used for acquiring mid-infrared spectra of real-world plastic waste, including those with additives, surface contaminants, and mixed plastics. Analysis of the standoff spectral data, using Convolutional Neural Network (CNN), showed 100% accuracy in selectively identifying real-world plastic waste according to their respective resin identification codes. Standoff photothermal spectroscopy, together with CNN analysis, offers a promising approach for the selective characterization of waste plastics in Material Recovery Facilities (MRFs).more » « less
An official website of the United States government

