An official website of the United States government
In the field of materials science, microscopy is the first and often only accessible method for structural characterization. There is a growing interest in the development of machine learning methods that can automate the analysis and interpretation of microscopy images. Typically training of machine learning models requires large numbers of images with associated structural labels, however, manual labeling of images requires domain knowledge and is prone to human error and subjectivity. To overcome these limitations, we present a semi-supervised transfer learning approach that uses a small number of labeled microscopy images for training and performs as effectively as methods trained on significantly larger image datasets. Specifically, we train an image encoder with unlabeled images using self-supervised learning methods and use that encoder for transfer learning of different downstream image tasks (classification and segmentation) with a minimal number of labeled images for training. We test the transfer learning ability of two self-supervised learning methods: SimCLR and Barlow-Twins on transmission electron microscopy (TEM) images. We demonstrate in detail how this machine learning workflow applied to TEM images of protein nanowires enables automated classification of nanowire morphologies ( e.g. , single nanowires, nanowire bundles, phase separated) as well as segmentation tasks that can serve as groundwork for quantification of nanowire domain sizes and shape analysis. We also extend the application of the machine learning workflow to classification of nanoparticle morphologies and identification of different type of viruses from TEM images.
more »
« less
Accurate automated segmentation of autophagic bodies in yeast vacuoles using cellpose 2.0
Segmenting autophagic bodies in yeast TEM images is a key technique for measuring changes in autophagosome size and number in order to better understand macroautophagy/autophagy. Manual segmentation of these images can be very time consuming, particularly because hundreds of images are needed for accurate measurements. Here we describe a validated Cellpose 2.0 model that can segment these images with accuracy comparable to that of human experts. This model can be used for fully automated segmentation, eliminating the need for manual body outlining, or for model-assisted segmentation, which allows human oversight but is still five times as fast as the current manual method. The model is specific to segmentation of autophagic bodies in yeast TEM images, but researchers working in other systems can use a similar process to generate their own Cellpose 2.0 models to attempt automated segmentations. Our model and instructions for its use are presented here for the autophagy community.
more »
« less
- Award ID(s):
- 2243163
- PAR ID:
- 10561007
- Publisher / Repository:
- Taylor & Francis
- Date Published:
- Journal Name:
- Autophagy
- Volume:
- 20
- Issue:
- 9
- ISSN:
- 1554-8627
- Page Range / eLocation ID:
- 2092 to 2099
- Subject(s) / Keyword(s):
- automated labeling autophagy computer vision electron microscopy image analysis machine learning
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Pack-ice seals are key indicator species in the Southern Ocean. Their large size (2–4 m) and continent-wide distribution make them ideal candidates for monitoring programs via very-high-resolution satellite imagery. The sheer volume of imagery required, however, hampers our ability to rely on manual annotation alone. Here, we present SealNet 2.0, a fully automated approach to seal detection that couples a sea ice segmentation model to find potential seal habitats with an ensemble of semantic segmentation convolutional neural network models for seal detection. Our best ensemble attains 0.806 precision and 0.640 recall on an out-of-sample test dataset, surpassing two trained human observers. Built upon the original SealNet, it outperforms its predecessor by using annotation datasets focused on sea ice only, a comprehensive hyperparameter study leveraging substantial high-performance computing resources, and post-processing through regression head outputs and segmentation head logits at predicted seal locations. Even with a simplified version of our ensemble model, using AI predictions as a guide dramatically boosted the precision and recall of two human experts, showing potential as a training device for novice seal annotators. Like human observers, the performance of our automated approach deteriorates with terrain ruggedness, highlighting the need for statistical treatment to draw global population estimates from AI output.more » « less
-
Autophagosomes and lysosomes work in tandem to conduct autophagy, an intracellular degradation system which is crucial for cellular homeostasis. Altered autophagy contributes to the pathophysiology of various diseases, including cancers and metabolic diseases. Although many studies have investigated autophagy to elucidate disease pathogenesis, specific identification of the various components of the cellular degradation machinery remains difficult. The goal of this paper is to describe an approach to reproducibly identify and distinguish subcellular structures involved in autophagy. We provide methods that avoid common pitfalls, including a detailed explanation for how to distinguish lysosomes and lipid droplets and discuss the differences between autophagosomes and inclusion bodies. These methods are based on using transmission electron microscopy (TEM), capable of generating nanometer-scale micrographs of cellular degradation components in a fixed sample. In addition to TEM, we discuss other imaging techniques, such as immunofluorescence and immunogold labeling, which can be utilized for the reliable and accurate classification of cellular organelles. Our results show how these methods may be employed to accurately quantify the cellular degradation machinery under various conditions, such as treatment with the endoplasmic reticulum stressor thapsigargin or the ablation of dynamin-related protein 1.more » « less
-
Abstract Eukaryotic cells have evolved membrane-bound organelles, including the endoplasmic reticulum (ER), Golgi, mitochondria, peroxisomes, chloroplasts (in plants and green algae) and lysosomes/vacuoles, for specialized functions. Organelle quality control and their proper interactions are crucial both for normal cell homeostasis and function and for environmental adaption. Dynamic turnover of organelles is tightly controlled, with autophagy playing an essential role. Autophagy is a programmed process for efficient clearing of unwanted or damaged macromolecules or organelles, transporting them to vacuoles for degradation and recycling and thereby enhancing plant environmental plasticity. The specific autophagic engulfment of organelles requires activation of a selective autophagy pathway, recognition of the organelle by a receptor, and selective incorporation of the organelle into autophagosomes. While some of the autophagy machinery and mechanisms for autophagic removal of organelles is conserved across eukaryotes, plants have also developed unique mechanisms and machinery for these pathways. In this review, we discuss recent progress in understanding autophagy regulation in plants, with a focus on autophagic degradation of membrane-bound organelles. We also raise some important outstanding questions to be addressed in the future.more » « less
-
The rapidly growing use of imaging infrastructure in the energy materials domain drives significant data accumulation in terms of their amount and complexity. The applications of routine techniques for image processing in materials research are often ad hoc , indiscriminate, and empirical, which renders the crucial task of obtaining reliable metrics for quantifications obscure. Moreover, these techniques are expensive, slow, and often involve several preprocessing steps. This paper presents a novel deep learning-based approach for the high-throughput analysis of the particle size distributions from transmission electron microscopy (TEM) images of carbon-supported catalysts for polymer electrolyte fuel cells. A dataset of 40 high-resolution TEM images at different magnification levels, from 10 to 100 nm scales, was annotated manually. This dataset was used to train the U-Net model, with the StarDist formulation for the loss function, for the nanoparticle segmentation task. StarDist reached a precision of 86%, recall of 85%, and an F1-score of 85% by training on datasets as small as thirty images. The segmentation maps outperform models reported in the literature for a similar problem, and the results on particle size analyses agree well with manual particle size measurements, albeit at a significantly lower cost.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1080/15548627.2024.2353458
