An official website of the United States government
Abstract Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to asvirtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.
more »
« less
Virtual staining of label-free tissue using deep learning
Deep learning techniques create new opportunities to revolutionize tissue staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, accurate and environmentally friendly alternatives to standard chemical staining methods. These deep learning-based virtual staining techniques can successfully generate different types of histological stains, including immunohistochemical stains, from label-free microscopic images of unstained samples by using, e.g., autofluorescence microscopy, quantitative phase imaging (QPI) and reflectance confocal microscopy. Similar approaches were also demonstrated for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this presentation, I will provide an overview of our recent work on the use of deep neural networks for label-free tissue staining, also covering their biomedical applications.
more »
« less
- Award ID(s):
- 2141157
- PAR ID:
- 10494698
- Editor(s):
- Ferraro, Pietro; Grilli, Simonetta; Psaltis, Demetri
- Publisher / Repository:
- SPIE
- Date Published:
- Journal Name:
- SPIE Optical Metrology
- ISBN:
- 9781510668263
- Page Range / eLocation ID:
- 11
- Format(s):
- Medium: X
- Location:
- Munich, Germany
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Histological staining is a vital step in diagnosing various diseases and has been used for more than a century to provide contrast in tissue sections, rendering the tissue constituents visible for microscopic analysis by medical experts. However, this process is time consuming, labour intensive, expensive and destructive to the specimen. Recently, the ability to virtually stain unlabelled tissue sections, entirely avoiding the histochemical staining step, has been demonstrated using tissue-stain-specific deep neural networks. Here, we present a new deep-learning-based framework that generates virtually stained images using label-free tissue images, in which different stains are merged following a micro-structure map defined by the user. This approach uses a single deep neural network that receives two different sources of information as its input: (1) autofluorescence images of the label-free tissue sample and (2) a “digital staining matrix”, which represents the desired microscopic map of the different stains to be virtually generated in the same tissue section. This digital staining matrix is also used to virtually blend existing stains, digitally synthesizing new histological stains. We trained and blindly tested this virtual-staining network using unlabelled kidney tissue sections to generate micro-structured combinations of haematoxylin and eosin (H&E), Jones’ silver stain, and Masson’s trichrome stain. Using a single network, this approach multiplexes the virtual staining of label-free tissue images with multiple types of stains and paves the way for synthesizing new digital histological stains that can be created in the same tissue cross section, which is currently not feasible with standard histochemical staining methods.more » « less
-
We present a method to generate multiple virtual stains on an image of label-free tissue using a single deep neural network, which is fed with the autofluorescence images of the unlabeled tissue alongside a user-defined digital-staining matrix. Users can indicate which stain to apply on each pixel by editing the digital-staining matrix and blend multiple virtual stains, creating entirely new stain combinations.more » « less
-
Virtual Staining of Defocused Autofluorescence Images of Unlabeled Tissue Using Deep Neural NetworksDeep learning-based virtual staining was developed to introduce image contrast to label-free tissue sections, digitally matching the histological staining, which is time-consuming, labor-intensive, and destructive to tissue. Standard virtual staining requires high autofocusing precision during the whole slide imaging of label-free tissue, which consumes a significant portion of the total imaging time and can lead to tissue photodamage. Here, we introduce a fast virtual staining framework that can stain defocused autofluorescence images of unlabeled tissue, achieving equivalent performance to virtual staining of in-focus label-free images, also saving significant imaging time by lowering the microscope’s autofocusing precision. This framework incorporates a virtual autofocusing neural network to digitally refocus the defocused images and then transforms the refocused images into virtually stained images using a successive network. These cascaded networks form a collaborative inference scheme: the virtual staining model regularizes the virtual autofocusing network through a style loss during the training. To demonstrate the efficacy of this framework, we trained and blindly tested these networks using human lung tissue. Using 4× fewer focus points with 2× lower focusing precision, we successfully transformed the coarsely-focused autofluorescence images into high-quality virtually stained H&E images, matching the standard virtual staining framework that used finely-focused autofluorescence input images. Without sacrificing the staining quality, this framework decreases the total image acquisition time needed for virtual staining of a label-free whole-slide image (WSI) by ~32%, together with a ~89% decrease in the autofocusing time, and has the potential to eliminate the laborious and costly histochemical staining process in pathology.more » « less
-
Objective and Impact Statement . We present a fully automated hematological analysis framework based on single-channel (single-wavelength), label-free deep-ultraviolet (UV) microscopy that serves as a fast, cost-effective alternative to conventional hematology analyzers. Introduction . Hematological analysis is essential for the diagnosis and monitoring of several diseases but requires complex systems operated by trained personnel, costly chemical reagents, and lengthy protocols. Label-free techniques eliminate the need for staining or additional preprocessing and can lead to faster analysis and a simpler workflow. In this work, we leverage the unique capabilities of deep-UV microscopy as a label-free, molecular imaging technique to develop a deep learning-based pipeline that enables virtual staining, segmentation, classification, and counting of white blood cells (WBCs) in single-channel images of peripheral blood smears. Methods . We train independent deep networks to virtually stain and segment grayscale images of smears. The segmented images are then used to train a classifier to yield a quantitative five-part WBC differential. Results. Our virtual staining scheme accurately recapitulates the appearance of cells under conventional Giemsa staining, the gold standard in hematology. The trained cellular and nuclear segmentation networks achieve high accuracy, and the classifier can achieve a quantitative five-part differential on unseen test data. Conclusion . This proposed automated hematology analysis framework could greatly simplify and improve current complete blood count and blood smear analysis and lead to the development of a simple, fast, and low-cost, point-of-care hematology analyzer.more » « less
