An official website of the United States government
Histopathology plays a critical role in the diagnosis and surgical management of cancer. However, access to histopathology services, especially frozen section pathology during surgery, is limited in resource-constrained settings because preparing slides from resected tissue is time-consuming, labor-intensive, and requires expensive infrastructure. Here, we report a deep-learning-enabled microscope, named DeepDOF-SE, to rapidly scan intact tissue at cellular resolution without the need for physical sectioning. Three key features jointly make DeepDOF-SE practical. First, tissue specimens are stained directly with inexpensive vital fluorescent dyes and optically sectioned with ultra-violet excitation that localizes fluorescent emission to a thin surface layer. Second, a deep-learning algorithm extends the depth-of-field, allowing rapid acquisition of in-focus images from large areas of tissue even when the tissue surface is highly irregular. Finally, a semi-supervised generative adversarial network virtually stains DeepDOF-SE fluorescence images with hematoxylin-and-eosin appearance, facilitating image interpretation by pathologists without significant additional training. We developed the DeepDOF-SE platform using a data-driven approach and validated its performance by imaging surgical resections of suspected oral tumors. Our results show that DeepDOF-SE provides histological information of diagnostic importance, offering a rapid and affordable slide-free histology platform for intraoperative tumor margin assessment and in low-resource settings.
more »
« less
Computational pathology for musculoskeletal conditions using machine learning: advances, trends, and challenges
Abstract Histopathology is widely used to analyze clinical biopsy specimens and tissues from pre-clinical models of a variety of musculoskeletal conditions. Histological assessment relies on scoring systems that require expertise, time, and resources, which can lead to an analysis bottleneck. Recent advancements in digital imaging and image processing provide an opportunity to automate histological analyses by implementing advanced statistical models such as machine learning and deep learning, which would greatly benefit the musculoskeletal field. This review provides a high-level overview of machine learning applications, a general pipeline of tissue collection to model selection, and highlights the development of image analysis methods, including some machine learning applications, to solve musculoskeletal problems. We discuss the optimization steps for tissue processing, sectioning, staining, and imaging that are critical for the successful generalizability of an automated image analysis model. We also commenting on the considerations that should be taken into account during model selection and the considerable advances in the field of computer vision outside of histopathology, which can be leveraged for image analysis. Finally, we provide a historic perspective of the previously used histopathological image analysis applications for musculoskeletal diseases, and we contrast it with the advantages of implementing state-of-the-art computational pathology approaches. While some deep learning approaches have been used, there is a significant opportunity to expand the use of such approaches to solve musculoskeletal problems.
more »
« less
- Award ID(s):
- 1750326
- PAR ID:
- 10325548
- Date Published:
- Journal Name:
- Arthritis Research & Therapy
- Volume:
- 24
- Issue:
- 1
- ISSN:
- 1478-6362
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to asvirtual staining, were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.more » « less
-
Reflectance confocal microscopy (RCM) is a noninvasive optical imaging technique that uses a laser to capture cellular-level resolution images based on differing refractive indices of tissue elements. RCM image interpretation is challenging and requires training to interpret and correlate the grayscale output images that lack nuclear features with tissue pathology. Here, we utilize a deep learning-based framework that uses a convolutional neural network to transform grayscale images into virtually-stained hematoxylin and eosin (H&E)-like images enabling the visualization of various skin layers. To train the deep-learning framework, a series of a minimum of 7 time-lapsed, successive “stacks” of RCM images of excised tissue, spaced 1.52μm apart to a depth of 60.96μm were obtained using the Vivascope 1500. The tissue samples were stained with a 50% acetic acid solution to enhance cell nuclei. These images served as the “ground truth” to train a deep convolutional neural network with a conditional generative adversarial network (GAN)-based machine learning algorithm to digitally convert the images into GAN-based H&E-stained digital images. The machine learning algorithm was initially trained and subsequently retrained with new samples, specifically focusing on squamous neoplasms. The trained algorithm was applied to skin lesions that had a clinical differential diagnosis of squamous neoplasms including squamous cell carcinoma, actinic keratosis, seborrheic keratosis, and basal cell carcinoma. Through continuous training and refinement, the algorithm was able to produce high-resolution, histological quality images of different squamous neoplasms. This algorithm may be used in the future to facilitate earlier diagnosis of cutaneous neoplasms and enable greater uptake of noninvasive imaging technology within the medical community.more » « less
-
Abstract Chemical imaging, especially mid-infrared spectroscopic microscopy, enables label-free biomedical analyses while achieving expansive molecular sensitivity. However, its slow speed and poor image quality impede widespread adoption. We present a microscope that provides high-throughput recording, low noise, and high spatial resolution where the bottom-up design of its optical train facilitates dual-axis galvo laser scanning of a diffraction-limited focal point over large areas using custom, compound, infinity-corrected refractive objectives. We demonstrate whole-slide, speckle-free imaging in ~3 min per discrete wavelength at 10× magnification (2 μm/pixel) and high-resolution capability with its 20× counterpart (1 μm/pixel), both offering spatial quality at theoretical limits while maintaining high signal-to-noise ratios (>100:1). The data quality enables applications of modern machine learning and capabilities not previously feasible – 3D reconstructions using serial sections, comprehensive assessments of whole model organisms, and histological assessments of disease in time comparable to clinical workflows. Distinct from conventional approaches that focus on morphological investigations or immunostaining techniques, this development makes label-free imaging of minimally processed tissue practical.more » « less
-
Ferraro, Pietro; Grilli, Simonetta; Psaltis, Demetri (Ed.)Deep learning techniques create new opportunities to revolutionize tissue staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, accurate and environmentally friendly alternatives to standard chemical staining methods. These deep learning-based virtual staining techniques can successfully generate different types of histological stains, including immunohistochemical stains, from label-free microscopic images of unstained samples by using, e.g., autofluorescence microscopy, quantitative phase imaging (QPI) and reflectance confocal microscopy. Similar approaches were also demonstrated for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this presentation, I will provide an overview of our recent work on the use of deep neural networks for label-free tissue staining, also covering their biomedical applications.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1186/s13075-021-02716-3
