An invasive biopsy followed by histological staining is the benchmark for pathological diagnosis of skin tumors. The process is cumbersome and time-consuming, often leading to unnecessary biopsies and scars. Emerging noninvasive optical technologies such as reflectance confocal microscopy (RCM) can provide label-free, cellular-level resolution, in vivo images of skin without performing a biopsy. Although RCM is a useful diagnostic tool, it requires specialized training because the acquired images are grayscale, lack nuclear features, and are difficult to correlate with tissue pathology. Here, we present a deep learning-based framework that uses a convolutional neural network to rapidly transform in vivo RCM images of unstained skin into virtually-stained hematoxylin and eosin-like images with microscopic resolution, enabling visualization of the epidermis, dermal-epidermal junction, and superficial dermis layers. The network was trained under an adversarial learning scheme, which takes ex vivo RCM images of excised unstained/label-free tissue as inputs and uses the microscopic images of the same tissue labeled with acetic acid nuclear contrast staining as the ground truth. We show that this trained neural network can be used to rapidly perform virtual histology of in vivo, label-free RCM images of normal skin structure, basal cell carcinoma, and melanocytic nevi with pigmented melanocytes, demonstrating similar histological features to traditional histology from the same excised tissue. This application of deep learning-based virtual staining to noninvasive imaging technologies may permit more rapid diagnoses of malignant skin neoplasms and reduce invasive skin biopsies.
- Award ID(s):
- 2141157
- NSF-PAR ID:
- 10403115
- Editor(s):
- Volpe, Giovanni; Pereira, Joana B.; Brunner, Daniel; Ozcan, Aydogan
- Date Published:
- Journal Name:
- SPIE Optics and Photonics Conference
- Page Range / eLocation ID:
- 35
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
An invasive biopsy followed by histological staining is the gold standard for traditional pathological identification of skin cancers, which requires a long processing time and often results in undesired biopsies and scarring. Reflectance confocal microscopy (RCM) can provide biopsy-free in vivo images of skin structure with a cellular-level resolution for skin disease evaluation; however, the RCM images are grayscale, miss nuclear features and have a low correlation with pathology, which requires specialized training for interpretation. Here, we report a deep learning-based method for performing non-invasive virtual skin histology using in vivo, label-free RCM images.more » « less
-
Shaked, Natan T. ; Hayden, Oliver (Ed.)We report label-free, in vivo virtual histology of skin using reflectance confocal microscopy (RCM). We trained a deep neural network to transform in vivo RCM images of unstained skin into virtually stained H&E-like microscopic images with nuclear contrast. This framework successfully generalized to diverse skin conditions, e.g., normal skin, basal cell carcinoma, and melanocytic nevi, as well as distinct skin layers, including the epidermis, dermal-epidermal junction, and superficial dermis layers. This label-free in vivo skin virtual histology framework can be transformative for faster and more accurate diagnosis of malignant skin neoplasms, with the potential to significantly reduce unnecessary skin biopsies.more » « less
-
more » « less
A miniature optical-sectioning fluorescence microscope with high sensitivity and resolution would enable non-invasive and real-time tissue inspection, with potential use cases including early disease detection and intraoperative guidance. Previously, we developed a miniature MEMS-based dual-axis confocal (DAC) microscope that enabled video-rate optically sectioned
in vivo microscopy of human tissues. However, the device’s clinical utility was limited due to a small field of view, a non-adjustable working distance, and a lack of a sterilization strategy. In our latest design, we have made improvements to achieve a 2x increase in the field of view (600 × 300 µm) and an adjustable working distance range of 150 µm over a wide range of excitation/emission wavelengths (488–750 nm), all while maintaining a high frame rate of 15 frames per second (fps). Furthermore, the device is designed to image through a disposable sterile plastic drape for convenient clinical use. We rigorously characterize the performance of the device and show example images ofex vivo tissues to demonstrate the optical performance of our new design, including fixed mouse skin and human prostate, as well as fresh mouse kidney, mouse intestine, and human head and neck surgical specimens with corresponding H&E histology. These improvements will facilitate clinical testing and translation.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1117/12.2632602
