skip to main content

Title: Deep learning extended depth-of-field microscope for fast and slide-free histology
Microscopic evaluation of resected tissue plays a central role in the surgical management of cancer. Because optical microscopes have a limited depth-of-field (DOF), resected tissue is either frozen or preserved with chemical fixatives, sliced into thin sections placed on microscope slides, stained, and imaged to determine whether surgical margins are free of tumor cells—a costly and time- and labor-intensive procedure. Here, we introduce a deep-learning extended DOF (DeepDOF) microscope to quickly image large areas of freshly resected tissue to provide histologic-quality images of surgical margins without physical sectioning. The DeepDOF microscope consists of a conventional fluorescence microscope with the simple addition of an inexpensive (less than $10) phase mask inserted in the pupil plane to encode the light field and enhance the depth-invariance of the point-spread function. When used with a jointly optimized image-reconstruction algorithm, diffraction-limited optical performance to resolve subcellular features can be maintained while significantly extending the DOF (200 µm). Data from resected oral surgical specimens show that the DeepDOF microscope can consistently visualize nuclear morphology and other important diagnostic features across highly irregular resected tissue surfaces without serial refocusing. With the capability to quickly scan intact samples with subcellular detail, the DeepDOF microscope can improve tissue sampling during intraoperative tumor-margin assessment, while offering an affordable tool to provide histological information from resected tissue specimens in resource-limited settings.  more » « less
Award ID(s):
1730574 1652633
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the National Academy of Sciences
Page Range / eLocation ID:
33051 to 33060
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Histopathology plays a critical role in the diagnosis and surgical management of cancer. However, access to histopathology services, especially frozen section pathology during surgery, is limited in resource-constrained settings because preparing slides from resected tissue is time-consuming, labor-intensive, and requires expensive infrastructure. Here, we report a deep-learning-enabled microscope, named DeepDOF-SE, to rapidly scan intact tissue at cellular resolution without the need for physical sectioning. Three key features jointly make DeepDOF-SE practical. First, tissue specimens are stained directly with inexpensive vital fluorescent dyes and optically sectioned with ultra-violet excitation that localizes fluorescent emission to a thin surface layer. Second, a deep-learning algorithm extends the depth-of-field, allowing rapid acquisition of in-focus images from large areas of tissue even when the tissue surface is highly irregular. Finally, a semi-supervised generative adversarial network virtually stains DeepDOF-SE fluorescence images with hematoxylin-and-eosin appearance, facilitating image interpretation by pathologists without significant additional training. We developed the DeepDOF-SE platform using a data-driven approach and validated its performance by imaging surgical resections of suspected oral tumors. Our results show that DeepDOF-SE provides histological information of diagnostic importance, offering a rapid and affordable slide-free histology platform for intraoperative tumor margin assessment and in low-resource settings.

    more » « less
  2. Optical coherence tomography (OCT) is a rapid non‐invasive imaging technique that has shown high sensitivity for intra‐operative surgical margin assessment in human breast cancer clinical trials. This promising technology has not been evaluated in veterinary medicine. The objective of this study was to correlate normal and abnormal histological features with OCT images for surgical margins from excised canine soft tissue sarcoma (STS) and to establish image evaluation criteria for identifying positive surgical margins. Fourteen client‐owned dogs underwent surgical resection of a STS and OCT imaging of 2 to 4 areas of interest on the resected specimen were performed. Following imaging these areas were marked with surgical ink and trimmed for histopathology evaluation. Results showed that different tissue types had distinct characteristic appearances on OCT imaging. Adipose tissue exhibited a relatively low scattering and a honey‐comb texture pattern. Skeletal muscle and sarcoma tissue were both dense and highly scattering. While sarcoma tissue was highly scattering, it did not have organized recognizable structure in contrast to muscle which showed clear fibre alignment patterns. In this investigation, we showed different tissue types had different and characteristic scattering and image texture appearances on OCT, which closely correlate with low‐power histology images. Given the differentiation between tissue types the results support that OCT could be used to identify positive surgical margins immediately following resection of STS. Further research is needed to assess the diagnostic accuracy of this method for surgical margin assessment.

    more » « less
  3. Abstract The first line of treatment for most solid tumors is surgical resection of the primary tumor with adequate negative margins. Incomplete tumor resections with positive margins account for over 75% of local recurrences and the development of distant metastases. In cases of oral cavity squamous cell carcinoma (OSCC), the rate of successful tumor removal with adequate margins is just 50–75%. Advanced real‐time imaging methods that improve the detection of tumor margins can help improve success rates,overall safety, and reduce the cost. Fluorescence imaging in the second near‐infrared (NIR‐II) window has the potential to revolutionize the field due to its high spatial resolution, low background signal, and deep tissue penetration properties, but NIR‐II dyes with adequate in vivo performance and safety profiles are scarce. A novel NIR‐II fluorophore, XW‐03‐66, with a fluorescence quantum yield (QY) of 6.0% in aqueous media is reported. XW‐03‐66 self‐assembles into nanoparticles (≈80 nm) and has a systemic circulation half‐life ( t 1/2 ) of 11.3 h. In mouse models of human papillomavirus (HPV)+ and HPV‐ OSCC, XW‐03‐66 outperformed indocyanine green (ICG), a clinically available NIR dye, and enabled intraoperative NIR‐II image‐guided resection of the tumor and adjacent draining lymph node with negative margins. In vitro and in vivo toxicity assessments revealed minimal safety concerns for in vivo applications. 
    more » « less
  4. null (Ed.)
    The depth of field (DoF) was extended 2.8-fold to achieve rapid crystal screening by retrofitting a custom-designed micro-retarder array (µRA) in the optical beam path of a nonlinear optical microscope. The merits of the proposed strategy for DoF enhancement were assessed in applications of second-harmonic generation imaging of protein crystals. It was found that DoF extension increased the number of crystals detected while simultaneously reducing the number of ` z -slices' required for screening. Experimental measurements of the wavelength-dependence of the extended DoF were in excellent agreement with theoretical predictions. These results provide a simple and broadly applicable approach to increase the throughput of existing nonlinear optical imaging methods for protein crystal screening. 
    more » « less
  5. A miniature optical-sectioning fluorescence microscope with high sensitivity and resolution would enable non-invasive and real-time tissue inspection, with potential use cases including early disease detection and intraoperative guidance. Previously, we developed a miniature MEMS-based dual-axis confocal (DAC) microscope that enabled video-rate optically sectionedin vivomicroscopy of human tissues. However, the device’s clinical utility was limited due to a small field of view, a non-adjustable working distance, and a lack of a sterilization strategy. In our latest design, we have made improvements to achieve a 2x increase in the field of view (600 × 300 µm) and an adjustable working distance range of 150 µm over a wide range of excitation/emission wavelengths (488–750 nm), all while maintaining a high frame rate of 15 frames per second (fps). Furthermore, the device is designed to image through a disposable sterile plastic drape for convenient clinical use. We rigorously characterize the performance of the device and show example images ofex vivotissues to demonstrate the optical performance of our new design, including fixed mouse skin and human prostate, as well as fresh mouse kidney, mouse intestine, and human head and neck surgical specimens with corresponding H&E histology. These improvements will facilitate clinical testing and translation.

    more » « less