An official website of the United States government
Multiple-surface segmentation in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak image boundaries. Recently, many deep learning-based methods have been developed for this task and yield remarkable performance. Unfortunately, due to the scarcity of training data in medical imaging, it is challenging for deep learning networks to learn the global structure of the target surfaces, including surface smoothness. To bridge this gap, this study proposes to seamlessly unify a U-Net for feature learning with a constrained differentiable dynamic programming module to achieve end-to-end learning for retina OCT surface segmentation to explicitly enforce surface smoothness. It effectively utilizes the feedback from the downstream model optimization module to guide feature learning, yielding better enforcement of global structures of the target surfaces. Experiments on Duke AMD (age-related macular degeneration) and JHU MS (multiple sclerosis) OCT data sets for retinal layer segmentation demonstrated that the proposed method was able to achieve subvoxel accuracy on both datasets, with the mean absolute surface distance (MASD) errors of 1.88 ± 1.96μmand 2.75 ± 0.94μm, respectively, over all the segmented surfaces.
more »
« less
Globally optimal OCT surface segmentation using a constrained IPM optimization
Segmentation of multiple surfaces in optical coherence tomography (OCT) images is a challenging problem, further complicated by the frequent presence of weak boundaries, varying layer thicknesses, and mutual influence between adjacent surfaces. The traditional graph-based optimal surface segmentation method has proven its effectiveness with its ability to capture various surface priors in a uniform graph model. However, its efficacy heavily relies on handcrafted features that are used to define the surface cost for the “goodness” of a surface. Recently, deep learning (DL) is emerging as a powerful tool for medical image segmentation thanks to its superior feature learning capability. Unfortunately, due to the scarcity of training data in medical imaging, it is nontrivial for DL networks toimplicitlylearn the global structure of the target surfaces, including surface interactions. This study proposes to parameterize the surface cost functions in the graph model and leverage DL to learn those parameters. The multiple optimal surfaces are then simultaneously detected by minimizing the total surface cost whileexplicitlyenforcing the mutual surface interaction constraints. The optimization problem is solved by the primal-dual interior-point method (IPM), which can be implemented by a layer of neural networks, enabling efficient end-to-end training of the whole network. Experiments on spectral-domain optical coherence tomography (SD-OCT) retinal layer segmentation demonstrated promising segmentation results with sub-pixel accuracy.
more »
« less
- PAR ID:
- 10531231
- Publisher / Repository:
- Optical Society of America
- Date Published:
- Journal Name:
- Optics Express
- Volume:
- 30
- Issue:
- 2
- ISSN:
- 1094-4087; OPEXFF
- Format(s):
- Medium: X Size: Article No. 2453
- Size(s):
- Article No. 2453
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Binary semantic segmentation in computer vision is a fundamental problem. As a model-based segmentation method, the graph-cut approach was one of the most successful binary segmentation methods thanks to its global optimality guarantee of the solutions and its practical polynomial-time complexity. Recently, many deep learning (DL) based methods have been developed for this task and yielded remarkable performance, resulting in a paradigm shift in this field. To combine the strengths of both approaches, we propose in this study to integrate the graph-cut approach into a deep learning network for end-to-end learning. Unfortunately, backward propagation through the graph-cut module in the DL network is challenging due to the combinatorial nature of the graph-cut algorithm. To tackle this challenge, we propose a novel residual graph-cut loss and a quasi-residual connection, enabling the backward propagation of the gradients of the residual graph-cut loss for effective feature learning guided by the graph-cut segmentation model. In the inference phase, globally optimal segmentation is achieved with respect to the graph-cut energy defined on the optimized image features learned from DL networks. Experiments on the public AZH chronic wound data set and the pancreas cancer data set from the medical segmentation decathlon (MSD) demonstrated promising segmentation accuracy and improved robustness against adversarial attacks.more » « less
-
In optical coherence tomography (OCT), the axial resolution is often superior to the lateral resolution, which is sacrificed for long imaging depths. To address this anisotropy, we previously developed optical coherence refraction tomography (OCRT), which uses images from multiple angles to computationally reconstruct an image with isotropic resolution, given by the OCT axial resolution. On the other hand, spectroscopic OCT (SOCT), an extension of OCT, trades axial resolution for spectral resolution and hence often has superior lateral resolution. Here, we present spectroscopic OCRT (SOCRT), which uses SOCT images from multiple angles to reconstruct a spectroscopic image with isotropic spatial resolution limited by the OCTlateralresolution. We experimentally show that SOCRT can estimate bead size based on Mie theory at simultaneously high spectral and isotropic spatial resolution. We also applied SOCRT to a biological sample, achieving axial resolution enhancement limited by the lateral resolution.more » « less
-
Optical coherence tomography (OCT) has seen widespread success as anin vivoclinical diagnostic 3D imaging modality, impacting areas including ophthalmology, cardiology, and gastroenterology. Despite its many advantages, such as high sensitivity, speed, and depth penetration, OCT suffers from several shortcomings that ultimately limit its utility as a 3D microscopy tool, such as its pervasive coherent speckle noise and poor lateral resolution required to maintain millimeter-scale imaging depths. Here, we present 3D optical coherence refraction tomography (OCRT), a computational extension of OCT that synthesizes an incoherent contrast mechanism by combining multiple OCT volumes, acquired across two rotation axes, to form a resolution-enhanced, speckle-reduced, refraction-corrected 3D reconstruction. Our label-free computational 3D microscope features a novel optical design incorporating a parabolic mirror to enable the capture of 5D plenoptic datasets, consisting of millimetric 3D fields of view over up to without moving the sample. We demonstrate that 3D OCRT reveals 3D features unobserved by conventional OCT in fruit fly, zebrafish, and mouse samples.more » « less
-
Multiple scattering is a major barrier that limits the optical imaging depth in scattering media. In order to alleviate this effect, we demonstrate aberration-diverse optical coherence tomography (AD-OCT), which exploits the phase correlation between the deterministic signals from single-scattered photons to suppress the random background caused by multiple scattering and speckle. AD-OCT illuminates the sample volume with diverse aberrated point spread functions, and computationally removes these intentionally applied aberrations. After accumulating 12 astigmatism-diverse OCT volumes, we show a 10 dB enhancement in signal-to-background ratio via a coherent average of reconstructed signals from a USAF target located 7.2 scattering mean free paths below a thick scattering layer, and a 3× speckle contrast reduction from an incoherent average of reconstructed signals inside the scattering layer. This AD-OCT method, when implemented using astigmatic illumination, is a promising approach for ultra-deep volumetric optical coherence microscopy.more » « less
