Medical image analysis using deep learning has recently been
prevalent, showing great performance for various downstream tasks including
medical image segmentation and its sibling, volumetric image
segmentation. Particularly, a typical volumetric segmentation network
strongly relies on a voxel grid representation which treats volumetric data
as a stack of individual voxel `slices', which allows learning to segment
a voxel grid to be as straightforward as extending existing image-based
segmentation networks to the 3D domain. However, using a voxel grid representation
requires a large memory footprint, expensive test-time and limiting
the scalability of the solutions. In this paper, we propose Point-Unet,
a novel method that incorporates the eciency of deep learning with 3D
point clouds into volumetric segmentation. Our key idea is to rst predict
the regions of interest in the volume by learning an attentional probability
map, which is then used for sampling the volume into a sparse point cloud
that is subsequently segmented using a point-based neural network. We
have conducted the experiments on the medical volumetric segmentation
task with both a small-scale dataset Pancreas and large-scale datasets
BraTS18, BraTS19, and BraTS20 challenges. A comprehensive benchmark
on di erent metrics has shown that our context-aware Point-Unet
robustly outperforms the SOTA voxel-based networks at both accuracies,
memory usage during training, and time consumption during testing.
more »
« less
Joint Low Dose CT Denoising And Kidney Segmentation
In this research, both image denoising and kidney segmentation tasks are addressed jointly via one multitask deep convolutional network. This multitasking scheme yields better results for both tasks compared to separate single-task methods. Also, to the best of our knowledge, this is a first time attempt at addressing these joint tasks in low-dose CT scans (LDCT). This new network is a conditional generative adversarial network (C-GAN) and is an extension of the image-to-image translation network. To investigate the generalized nature of the network, two other conventional single task networks are also exploited, including the well-known 2D UNet method for segmentation and the more recently proposed method WGAN for LDCT denoising. Implementation results proved that the proposed method outperforms UNet and WGAN for both tasks.
more »
« less
- NSF-PAR ID:
- 10185416
- Date Published:
- Journal Name:
- IEEE 17th International Symposium on Biomedical Imaging Workshops (ISBI)
- Page Range / eLocation ID:
- 1 to 4
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Medical image segmentation is one of the most challenging tasks in medical image analysis and widely developed for many clinical applications. While deep learning-based approaches have achieved impressive performance in semantic segmentation, they are limited to pixel-wise settings with imbalanced-class data problems and weak boundary object segmentation in medical images. In this paper, we tackle those limitations by developing a new two-branch deep network architecture which takes both higher level features and lower level features into account. The first branch extracts higher level feature as region information by a common encoder-decoder network structure such as Unet and FCN, whereas the second branch focuses on lower level features as support information around the boundary and processes in parallel to the first branch. Our key contribution is the second branch named Narrow Band Active Contour (NB-AC) attention model which treats the object contour as a hyperplane and all data inside a narrow band as support information that influences the position and orientation of the hyperplane. Our proposed NB-AC attention model incorporates the contour length with the region energy involving a fixed-width band around the curve or surface. The proposed network loss contains two fitting terms: (i) a high level feature (i.e., region) fitting term from the first branch; (ii) a lower level feature (i.e., contour) fitting term from the second branch including the (ii1) length of the object contour and (ii2) regional energy functional formed by the homogeneity criterion of both the inner band and outer band neighboring the evolving curve or surface. The proposed NB-AC loss can be incorporated into both 2D and 3D deep network architectures. The proposed network has been evaluated on different challenging medical image datasets, including DRIVE, iSeg17, MRBrainS18 and Brats18. The experimental results have shown that the proposed NB-AC loss outperforms other mainstream loss functions: Cross Entropy, Dice, Focal on two common segmentation frameworks Unet and FCN. Our 3D network which is built upon the proposed NB-AC loss and 3DUnet framework achieved state-of-the-art results on multiple volumetric datasets.more » « less
-
Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.more » « less
-
This paper introduces a new image smoothing filter based on a feed-forward convolutional neural network (CNN) in presence of impulse noise. This smoothing filter integrates a very deep architecture, a regularization method, and a batch normalization process. This fully integrated approach yields an effectively denoised and smoothed image yielding a high similarity measure with the original noise free image. Specific structural metrics are used to assess the denoising process and how effective was the removal of the impulse noise. This CNN model can also deal with other noise levels not seen during the training phase. The proposed CNN model is constructed through a 20-layer network using 400 images from the Berkeley Segmentation Dataset (BSD) in the training phase. Results are obtained using the standard testing set of 8 natural images not seen in the training phase. The merits of this proposed method are weighed in terms of high similarity measure and structural metrics that conform to the original image and compare favorably to the different results obtained using state-of-art denoising filters.more » « less
-
more » « less
This paper presents a few comprehensive experimental studies for automated Structural Damage Detection (SDD) in extreme events using deep learning methods for processing 2D images. In the first study, a 152-layer Residual network (ResNet) is utilized to classify multiple classes in eight SDD tasks, which include identification of scene levels, damage levels, and material types. The proposed ResNet achieved high accuracy for each task while the positions of the damage are not identifiable. In the second study, the existing ResNet and a segmentation network (U-Net) are combined into a new pipeline, cascaded networks, for categorizing and locating structural damage. The results show that the accuracy of damage detection is significantly improved compared to only using a segmentation network. In the third and fourth studies, end-to-end networks are developed and tested as a new solution to directly detect cracks and spalling in the image collections of recent large earthquakes. One of the proposed networks can achieve an accuracy above 67 .6% for all tested images at various scales and resolutions, and shows its robustness for these human-free detection tasks. As a preliminary field study, we applied the proposed method to detect damage in a concrete structure that was tested to study its progressive collapse performance. The experiments indicate that these solutions for automatic detection of structural damage using deep learning methods are feasible and promising. The training datasets and codes will be made available for the public upon the publication of this paper.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ISBIWorkshops50223.2020.9153392
