Deep learning (DL) convolutional neural networks (CNNs) have been rapidly adapted in very high spatial resolution (VHSR) satellite image analysis. DLCNN-based computer visions (CV) applications primarily aim for everyday object detection from standard red, green, blue (RGB) imagery, while earth science remote sensing applications focus on geo object detection and classification from multispectral (MS) imagery. MS imagery includes RGB and narrow spectral channels from near- and/or middle-infrared regions of reflectance spectra. The central objective of this exploratory study is to understand to what degree MS band statistics govern DLCNN model predictions. We scaffold our analysis on a case study that uses Arctic tundra permafrost landform features called ice-wedge polygons (IWPs) as candidate geo objects. We choose Mask RCNN as the DLCNN architecture to detect IWPs from eight-band Worldview-02 VHSR satellite imagery. A systematic experiment was designed to understand the impact on choosing the optimal three-band combination in model prediction. We tasked five cohorts of three-band combinations coupled with statistical measures to gauge the spectral variability of input MS bands. The candidate scenes produced high model detection accuracies for the F1 score, ranging between 0.89 to 0.95, for two different band combinations (coastal blue, blue, green (1,2,3) and green, yellow, red (3,4,5)). The mapping workflow discerned the IWPs by exhibiting low random and systematic error in the order of 0.17–0.19 and 0.20–0.21, respectively, for band combinations (1,2,3). Results suggest that the prediction accuracy of the Mask-RCNN model is significantly influenced by the input MS bands. Overall, our findings accentuate the importance of considering the image statistics of input MS bands and careful selection of optimal bands for DLCNN predictions when DLCNN architectures are restricted to three spectral channels.
more »
« less
A region-based deep learning algorithm for detecting and tracking objects in manufacturing plants
In today's competitive production era, the ability to identify and track important objects in a near real-time manner is greatly desired among manufacturers who are moving towards the streamline production. Manually keeping track of every object in a complex manufacturing plant is infeasible; therefore, an automatic system of that functionality is greatly in need. This study was motivated to develop a Mask Region-based Convolutional Neural Network (Mask RCNN) model to semantically segment objects and important zones in manufacturing plants. The Mask RCNN was trained through transfer learning that used a neural network (NN) pre-trained with the MS-COCO dataset as the starting point and further fine-tuned that NN using a limited number of annotated images. Then the Mask RCNN model was modified to have consistent detection results from videos, which was realized through the use of a two-staged detection threshold and the analysis of the temporal coherence information of detected objects. The function of object tracking was added to the system for identifying the misplacement of objects. The effectiveness and efficiency of the proposed system were demonstrated by analyzing a sample of video footages.
more »
« less
- Award ID(s):
- 1646162
- PAR ID:
- 10129789
- Date Published:
- Journal Name:
- The 25th International Conference on Production Research (ICPR’19)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Recent efforts in deploying Deep Neural Networks for object detection in real world applications, such as autonomous driving, assume that all relevant object classes have been observed during training. Quantifying the performance of these models in settings when the test data is not represented in the training set has mostly focused on pixel-level uncertainty estimation techniques of models trained for semantic segmentation. This paper proposes to exploit additional predictions of semantic segmentation models and quantifying its confidences, followed by classification of object hypotheses as known vs. unknown, out of distribution objects. We use object proposals generated by Region Proposal Network (RPN) and adapt distance aware uncertainty estimation of semantic segmentation using Radial Basis Functions Networks (RBFN) for class agnostic object mask prediction. The augmented object proposals are then used to train a classifier for known vs. unknown objects categories. Experimental results demonstrate that the proposed method achieves parallel performance to state of the art methods for unknown object detection and can also be used effectively for reducing object detectors' false positive rate. Our method is well suited for applications where prediction of non-object background categories obtained by semantic segmentation is reliable.more » « less
-
Current metal additive manufacturing (AM) systems suffer from limitations on the minimum feature sizes they can produce during part formation. The microscale selective laser sintering (μ-SLS) system addresses this drawback by enabling the production of parts with minimum feature resolutions of the order of a single micrometer. However, the production of microscale parts is challenging due to unwanted heat conduction within the nanoparticle powder bed. As a result, finite element (FE) thermal models have been developed to predict the evolution of temperature within the particle bed during laser sintering. These thermal models are not only computationally expensive but also must be integrated into an iterative model-based control framework to optimize the digital mask used to control the distribution of laser power. These limitations necessitate the development of a machine learning (ML) surrogate model to quickly and accurately predict the temperature evolution within the μ-SLS particle bed using minimal training data. The regression model presented in this work uses an “Element-by-Element” approach, where models are trained on individual finite elements to learn the relationship between thermal conditions experienced by each element at a given time-step and the element's temperature at the next time-step. An existing bed-scale FE thermal model of the μ-SLS system is used to generate element-by-element tabular training data for the ML model. A data-efficient artificial neural network (NN) is then trained to predict the temperature evolution of a 2D powder-bed over a 2 s sintering window with high accuracy.more » « less
-
Time lapse microscopy is essential for quantifying the dynamics of cells, subcellular organelles and biomolecules. Biologists use different fluorescent tags to label and track the subcellular structures and biomolecules within cells. However, not all of them are compatible with time lapse imaging, and the labeling itself can perturb the cells in undesirable ways. We hypothesized that phase image has the requisite information to identify and track nuclei within cells. By utilizing both traditional blob detection to generate binary mask labels from the stained channel images and the deep learning Mask RCNN model to train a detection and segmentation model, we managed to segment nuclei based only on phase images. The detection average precision is 0.82 when the IoU threshold is to be set 0.5. And the mean IoU for masks generated from phase images and ground truth masks from experts is 0.735. Without any ground truth mask labels during the training time, this is good enough to prove our hypothesis. This result enables the ability to detect nuclei without the need for exogenous labeling.more » « less
-
more » « less
Abstract Objectives Instance-level tooth segmentation extracts abundant localization and shape information from panoramic radiographs (PRs). The aim of this study was to evaluate the performance of a mask refinement network that extracts precise tooth edges.
Methods A public dataset which consists of 543 PRs and 16211 labelled teeth was utilized. The structure of a typical Mask Region-based Convolutional Neural Network (Mask RCNN) was used as the baseline. A novel loss function was designed focus on producing accurate mask edges. In addition to our proposed method, 3 existing tooth segmentation methods were also implemented on the dataset for comparative analysis. The average precisions (APs), mean intersection over union (mIoU), and mean Hausdorff distance (mHAU) were exploited to evaluate the performance of the network.
Results A novel mask refinement region-based convolutional neural network was designed based on Mask RCNN architecture to extract refined masks for individual tooth on PRs. A total of 3311 teeth were correctly detected from 3382 tested teeth in 111 PRs. The AP, precision, and recall were 0.686, 0.979, and 0.952, respectively. Moreover, the mIoU and mHAU achieved 0.941 and 9.7, respectively, which are significantly better than the other existing segmentation methods.
Conclusions This study proposed an efficient deep learning algorithm for accurately extracting the mask of any individual tooth from PRs. Precise tooth masks can provide valuable reference for clinical diagnosis and treatment. This algorithm is a fundamental basis for further automated processing applications.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
