An official website of the United States government
Deep learning (DL) convolutional neural networks (CNNs) have been rapidly adapted in very high spatial resolution (VHSR) satellite image analysis. DLCNN-based computer visions (CV) applications primarily aim for everyday object detection from standard red, green, blue (RGB) imagery, while earth science remote sensing applications focus on geo object detection and classification from multispectral (MS) imagery. MS imagery includes RGB and narrow spectral channels from near- and/or middle-infrared regions of reflectance spectra. The central objective of this exploratory study is to understand to what degree MS band statistics govern DLCNN model predictions. We scaffold our analysis on a case study that uses Arctic tundra permafrost landform features called ice-wedge polygons (IWPs) as candidate geo objects. We choose Mask RCNN as the DLCNN architecture to detect IWPs from eight-band Worldview-02 VHSR satellite imagery. A systematic experiment was designed to understand the impact on choosing the optimal three-band combination in model prediction. We tasked five cohorts of three-band combinations coupled with statistical measures to gauge the spectral variability of input MS bands. The candidate scenes produced high model detection accuracies for the F1 score, ranging between 0.89 to 0.95, for two different band combinations (coastal blue, blue, green (1,2,3) and green, yellow, red (3,4,5)). The mapping workflow discerned the IWPs by exhibiting low random and systematic error in the order of 0.17–0.19 and 0.20–0.21, respectively, for band combinations (1,2,3). Results suggest that the prediction accuracy of the Mask-RCNN model is significantly influenced by the input MS bands. Overall, our findings accentuate the importance of considering the image statistics of input MS bands and careful selection of optimal bands for DLCNN predictions when DLCNN architectures are restricted to three spectral channels.
more »
« less
A region-based deep learning algorithm for detecting and tracking objects in manufacturing plants
In today's competitive production era, the ability to identify and track important objects in a near real-time manner is greatly desired among manufacturers who are moving towards the streamline production. Manually keeping track of every object in a complex manufacturing plant is infeasible; therefore, an automatic system of that functionality is greatly in need. This study was motivated to develop a Mask Region-based Convolutional Neural Network (Mask RCNN) model to semantically segment objects and important zones in manufacturing plants. The Mask RCNN was trained through transfer learning that used a neural network (NN) pre-trained with the MS-COCO dataset as the starting point and further fine-tuned that NN using a limited number of annotated images. Then the Mask RCNN model was modified to have consistent detection results from videos, which was realized through the use of a two-staged detection threshold and the analysis of the temporal coherence information of detected objects. The function of object tracking was added to the system for identifying the misplacement of objects. The effectiveness and efficiency of the proposed system were demonstrated by analyzing a sample of video footages.
more »
« less
- Award ID(s):
- 1646162
- PAR ID:
- 10129789
- Date Published:
- Journal Name:
- The 25th International Conference on Production Research (ICPR’19)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Recent efforts in deploying Deep Neural Networks for object detection in real world applications, such as autonomous driving, assume that all relevant object classes have been observed during training. Quantifying the performance of these models in settings when the test data is not represented in the training set has mostly focused on pixel-level uncertainty estimation techniques of models trained for semantic segmentation. This paper proposes to exploit additional predictions of semantic segmentation models and quantifying its confidences, followed by classification of object hypotheses as known vs. unknown, out of distribution objects. We use object proposals generated by Region Proposal Network (RPN) and adapt distance aware uncertainty estimation of semantic segmentation using Radial Basis Functions Networks (RBFN) for class agnostic object mask prediction. The augmented object proposals are then used to train a classifier for known vs. unknown objects categories. Experimental results demonstrate that the proposed method achieves parallel performance to state of the art methods for unknown object detection and can also be used effectively for reducing object detectors' false positive rate. Our method is well suited for applications where prediction of non-object background categories obtained by semantic segmentation is reliable.more » « less
-
Current metal additive manufacturing (AM) systems suffer from limitations on the minimum feature sizes they can produce during part formation. The microscale selective laser sintering (μ-SLS) system addresses this drawback by enabling the production of parts with minimum feature resolutions of the order of a single micrometer. However, the production of microscale parts is challenging due to unwanted heat conduction within the nanoparticle powder bed. As a result, finite element (FE) thermal models have been developed to predict the evolution of temperature within the particle bed during laser sintering. These thermal models are not only computationally expensive but also must be integrated into an iterative model-based control framework to optimize the digital mask used to control the distribution of laser power. These limitations necessitate the development of a machine learning (ML) surrogate model to quickly and accurately predict the temperature evolution within the μ-SLS particle bed using minimal training data. The regression model presented in this work uses an “Element-by-Element” approach, where models are trained on individual finite elements to learn the relationship between thermal conditions experienced by each element at a given time-step and the element's temperature at the next time-step. An existing bed-scale FE thermal model of the μ-SLS system is used to generate element-by-element tabular training data for the ML model. A data-efficient artificial neural network (NN) is then trained to predict the temperature evolution of a 2D powder-bed over a 2 s sintering window with high accuracy.more » « less
-
null (Ed.)A widely-regarded approach in Printed Circuit Board (PCB) reverse engineering (RE) uses non-destructive Xray computed tomography (CT) to produce three-dimensional volumes with several slices of data corresponding to multi-layered PCBs. The noise sources specific to X-ray CT and variability from designers make it difficult to acquire the features needed for the RE process. Hence, these X-ray CT images require specialized image processing techniques to examine the various features of a single PCB to later be translated to a readable CAD format. Previously, we presented an approach where the Hough Circle Transform was used for initial feature detection, and then an iterative false positive removal process was developed specifically for detecting vias on PCBs. Its performance was compared to an off-the-shelf application of the Mask Region-based Convolutional Network (M-RCNN). M-RCNN is an excellent deep learning approach that is able to localize and classify numerous objects of different scales within a single image. In this paper, we present a version of M-RCNN that is fine-tuned for via detection. Changes include polygon boundary annotations on the single X-ray images of vias for training and transfer learning to leverage the full potential of the network. We discuss the challenges of detecting vias using deep learning, our working solution, and our experimental procedure. Additionally, we provide a qualitative evaluation of our approach and use quantitative metrics to compare the proposed approach with the previous iterative one.more » « less
-
Time lapse microscopy is essential for quantifying the dynamics of cells, subcellular organelles and biomolecules. Biologists use different fluorescent tags to label and track the subcellular structures and biomolecules within cells. However, not all of them are compatible with time lapse imaging, and the labeling itself can perturb the cells in undesirable ways. We hypothesized that phase image has the requisite information to identify and track nuclei within cells. By utilizing both traditional blob detection to generate binary mask labels from the stained channel images and the deep learning Mask RCNN model to train a detection and segmentation model, we managed to segment nuclei based only on phase images. The detection average precision is 0.82 when the IoU threshold is to be set 0.5. And the mean IoU for masks generated from phase images and ground truth masks from experts is 0.735. Without any ground truth mask labels during the training time, this is good enough to prove our hypothesis. This result enables the ability to detect nuclei without the need for exogenous labeling.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
