skip to main content

Title: In-situ particle analysis with heterogeneous background: a machine learning approach

We propose a novel framework that combines state-of-the-art deep learning approaches with pre- and post-processing algorithms for particle detection in complex/heterogeneous backgrounds common in the manufacturing domain. Traditional methods, like size analyzers and those based on dilution, image processing, or deep learning, typically excel with homogeneous backgrounds. Yet, they often fall short in accurately detecting particles against the intricate and varied backgrounds characteristic of heterogeneous particle–substrate (HPS) interfaces in manufacturing. To address this, we've developed a flexible framework designed to detect particles in diverse environments and input types. Our modular framework hinges on model selection and AI-guided particle detection as its core, with preprocessing and postprocessing as integral components, creating a four-step process. This system is versatile, allowing for various preprocessing, AI model selections, and post-processing strategies. We demonstrate this with an entrainment-based particle delivery method, transferring various particles onto substrates that mimic the HPS interface. By altering particle and substrate properties (e.g., material type, size, roughness, shape) and process parameters (e.g., capillary number) during particle entrainment, we capture images under different ambient lighting conditions, introducing a range of HPS background complexities. In the preprocessing phase, we apply image enhancement and sharpening techniques to improve detection accuracy. Specifically, image enhancement adjusts the dynamic range and histogram, while sharpening increases contrast by combining the high pass filter output with the base image. We introduce an image classifier model (based on the type of heterogeneity), employing Transfer Learning with MobileNet as a Model Selector, to identify the most appropriate AI model (i.e., YOLO model) for analyzing each specific image, thereby enhancing detection accuracy across particle–substrate variations. Following image classification based on heterogeneity, the relevant YOLO model is employed for particle identification, with a distinct YOLO model generated for each heterogeneity type, improving overall classification performance. In the post-processing phase, domain knowledge is used to minimize false positives. Our analysis indicates that the AI-guided framework maintains consistent precision and recall across various HPS conditions, with the harmonic mean of these metrics comparable to those of individual AI model outcomes. This tool shows potential for advancing in-situ process monitoring across multiple manufacturing operations, including high-density powder-based 3D printing, powder metallurgy, extreme environment coatings, particle categorization, and semiconductor manufacturing.

more » « less
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Scientific Reports
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    In manufacturing industries, spherical micro-particles are commonly used as (e.g., brazing powder, metal filler, and 3D printing powder) which are produced with droplet-based particle fabrication techniques. Such processes create spherical morphology but introduce polydispersity and follow a continuous exponential pattern commonly expressed with Rosin-Rammler expression. Sorting those micro-particles in a narrower size range is an important but difficult, costly, and challenging process. Here we demonstrate the successful separation of the particles from a poly-disperse mixture with a particle volume fraction of 10% by dipping process. Nickel-based micro-particles (avg. dia. 5.69 μm) are added in a binder-based liquid carrier system. To encounter the gravitational force, external kinetic energy in the form of agitation is applied to ensure the uniform dispersion of the particles. The cylindrical substrate is prepared and dipped in the ‘pseudo suspension’ to separate the particles by adhering to it. The substrate is dried, and images are taken to characterize the separated particles using image J software. A clear size distribution can be observed which is also plotted. Additionally, a relationship between the process parameter and sorted particles has been established. The proposed method is quick, controllable, and easy to implement, which can be a useful tool for sorting wide-range poly-disperse particles.

    more » « less
  2. Abstract

    In this work, the physical phenomenon of the polydisperse micro-particle entrainment process from density mismatch mixture is investigated with the variation of substrate withdrawal speed. A liquid carrier system (LCS) is prepared by a polymer-based binder and an evaporating solvent. Nickel-based inorganic and spherical particles with a. moderate vol%. of 35% are added to the LCS solution. The cylindrical AISI 1006 mild steel wire substrate is dipped at different withdrawal speed ranging from 0.01 mms-1 to 20 mms-1. The binder vol%. is varied between 6.5% and 10.5%. Once the cylindrical substrate is extracted from the mixture, the surface coverage and the particle size are measured following the image analysis technique. The average particle size, coating thickness and the surface packing coverage by the particles are increasing with the higher withdrawal speed of the substrate. We observed relatively low size of particles (< 10 micrometers) as well as low surface coverage (∼33%) when the withdrawal speed remains at 0.01 mm/s. However, with high withdrawal speed (20 mm/s), we found all sizes of particles present on the substrate with a surface coverage of over 90%. The finding of this research will help to understand the high-volume solid transfer technique and develop a novel manufacturing process.

    more » « less
  3. Abstract

    Powder bed fusion (PBF) is an additive manufacturing process in which laser heat liquefies blown powder particles on top of a powder bed, and cooling solidifies the melted powder particles. During this process, the laser beam heat interacts with the powder causing thermal emission and affecting the melt pool. This paper aims to predict heat emission in PBF by harnessing the strengths of recurrent neural networks. Long short-term memory (LSTM) networks are developed to learn from sequential data (emission readings), while the learning is guided by process physics including laser power, laser speed, layer number, and scanning patterns. To reduce the computational efforts on model training, the LSTM models are integrated with a new approach for down-sampling the pyrometry raw data and extracting useful statistical features from raw data. The structure and hyperparameters of the LSTM model reflect several iterations of tuning based on the training on the pyrometer readings data. Results reveal useful knowledge on how raw pyrometer data should be processed to work the best with LSTM, how physics features are informative in predicting overheating, and the effectiveness of physics-guided LSTM in emission prediction.

    more » « less
  4. null (Ed.)
    Process optimization for directed-energy-deposition, an industrial laser-based additive manufacturing technique, is a time-intensive endeavor for manufacturers. Herein we investigate the use of a modified analytical process-model based on powder-bed-fusion techniques, to predict quality build parameters by incorporating the effects of three key parameters: laser-power, scanning-speed, and powder flowrate. Titanium alloy (Ti6Al4V) tracks of varying parameters were built, studied, and used to predict parameters for quality builds used at different parameters. The model agreed well with experimental build quality at powder flowrates less than 6.5g/min, whereas, higher flowrates created significant unmelted-particle regions, despite optimal parameter predictions. Processing of multi-layer bulk samples revealed that parameters in the optimal range account for relative densities >99%, indicating quality bulk processing parameters. Our results indicate that process modeling with the incorporation of powder feedrate as a key parameter is possible using a commercial laser-based additive manufacturing system. 
    more » « less
  5. Purpose

    The purpose of this study is to develop a deep learning framework for additive manufacturing (AM), that can detect different defect types without being trained on specific defect data sets and can be applied for real-time process control.


    This study develops an explainable artificial intelligence (AI) framework, a zero-bias deep neural network (DNN) model for real-time defect detection during the AM process. In this method, the last dense layer of the DNN is replaced by two consecutive parts, a regular dense layer denoted (L1) for dimensional reduction, and a similarity matching layer (L2) for equal weight and non-biased cosine similarity matching. Grayscale images of 3D printed samples acquired during printing were used as the input to the zero-bias DNN.


    This study demonstrates that the approach is capable of successfully detecting multiple types of defects such as cracks, stringing and warping with high accuracy without any prior training on defective data sets, with an accuracy of 99.5%.

    Practical implications

    Once the model is set up, the computational time for anomaly detection is lower than the speed of image acquisition indicating the potential for real-time process control. It can also be used to minimize manual processing in AI-enabled AM.


    To the best of the authors’ knowledge, this is the first study to use zero-bias DNN, an explainable AI approach for defect detection in AM.

    more » « less