Resistance Spot Welding (RSW) is one of the most vital processes of industrial assemblies. Widely used within aerospace and automotive sectors, this joining operation creates permanent bonds between sheet metal parts [1]. RSW employs an electrical current to melt a localized area of the components within a stack-up. This area cools and solidifies into a nugget, a continuous region of metal bridging the components [2]. The advantage of RSW over other joining operations is that it only takes a single step to form the bond. Conversely, fasteners require an additional pre-processing step in addition to needing additional components that can increase the overall weight of the assembly.
These benefits are offset by the inherent inconsistency of the RSW process. This inconsistency is the result of the many mechanical phenomena that occur during the formation of the nugget. Electrical, thermal, fluid, and structural mechanics operate and influence one another through the process, adding many avenues of uncertainty. While universal manufacturing concerns, such as tool wear or fit-up conditions, are present, the multiple phase changes make material variations, such as grain structure or element composition, much more impactful [3]. This complexity and inconsistency have two detrimental effects: quality control and process planning. For quality control, confirming if a nugget has formed is trivial, but there are two primary defects that could have occurred during the process. The first is an underdeveloped nugget, which would lack the strength of a fully formed bond. The second defect is expulsion, which is a sudden ejection of molten metal from the forming nugget, again leading to a weaker bond. At present, determining if an adequate nugget has formed is primarily done with costly and time-consuming ultrasonic or x-ray testing [4]. With respect to process planning, most welding schedules are developed by trial-and-error to determine the input parameters required to maximize the likelihood of a successful weld [5]. This makes it costly and inefficient to develop new stack-ups, stymieing innovation. In order to improve quality control and streamline schedule development, inline process monitoring is needed on the shop floor.
Process monitoring is one of the main pillars of "Industry 4.0". It allows for data-driven analytics and algorithms to improve defect detection, quality assurance, and schedule planning [6]. But gaining these benefits comes with a cost. While the price of sensors is obvious, there are other, more subtle, investments needed for producers to transition to this new paradigm. Firstly, mature computational hardware and networks are needed to handle the data produced by the sensors and the algorithms deployed to analyze the process. Secondly, installation of sensors into extant production lines incur downtime, lowering productivity [7]. Luckily, many production RSW lines are already equipped with process sensors and appropriate IT infrastructure. This makes RSW a good starting point for manufacturers to explore the benefits of inline process monitoring with minimal investment.
The most used signal in industry and research is the Dynamic Resistance (DR) due to its ability to reflect the thermal and electrical mechanics [8]. Classical process analysis techniques have been used in concert with the DR signal to detect improper sheet fit-up conditions [9], electrode misalignment [10], and weld nugget strength [11]. The signal has also shown promise with Machine Learning (ML) algorithms. Recurrent neural networks have been investigated to predict the Heat Affected Zone (HAZ) [12], a neuro-fuzzy interface model allowed the prediction of weld strength [13], random forest models were able to classify cold or expulsed welds [14], and Support Vector Regression (SVR) was used to predict overall quality [15].
While less used in industry, electrode displacement and expansion force are popular signals for research due to their reflection of the thermal strain and expansion that occur during welding [16]. These signals give a more rounded picture of the RSW process, providing a more thorough explanation to what is occurring as opposed to just observing the electrical signals. Like the DR, the force and displacement signals have been shown to be useful for inline fault monitoring. They have been used to detect expulsion occurrences [17 ,18], different sheet fit-up conditions [19], and shunting, a defect caused when the welding current partially flows through a previous nugget [20]. Additionally, they have been used to predict weld penetration [21] and electrode wear [22].
Combining the information from all three discussed sensors allows for the observation and detection of individual phases of the RSW process [23]. Features derived from these phases, and the multiple process signals, can reveal important process information when paired with a simple ML model. The myriad of applications makes it clear that the best decision making can be made with the most signals. For lines with extant DR sensing capabilities, incorporating force and displacement signals is possible using physical sensors. Alternatively, we can also employ virtual sensors to generate the desired information from the existing DR signal, removing the temporal and financial cost of installation.
The generation of a processing signal through indirect measurement can be done with a virtual sensor. This "sensor" is a model that converts one or more signals and inputs into another desired, but unmonitored, signal. For industrial purposes, this method was developed in chemical production plants to observe chemical concentrations [24]. For manufacturing, virtual sensing has been sparsely researched, but the technology shows promise. For example, an empiricalbased virtual sensor was developed to correlate input pressures to nozzle pressure in injection molding [25]. Additionally, simple ML models, such as SVM or Particle Filters, have been used to estimate tool wear in cutting machines through the vibration and force signals [26,27]. For welding, virtual sensing seems to be entirely unexplored, in spite of the vast amount of data and popularity for the process. In this, there is an opportunity to leverage advances in ML signal generation to improve the understanding and monitoring of RSW.
With the rapid advancement of the ML field, there are a myriad of avenues that could be explored for the general structure of a virtual sensor. The obvious choice is the popular Generative Adversarial Network (GAN) [28]. This structure works by training a generator and discriminator. The discriminator is fed both real signals and signals made by the generator. The discriminator determines whether it was given a real or generated signal, making its objective function a maximization of classification accuracy. The generator is trained to "fool" the discriminator by creating signals indistinguishable from the real ones. This structure has been shown to be applicable for signal generation, but it is primarily kept for augmentation of training data as opposed to creating a virtual sensor conversion [29]. Since the purpose of this work is to generate a time series based on given information of the system, the use of forecasting algorithms was explored.
Forecasting models are designed to predict the future given a precedent signal. For manufacturing, these models are typically used in concert with condition monitoring to predict when a machine or tool will fail. Typical models for this are SVR [30] and Long Short-Term Memory (LSTM) [31]. Recently, the development of the transformer model has shown that a feed-forward residual network is a powerful structure for forecasting [32]. Conversely, while transformers outperform earlier forecasting models, they are outclassed by more primitive linear networks [33]. For this reason, the virtual sensor structure takes inspiration from the recently developed Time-series Dense Encoder or TiDE [34]. TiDE combines the powerful skip connections used in LSTM and Transformer models with the forecasting capabilities of an MLP.
This paper presents the development and applicability of a novel virtual sensor for use in RSW production lines. By taking the TiDE architecture as inspiration, we can develop a datadriven virtual sensor model that is able to convert the available electrical based DR signal into the two mechanical based displacement and force signals. The result is the Resistance Inferred Process Time-series by Dense Encoder or RIPTiDE model. RIPTiDE will allow producers to understand and observe more facets of the RSW process while minimizing their investment for better decision making and process monitoring.
This section presents information about residual blocks, the original TiDE network, and the development and final architecture of the RIPTiDE model.
The main building blocks of dense encoder networks are Residual Blocks, shown on FIGURE 1. The defining aspect of these blocks is their inclusion of skip connections. Skip connections were developed when it was realized that deeper networks did not produce better training performance than shallower ones. This runs contrary to the theoretical performance of Neural Networks (NN).
Hypothetically, if a network reaches peak performance at a certain depth, any subsequent layers should "learn" to converge onto the identity function, in effect removing them from the chain. This behavior would cause the network to be mathematically equivalent to the more optimal, but shallower network. However, this is not the case. It was observed that overly developed models demonstrated clearly reduced performance than appropriately shallow ones. This is known as the degradation problem. The cause of this behavior is theorized to be caused by NN models being unable to properly converge to the identity function. To address this problem, Residual Networks (ResNets) were developed, along with their unique inclusion: skip connections [35] Though skip connections were used to develop the LSTM Recurrent Neural Network (RNN), ResNet was the first instance of using them in feed forward networks [36]. Skip connections effectively takes the inputs from one layer and add them to the outputs of a layer further along in the network. Typically, skip connections pass multiple layers before reincorporating the values. Traditionally, skip connections were simply the identity function, but more recent networks use a trainable linear layer instead.
The original TiDE model is presented on the left of FIGURE 2. The network is comprised of residual blocks (blue mosaics) arranged to predict the future of the lookback signal from the present (L) up to a future time horizon(L+H). In addition to the lookback, the network takes two other inputs; attributes and dynamic covariates. The attributes are static constants within the system being predicted. The dynamic covariates are relevant information that changes over the course of the lookback and prediction. The dynamic covariates can be viewed as a signal of length L+H, and there can be multiple dynamic covariate signals depending on the system being forecasted. In order to keep the network inputs reasonable, the dynamic covariates channels are compressed using the Feature Projection block, reducing the total input channels. The inputs are then stacked and fed into the encoder-decoder, (yellow block and pink block respectively) comprised of 𝑛 𝑒 and 𝑛 𝑑 residual blocks respectively. The dense encoder produces the salient features needed to predict the future of the lookback signal. These features are then stacked with the reduced dynamic covariates before being fed into the final temporal decoder to reduce the stacked inputs to a single channel. This resulting signal is then combined with the lookback signal that was mapped to the appropriate length using a linear skip connection (small blue rectangle).
The development of RIPTiDE was based around modifications of the extant TiDE structure to make it applicable to the virtual sensing task. An overview of these changes are shown on the right side of FIGURE 2. The largest change is the removal of the lookback, since this is no longer a forecasting model, necessitating the removal of the associated skip connection. Secondly, the dynamic covariates were replaced with the DR signal, as this correlates with the known parameters across the entire time range. Finally, the outputs are now predicted across the input signal range as opposed to the future.
The first construction of RIPTiDE corresponded with the hyperparameters as proposed in the original TiDE work for power consumption prediction. The full details of this architecture are shown on FIGURE 3. The model used two encoders and decoders, all with a hidden layer size of 1024. The encoder-decoder chain ended with 8 output channels. The temporal decoder had 64 hidden neurons, an overall dropout rate of 50% was used and LayerNorm was not used. Of note, the original TiDE model stated that the encoder and decoder blocks could have separate hyperparameters, but they were always constructed with the same structure. For this reason, we can present them as one repeated component, and display the structure in a Compressed Chart to simplify and streamline the model representation in FIGURE 4. The model was trained to use the DR signal along with the preprogrammed welding force and current to generate electrode displacement and force signals. The initial results of the network showed promise and would accurately recreate the trends of the two target signals, it could even replicate expulsion well. The primary issue with this structure is the generated signal was much noisier than the true values. This is likely because the expulsion cases enact much more noise than normal welds, but the network has generalized to having some noise throughout the signal. In addition, the magnitude of the signal did not always match the true values. These results led to the modification and tuning of the initial model.
During tuning, layer normalization was found to be detrimental, and was not used. Additionally different model structures (i.e., encoder amounts, encoder channels, hidden residual block neurons), dropout rates, and activation functions were explored. This resulted in three modified networks labeled Dense, Dropout 20%, and Sigmoid f.
"Dense" doubled the number of hidden neurons in each residual block, increased the number of encoders to 6, and increased the number of encoder output channels to 16. "Dropout 20%" reduced the dropout rate from 50% to 20%. Finally, "Sigmoid f" replaced the ReLU activation function with the sigmoid activation function.
While the sigmoid function produced worse results, the higher density and lower dropout rate proved to be welcome additions, both keeping closer to the true signal values, while being slightly smoother. In order to promote further smoothness, modifications to the loss function were attempted. Both total variation and L1 losses were added to the MSE loss. Due to magnitude disparity, the Total Variation loss had to be scaled down to one ten thousandths of the original value.
While the total variation did produce smoother curves, it also produced less accurate signals. Conversely, the L1 term provided additional smoothness without the loss of accuracy. With these insights, the tuned RIPTiDE model (RIPTiDE 1.5) was made. It consisted of the "Dense" structure, a 20% dropout rate, and was trained with the MSE plus L1 loss function. The structure is shown on FIGURE 5. The resulting structure is the proposed virtual sensor model for RSW production, dubbed "RIPTiDE 2.0", presented in FIGURE 6. The purpose of this version was to solve the smoothing problem, as the accuracy of the previous models were acceptable at this time. This structure took the previously discussed RIPTiDE 1.5 and added an additional input and network layer to achieve this. The first addition was the inclusion of the input called "Large Drops" This is a sparse signal that only has significant DR drops. Insignificant drops and all rises are set to zero. The term "significant" was established as any drop with a magnitude larger than the greatest rise, shown on EQUATION 1.
where 𝑧 𝑖 is the i th entry of the large drop signal, 𝑥 ∆𝑖 is the i th entry of the finite difference of DR signal. The purpose of this inclusion was to emphasize expulsion events and times of large noise as opposed to it generalizing to them occurring constantly.
The second addition was a convolution layer at the end of the network with a size of 5 and pad of 2. This acts as a trainable moving window filter that is unique to each signal to encourage smoothness. It again used the MSE plus L1 loss function and is the final iteration of the RIPTiDE virtual sensor.
In order to train and validate the proposed model, experimental RSW data was required. These experiments involved hundreds of RSW welds executed under different welding conditions. The resulting process measurement data, including DR, electrode force, and displacement, were recorded, and collected for analysis.
Each weld was manually performed using a Milco RSW gun driven by a WTC medium frequency controller, which measured the secondary welding current. The electrodes were C15000 CuZr button caps with a face diameter of 6mm. A Kistler strain gauge with 2% error precision was mounted on the lower electrode arm to measure the electrode force. A Heidenhain linear encoder with an accuracy of ±5 microns was mounted on the lower electrode arm to measure the electrode displacement. The sheet metal used was Usibor® 1500, which had been coated with a 40-micron layer of Aluminum Silicone. The experiments were performed under 3 unique sets of sheet thickness, welding force, and time. The welding forces used were 585lbs, 899lbs, and 1124lbs with welding times of 170ms, 200ms, and 240ms. The sheet thicknesses (t) used were 1,1.4, and 1.8 mm. These sheets were further differentiated by sorting them based on their measured IDL thickness. These sheets were then welded under varying currents, as detailed in TABLE 1.
The initial configuration involved aligning two sheets to create a stack-up. Subsequently, three spot welds were executed, starting from the center, and progressing to one on each side of the initial weld. Throughout the welding process, if any noticeable expulsion occurred and was observed, the respective weld was designated as having experienced expulsion. Electrodes were dressed between each trial. Due to the varying welding times, all three signal lengths were 0 padded to 240 ms.
Each iteration of the RIPTiDE network was trained using welding data described in SECTION 3. This data was split 80%/20% for training and validation. The networks leveraged three inputs: the DR signal, the input current, and the input force to generate the displacement and force signals. Presented here are the results for the models described in SECTION 2.3. Unless otherwise stated, the models were trained for 5000 epochs with the Adam optimizer with a learning rate of 0.0001.
For RIPTiDE 1.0, the generated signals were compared to the true signals with the MSE loss function. Training was performed for only 1000 epochs since the initial attempt was only exploratory. Example results are shown on FIGURE 7. As previously discussed, the trends were appropriate, but the noise and magnitude were areas for improvement. 3, whereas the other two correspond to the associated loss function. The inclusion of total variation does improve overall smoothness, but at the cost of accuracy. L1 produces a smoother signal than the baseline, but not as much as total variation. L1's greater accuracy, however, makes it a more desirable choice. Displacement. These times could be automatically extracted through an algorithm that required topographic signal processing and comprehensive selection logic. Salient features could then be calculated from these points, such as the values of individual signals at those points and rate of change between two points. These features could then be leveraged to predict coated material's InterDiffusion Layer (IDL). It was also shown that if the phases were known without the use of the two mechanical signals, a network of high accuracy could be produced with only information taken from the DR signal [23].
RIPTiDE was deployed to generate the two mechanical signals needed for the feature extraction algorithm to determine if it would serve as an acceptable substitution for real sensors. Since the original extraction algorithm was tuned specifically for the trends and behavior of the real process signals, there needed to be some slight adjustments, primarily the addition of a smoothing average filter along the generated displacement curve since the noise spikes could drastically impact the Maximum Displacement. These changes led to predicted points that were relatively accurate, though the latter two phases were less precise, as seen on FIGURE 12.
To evaluate the performance of the generated features, all 663 unique DR signals were input into a fully trained RIPTiDE model. The generate displacement and force signals were then used as inputs to the previously discussed feature extractor. From here, two datasets were compiled. The first leveraged only the values of the DR signal at the phase points detected (FIGURE 13.A.1). The second used values from the DR and the two generated signals for its process features (FIGURE 13.A.2). Each data set was used to train an MLP in the same manner as [23]. The results were compared with the network performance of the dataset leveraging only information gleanable from the DR signal (FIGURE 13.B.1), and the theoretical maximum performance of features taken from the DR signal if all key points were known (FIGURE 13.B.2). The results of the IDL prediction are shown on FIGURE 13.
Both RIPTiDE networks produced similar results, with the more feature inclusive network being slightly more accurate. Both were significant improvements over the DR only model, and only slightly below the idealized performance. Examining the standard deviation of the networks shows that the RIPTiDE models were as consistent as the ones presented in the original work, as seen in FIGURE 14. This demonstrates that the virtual sensors are clearly able to add information about the RSW process utilizing only DR signals. Though not as accurate as the true process signals, RIPTiDE has demonstrated the ability reflect the underlying mechanical phenomena of the RSW process and provide a noted performance improvement to monitoring tasks with limited signals.
A computationally efficient technique called RIPTiDE is developed for virtual generation of electrode force and displacement signals upon physically measured DR signal in RSW process monitoring. RSW domain knowledge, such as expulsion defect-induced DR drops, have been integrated into the encoder to enhance the performance and interpretability of data-driven model. Experimental results have shown that RIPTiDE-based virtual sensing could achieve comparable performance as physical sensing in characterizing the RSW process dynamics and predicting the part qualities. Future research will investigate integration of more RSW physical theories or empirical information into the RIPTiDE modeling in order to generate virtual signals with less noises and better alignment with process dynamics.