skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Process signature-driven high spatio-temporal resolution alignment of multimodal data
We present HiRA-Pro, a novel procedure to align, at high spatio-temporal resolutions, multimodal signals from real-world processes and systems that exhibit diverse transient, nonlinear stochastic dynamics, such as manufacturing machines. It is based on discerning and synchronizing the process signatures of salient kinematic and dynamic events in these disparate signals. HiRA-Pro addresses the challenge of aligning data with sub-millisecond phenomena, where traditional timestamp, external trigger, or clock-based alignment methods fall short. The effectiveness of HiRA-Pro is demonstrated in a smart manufacturing context, where it aligns data from 13+ channels acquired during 3D-printing and milling operations on an Optomec-LENSÂŽ MTS 500 hybrid machine. The aligned data is then voxelized to generate 0.25 second aligned data chunks that correspond to physical voxels on the produced part. The superiority of HiRA-Pro is further showcased through case studies in additive manufacturing, demonstrating improved machine learning-based predictive performance due to precise multimodal data alignment. Specifically, testing classification accuracies improved by almost 35% with the application of HiRA-Pro, even with limited data, allowing for precise localization of artifacts. The paper also provides a comprehensive discussion on the proposed method, its applications, and comparative qualitative analysis with a few other alignment methods. HiRA-Pro achieves temporal-spatial resolutions of 10-1000 𝜇s and 100 𝜇m in order to generate datasets that register with physical voxels on the 3D-printed and milled part. These resolutions are at least an order of magnitude finer than the existing alignment methods that employ individual timestamps, statistical correlations, or common clocks, which achieve precision of hundreds of milliseconds.  more » « less
Award ID(s):
1849085
PAR ID:
10499081
Author(s) / Creator(s):
Publisher / Repository:
ArXiv
Date Published:
Journal Name:
arXivorg
ISSN:
2331-8422
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Assembling 2D materials such as MXenes into functional 3D aerogels using 3D printing technologies gains attention due to simplicity of fabrication, customized geometry and physical properties, and improved performance. Also, the establishment of straightforward electrode fabrication methods with the aim to hinder the restack and/or aggregation of electrode materials, which limits the performance of the electrode, is of great significant. In this study, unidirectional freeze casting and inkjet‐based 3D printing are combined to fabricate macroscopic porous aerogels with vertically aligned Ti3C2Txsheets. The fabrication method is developed to easily control the aerogel microstructure and alignment of the MXene sheets. The aerogels show excellent electromechanical performance so that they can withstand almost 50% compression before recovering to the original shape and maintain their electrical conductivities during continuous compression cycles. To enhance the electrochemical performance, an inkjet‐printed MXene current collector layer is added with horizontally aligned MXene sheets. This combines the superior electrical conductivity of the current collector layer with the improved ionic diffusion provided by the porous electrode. The cells fabricated with horizontal MXene sheets alignment as current collector with subsequent vertical MXene sheets alignment layers show the best electrochemical performance with thickness‐independent capacitive behavior. 
    more » « less
  2. Understanding neural function often requires multiple modalities of data, including electrophysiogical data, imaging techniques, and demographic surveys. In this paper, we introduce a novel neurophysiological model to tackle major challenges in modeling multimodal data. First, we avoid non-alignment issues between raw signals and extracted, frequency-domain features by addressing the issue of variable sampling rates. Second, we encode modalities through “cross-attention” with other modalities. Lastly, we utilize properties of our parent transformer architecture to model long-range dependencies between segments across modalities and assess intermediary weights to better understand how source signals affect prediction. We apply our Multimodal Neurophysiological Transformer (MNT) to predict valence and arousal in an existing open-source dataset. Experiments on non-aligned multimodal time-series show that our model performs similarly and, in some cases, outperforms existing methods in classification tasks. In addition, qualitative analysis suggests that MNT is able to model neural influences on autonomic activity in predicting arousal. Our architecture has the potential to be fine-tuned to a variety of downstream tasks, including for BCI systems. 
    more » « less
  3. The purpose of this research is to provide a framework for Large Scale Additive Metals Manufacturing (LSAMM) in arbitrary directions. Traditionally, slicing and path planning is done along the gravity aligned direction of a part, causing more complex geometrical shapes to have unsupported overhangs. The overhangs can be managed using a part positioner or a powder bed process. A different framework for slicing and building parts out of gravity alignment could improve current capabilities of LSAMM processes. The presented research focuses on segmenting more complex geometrical parts into gravity aligned (GA), non-gravity aligned (NGA), and transition segments to help generate toolpaths. Initial research of segment planning for complex geometrical shapes will be presented, as well as current results from builds completed at the University of Tennessee- Knoxville. The completed builds show that more consistent thermal evolution of a part based on the path sequence and torch angle results in successful builds. 
    more » « less
  4. Abstract Highly sensitive and multimodal sensors have recently emerged for a wide range of applications, including epidermal electronics, robotics, health‐monitoring devices and human–machine interfaces. However, cross‐sensitivity prevents accurate measurements of the target input signals when a multiple of them are simultaneously present. Therefore, the selection of the multifunctional materials and the design of the sensor structures play a significant role in multimodal sensors with decoupled sensing mechanisms. Hence, this review article introduces varying methods to decouple different input signals for realizing truly multimodal sensors. Early efforts explore different outputs to distinguish the corresponding input signals applied to the sensor in sequence. Next, this study discusses the methods for the suppression of the interference, signal correction, and various decoupling strategies based on different outputs to simultaneously detect multiple inputs. The recent insights into the materials' properties, structure effects, and sensing mechanisms in recognition of different input signals are highlighted. The presence of the various decoupling methods also helps avoid the use of complicated signal processing steps and allows multimodal sensors with high accuracy for applications in bioelectronics, robotics, and human–machine interfaces. Finally, current challenges and potential opportunities are discussed in order to motivate future technological breakthroughs. 
    more » « less
  5. In recent years, magnetic particle imaging (MPI) has emerged as a promising imaging technique depicting high sensitivity and spatial resolution. It originated in the early 2000s where it proposed a new approach to challenge the low spatial resolution achieved by using relaxometry in order to measure the magnetic fields. MPI presents 2D and 3D images with high temporal resolution, non‐ionizing radiation, and optimal visual contrast due to its lack of background tissue signal. Traditionally, the images were reconstructed by the conversion of signal from the induced voltage by generating system matrix and X‐space based methods. Because image reconstruction and analyses play an integral role in obtaining precise information from MPI signals, newer artificial intelligence‐based methods are continuously being researched and developed upon. In this work, we summarize and review the significance and employment of machine learning and deep learning models for applications with MPI and the potential they hold for the future. Level of Evidence5 Technical EfficacyStage 1 
    more » « less