An official website of the United States government
This paper presents a computational method, called Bootstrapped Koopman Direct Encoding (B-KDE) that allows us to approximate the Koopman operator with high accuracy by combining Koopman Direct Encoding (KDE) with a deep neural network. Deep learning has been applied to the Koopman operator method for finding an effective set of observable functions. Training the network, however, inevitably faces difficulties such as local minima, unless enormous computational efforts are made. Incorporating KDE can solve or alleviate this problem, producing an order of magnitude more accurate prediction. KDE converts the state transition function of a nonlinear system to a linear model in the lifted space of observables that are generated by deep learning. The combined KDE-deep model achieves higher accuracy than that of the deep learning alone. In B-KDE, the combined model is further trained until it reaches a plateau, and this computation is alternated between the neural network learning and the KDE computation. The result of the MSE loss implies that the neural network may get rid of local minima or at least find a smaller local minimum, and further improve the prediction accuracy. The KDE computation however, entails an effective algorithm for computing the inner products of observables and the nonlinear functions of the governing dynamics. Here, a computational method based on the Quasi-Monte Carlo integration is presented. The method is applied to a three-cable suspension robot, which exhibits complex switched nonlinear dynamics due to slack in each cable. The prediction accuracy is compared against its traditional counterparts.
more »
« less
Progress in developing a hybrid deep learning algorithm for identifying and locating primary vertices
The locations of proton-proton collision points in LHC experiments are called primary vertices (PVs). Preliminary results of a hybrid deep learning algorithm for identifying and locating these, targeting the Run 3 incarnation of LHCb, have been described at conferences in 2019 and 2020. In the past year we have made significant progress in a variety of related areas. Using two newer Kernel Density Estimators (KDEs) as input feature sets improves the fidelity of the models, as does using full LHCb simulation rather than the “toy Monte Carlo” originally (and still) used to develop models. We have also built a deep learning model to calculate the KDEs from track information. Connecting a tracks-to-KDE model to a KDE-to-hists model used to find PVs provides a proof-of-concept that a single deep learning model can use track information to find PVs with high efficiency and high fidelity. We have studied a variety of models systematically to understand how variations in their architectures affect performance. While the studies reported here are specific to the LHCb geometry and operating conditions, the results suggest that the same approach could be used by the ATLAS and CMS experiments.
more »
« less
- PAR ID:
- 10311259
- Editor(s):
- Biscarat, C.; Campana, S.; Hegner, B.; Roiser, S.; Rovelli, C.I.; Stewart, G.A.
- Date Published:
- Journal Name:
- EPJ Web of Conferences
- Volume:
- 251
- ISSN:
- 2100-014X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The objective of this work is to detect process instabilities in laser wire directed energy deposition additive manufacturing process using real-time data from a high-speed imaging meltpool sensor. The laser wire directed energy deposition process combines the advantages of powder directed energy deposition and other wire-based additive manufacturing processes, such as wire arc additive manufacturing, as it provides both appreciable resolution and high deposition rates. However, the process tends to create sub-optimal quality parts with poor surface finish, geometric distortion, and delamination in extreme cases. This sub-optimal quality stems from poorly understood thermophysical phenomena and stochastic effects. Hence, flaw formation often occurs despite considerable effort to optimize the processing parameters. In order to overcome this limitation of laser wire directed energy deposition, real-time and accurate monitoring of the process quality state is the essential first step for future closed-loop quality control of the process. In this work we extracted low-level, physically intuitive, features from acquired meltpool images. Physically intuitive features such as meltpool shape, size, and brightness provide a fundamental understanding of the processing regimes that are understandable by human operators. These physically intuitive features were used as inputs to simple machine learning models, such as k-nearest neighbors, support vector machine, etc., trained to classify the process state into one of four possible regimes. Using simple machine learning models forgoes the need to use complex black box modeling such as convolutional neural networks to monitor the high speed meltpool images to determine process stability. The classified regimes identified in this work were stable, dripping, stubbing, and incomplete melting. Regimes such as dripping, stubbing, and incomplete melting regimes fall under the realm of unstable processing conditions that are liable to lead to flaw formation in the laser wire directed energy deposition process. The foregoing three process regimes are the primary source of sub-optimal quality parts due to the degradation of the single-track quality that are the fundamental building block of all manufactured samples. Through a series of single-track experiments conducted over 128 processing conditions, we show that the developed approach is capable of accurately classifying the process state with a statistical fidelity approaching 90% F-score. This level of statistical fidelity was achieved using eight physically intuitive meltpool morphology and intensity features extracted from 159,872 meltpool images across all 128 process conditions. These eight physically intuitive features were then used for the training and testing of a support vector machine learning model. This prediction fidelity achieved using physically intuitive features is at par with computationally intense deep learning methods such as convolutional neural networks.more » « less
-
By querying approximate surrogate models of different fidelity as available information sources, Multi-Fidelity Bayesian Optimization (MFBO) aims at optimizing unknown functions that are costly or infeasible to evaluate. Existing MFBO methods often assume that approximate surrogates have consistently high or low fidelity across the input domain. However, approximate evaluations from the same surrogate can have different fidelity at different input regions due to data availability and model constraints, especially when considering machine learning surrogates. In this work, we investigate MFBO when multi-fidelity approximations have input-dependent fidelity. By explicitly capturing input dependency for multi-fidelity queries in a Gaussian Process (GP), our new input-dependent MFBO (iMFBO) with learnable noise models better captures the fidelity of each information source in an intuitive way. We further design a new acquisition function for iMFBO and prove that the queries selected by iMFBO have higher quality than those by naive MFBO methods, with a derived sub-linear regret bound. Experiments on both synthetic and real-world data demonstrate its superior empirical performance.more » « less
-
Catalysis is inherently complex. The lack of precise knowledge available to experimental researchers about the microenvironment, catalytic sites, mechanisms, and changes that occur under reaction conditions has hindered the effectiveness of deep-learning artificial intelligence algorithms to predict catalyst behavior under reaction conditions. Given the type and quality of data available in the scientific literature, there are still open questions on how machine learning can be used by experimentalists working in the field of catalysis to accelerate catalyst design. Here, we present a framework that leverages large language models to extract textual data from known and trusted sources to automatically generate large, but relatively low-fidelity, experimental catalysis data sets across many research groups. We also show that instead of using deep-learning models, which require highquality data, shallow learning models with posthoc interpretability can extract valuable information about experimental catalytic systems from these low-fidelity data sets. The innovation of this work lies not in the model development but in the prompt engineering, data encoding, and question architectures employed to extract meaningful information. We applied this framework to two different model reactions: the electrocatalytic reduction of carbon dioxide and the electrocatalytic oxygen reduction reaction. We showcase that this framework has the ability to uncover known and established facts within the catalysis community, such as the catalytic properties of Cu, as well as novel insights, including the critical role of voltages above a certain threshold in producing multicarbon products from CO2. We anticipate that this proposed framework will serve as an entryway for experimental catalytic researchers to utilize machine learning to rapidly process literature, generate novel hypotheses, and design experiments to accelerate catalyst development.more » « less
-
Abstract Ranking models are the main components of information retrieval systems. Several approaches to ranking are based on traditional machine learning algorithms using a set of hand-crafted features. Recently, researchers have leveraged deep learning models in information retrieval. These models are trained end-to-end to extract features from the raw data for ranking tasks, so that they overcome the limitations of hand-crafted features. A variety of deep learning models have been proposed, and each model presents a set of neural network components to extract features that are used for ranking. In this paper, we compare the proposed models in the literature along different dimensions in order to understand the major contributions and limitations of each model. In our discussion of the literature, we analyze the promising neural components, and propose future research directions. We also show the analogy between document retrieval and other retrieval tasks where the items to be ranked are structured documents, answers, images and videos.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1051/epjconf/202125104012
