Scenario development in tokamaks is an open area of investigation that can be approached in a variety of different ways. Experimental trial and error has been the traditional method, but this required a massive amount of experimental time and resources. As high fidelity predictive models have become available, offline development and testing of proposed scenarios has become an option to reduce the required experimental resources. The use of predictive models also offers the possibility of using a numerical optimization process to find the controllable inputs that most closely achieve the desired plasma state. However, this type of optimization can require as many as hundreds or thousands of predictive simulation cases to converge to a solution; many of the commonly used high fidelity models have high computational burdens, so it is only reasonable to run a handful of predictive simulations. In order to make use of numerical optimization approaches, a compromise needs to be found between model fidelity and computational burden. This compromise can be achieved using neural networks surrogates of high fidelity models that retain nearly the same level of accuracy as the models they are trained to replicate while reducing the computation time by orders of magnitude. In this work, a model-based numerical optimization tool for scenario development is described. The predictive model used by the optimizer includes neural network surrogate models integrated into the fast Control-Oriented Transport simulation framework. This optimization scheme is able to converge to the optimal values of the controllable inputs that produce the target plasma scenario by running thousands of predictive simulations in under an hour without sacrificing too much prediction accuracy.
Deep Learning Approaches to Surrogates for Solving the Diffusion Equation for Mechanistic Real-World Simulations
In many mechanistic medical, biological, physical, and engineered spatiotemporal dynamic models the numerical solution of partial differential equations (PDEs), especially for diffusion, fluid flow and mechanical relaxation, can make simulations impractically slow. Biological models of tissues and organs often require the simultaneous calculation of the spatial variation of concentration of dozens of diffusing chemical species. One clinical example where rapid calculation of a diffusing field is of use is the estimation of oxygen gradients in the retina, based on imaging of the retinal vasculature, to guide surgical interventions in diabetic retinopathy. Furthermore, the ability to predict blood perfusion and oxygenation may one day guide clinical interventions in diverse settings, i.e., from stent placement in treating heart disease to BOLD fMRI interpretation in evaluating cognitive function (Xie et al., 2019 ; Lee et al., 2020 ). Since the quasi-steady-state solutions required for fast-diffusing chemical species like oxygen are particularly computationally costly, we consider the use of a neural network to provide an approximate solution to the steady-state diffusion equation. Machine learning surrogates, neural networks trained to provide approximate solutions to such complicated numerical problems, can often provide speed-ups of several orders of magnitude compared to direct calculation. Surrogates of PDEs could enable use of larger and more detailed models than are possible with direct calculation and can make including such simulations in real-time or near-real time workflows practical. Creating a surrogate requires running the direct calculation tens of thousands of times to generate training data and then training the neural network, both of which are computationally expensive. Often the practical applications of such models require thousands to millions of replica simulations, for example for parameter identification and uncertainty quantification, each of which gains speed from surrogate use and rapidly recovers the up-front costs of surrogate generation. We use a Convolutional Neural Network to approximate the stationary solution to the diffusion equation in the case of two equal-diameter, circular, constant-value sources located at random positions in a two-dimensional square domain with absorbing boundary conditions. Such a configuration caricatures the chemical concentration field of a fast-diffusing species like oxygen in a tissue with two parallel blood vessels in a cross section perpendicular to the two blood vessels. To improve convergence during training, we apply a training approach that uses roll-back to reject stochastic changes to the network that increase the loss function. The trained neural network approximation is about 1000 times faster than the direct calculation for individual replicas. Because different applications will have different criteria for acceptable approximation accuracy, we discuss a variety of loss functions and accuracy estimators that can help select the best network for a particular application. We briefly discuss some of the issues we encountered with overfitting, mismapping of the field values and the geometrical conditions that lead to large absolute and relative errors in the approximate solution.
more »
« less
- Award ID(s):
- 1720625
- NSF-PAR ID:
- 10359456
- Date Published:
- Journal Name:
- Frontiers in Physiology
- Volume:
- 12
- ISSN:
- 1664-042X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract The reaction-diffusion system is naturally used in chemistry to represent substances reacting and diffusing over the spatial domain. Its solution illustrates the underlying process of a chemical reaction and displays diverse spatial patterns of the substances. Numerical methods like finite element method (FEM) are widely used to derive the approximate solution for the reaction-diffusion system. However, these methods require long computation time and huge computation resources when the system becomes complex. In this paper, we study the physics of a two-dimensional one-component reaction-diffusion system by using machine learning. An encoder-decoder based convolutional neural network (CNN) is designed and trained to directly predict the concentration distribution, bypassing the expensive FEM calculation process. Different simulation parameters, boundary conditions, geometry configurations and time are considered as the input features of the proposed learning model. In particular, the trained CNN model manages to learn the time-dependent behaviour of the reaction-diffusion system through the input time feature. Thus, the model is capable of providing concentration prediction at certain time directly with high test accuracy (mean relative error <3.04%) and 300 times faster than the traditional FEM. Our CNN-based learning model provides a rapid and accurate tool for predicting the concentration distribution of the reaction-diffusion system.
-
Lee, Jonghyun ; Darve, Eric F. ; Kitanidis, Peter K. ; Mahoney, Michael W. ; Karpatne, Anuj ; Farthing, Matthew W. ; Hesser, Tyler (Ed.)Modern design, control, and optimization often require multiple expensive simulations of highly nonlinear stiff models. These costs can be amortized by training a cheap surrogate of the full model, which can then be used repeatedly. Here we present a general data-driven method, the continuous time echo state network (CTESN), for generating surrogates of nonlinear ordinary differential equations with dynamics at widely separated timescales. We empirically demonstrate the ability to accelerate a physically motivated scalable model of a heating system by 98x while maintaining relative error of within 0.2 %. We showcase the ability for this surrogate to accurately handle highly stiff systems which have been shown to cause training failures with common surrogate methods such as Physics-Informed Neural Networks (PINNs), Long Short Term Memory (LSTM) networks, and discrete echo state networks (ESN). We show that our model captures fast transients as well as slow dynamics, while demonstrating that fixed time step machine learning techniques are unable to adequately capture the multi-rate behavior. Together this provides compelling evidence for the ability of CTESN surrogates to predict and accelerate highly stiff dynamical systems which are unable to be directly handled by previous scientific machine learning techniques.more » « less
-
Abstract Two-photon lithography (TPL) is a direct laser writing process that enables the fabrication of cm-scale complex three-dimensional polymeric structures with submicrometer resolution. In contrast to the slow and serial writing scheme of conventional TPL, projection TPL (P-TPL) enables rapid printing of entire layers at once. However, process prediction remains a significant challenge in P-TPL due to the lack of computationally efficient models. In this work, we present machine learning-based surrogate models to predict the outcomes of P-TPL to >98% of the accuracy of a physics-based reaction-diffusion finite element simulation. A classification neural network was trained using data generated from the physics-based simulations. This enabled us to achieve computationally efficient and accurate prediction of whether a set of printing conditions will result in precise and controllable polymerization and the desired printing versus no printing or runaway polymerization. We interrogate this surrogate model to investigate the parameter regimes that are promising for successful printing. We predict combinations of photoresist reaction rate constants that are necessary to print for a given set of processing conditions, thereby generating a set of printability maps. The surrogate models reduced the computational time that is required to generate these maps from more than 10 months to less than a second. Thus, these models can enable rapid and informed selection of photoresists and printing parameters during process control and optimization.more » « less
-
null (Ed.)Photochemical reactions are widely used by academic and industrial researchers to construct complex molecular architectures via mechanisms that often require harsh reaction conditions. Photodynamics simulations provide time-resolved snapshots of molecular excited-state structures required to understand and predict reactivities and chemoselectivities. Molecular excited-states are often nearly degenerate and require computationally intensive multiconfigurational quantum mechanical methods, especially at conical intersections. Non-adiabatic molecular dynamics require thousands of these computations per trajectory, which limits simulations to ∼1 picosecond for most organic photochemical reactions. Westermayr et al. recently introduced a neural-network-based method to accelerate the predictions of electronic properties and pushed the simulation limit to 1 ns for the model system, methylenimmonium cation (CH 2 NH 2 + ). We have adapted this methodology to develop the Python-based, Python Rapid Artificial Intelligence Ab Initio Molecular Dynamics (PyRAI 2 MD) software for the cis – trans isomerization of trans -hexafluoro-2-butene and the 4π-electrocyclic ring-closing of a norbornyl hexacyclodiene. We performed a 10 ns simulation for trans -hexafluoro-2-butene in just 2 days. The same simulation would take approximately 58 years with traditional multiconfigurational photodynamics simulations. We generated training data by combining Wigner sampling, geometrical interpolations, and short-time quantum chemical trajectories to adaptively sample sparse data regions along reaction coordinates. The final data set of the cis – trans isomerization and the 4π-electrocyclic ring-closing model has 6207 and 6267 data points, respectively. The training errors in energy using feedforward neural networks achieved chemical accuracy (0.023–0.032 eV). The neural network photodynamics simulations of trans -hexafluoro-2-butene agree with the quantum chemical calculations showing the formation of the cis -product and reactive carbene intermediate. The neural network trajectories of the norbornyl cyclohexadiene corroborate the low-yielding syn -product, which was absent in the quantum chemical trajectories, and revealed subsequent thermal reactions in 1 ns.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/fphys.2021.667828
