An official website of the United States government
PurposeTo develop an improved k‐space reconstruction method using scan‐specific deep learning that is trained on autocalibration signal (ACS) data. TheoryRobust artificial‐neural‐networks for k‐space interpolation (RAKI) reconstruction trains convolutional neural networks on ACS data. This enables nonlinear estimation of missing k‐space lines from acquired k‐space data with improved noise resilience, as opposed to conventional linear k‐space interpolation‐based methods, such as GRAPPA, which are based on linear convolutional kernels. MethodsThe training algorithm is implemented using a mean square error loss function over the target points in the ACS region, using a gradient descent algorithm. The neural network contains 3 layers of convolutional operators, with 2 of these including nonlinear activation functions. The noise performance and reconstruction quality of the RAKI method was compared with GRAPPA in phantom, as well as in neurological and cardiac in vivo data sets. ResultsPhantom imaging shows that the proposed RAKI method outperforms GRAPPA at high (≥4) acceleration rates, both visually and quantitatively. Quantitative cardiac imaging shows improved noise resilience at high acceleration rates (rate 4:23% and rate 5:48%) over GRAPPA. The same trend of improved noise resilience is also observed in high‐resolution brain imaging at high acceleration rates. ConclusionThe RAKI method offers a training database‐free deep learning approach for MRI reconstruction, with the potential to improve many existing reconstruction approaches, and is compatible with conventional data acquisition protocols.
more »
« less
Deep learning or interpolation for inverse modelling of heat and fluid flow problems?
Purpose The purpose of this study is to compare interpolation algorithms and deep neural networks for inverse transfer problems with linear and nonlinear behaviour. Design/methodology/approach A series of runs were conducted for a canonical test problem. These were used as databases or “learning sets” for both interpolation algorithms and deep neural networks. A second set of runs was conducted to test the prediction accuracy of both approaches. Findings The results indicate that interpolation algorithms outperform deep neural networks in accuracy for linear heat conduction, while the reverse is true for nonlinear heat conduction problems. For heat convection problems, both methods offer similar levels of accuracy. Originality/value This is the first time such a comparison has been made.
more »
« less
- PAR ID:
- 10303915
- Date Published:
- Journal Name:
- International Journal of Numerical Methods for Heat & Fluid Flow
- Volume:
- 31
- Issue:
- 9
- ISSN:
- 0961-5539
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Deep Learning (DL), in particular deep neural networks (DNN), by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, and hence promoting better generalization. This paper develops a model-constrained deep learning approach and its variant TNet—a Tikhonov neural network—that are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solution with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions in any desired accuracy for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for 1D deconvolution, 5 for inverse 2D heat conductivity problem, 100 for inverse initial conditions for time-dependent 2D Burgers’ equation, and 50 for inverse initial conditions for 2D Navier-Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. This is possible owing to the model-constrained term, replications, and randomization.more » « less
-
This paper studies the prediction task of tensor-on-tensor regression in which both covariates and responses are multi-dimensional arrays (a.k.a., tensors) across time with arbitrary tensor order and data dimension. Existing methods either focused on linear models without accounting for possibly nonlinear relationships between covariates and responses, or directly employed black-box deep learning algorithms that failed to utilize the inherent tensor structure. In this work, we propose a Factor Augmented Tensor-on-Tensor Neural Network (FATTNN) that integrates tensor factor models into deep neural networks. We begin with summarizing and extracting useful predictive information (represented by the ``factor tensor'') from the complex structured tensor covariates, and then proceed with the prediction task using the estimated factor tensor as input of a temporal convolutional neural network. The proposed methods effectively handle nonlinearity between complex data structures, and improve over traditional statistical models and conventional deep learning approaches in both prediction accuracy and computational cost. By leveraging tensor factor models, our proposed methods exploit the underlying latent factor structure to enhance the prediction, and in the meantime, drastically reduce the data dimensionality that speeds up the computation. The empirical performances of our proposed methods are demonstrated via simulation studies and real-world applications to three public datasets. Numerical results show that our proposed algorithms achieve substantial increases in prediction accuracy and significant reductions in computational time compared to benchmark methods.more » « less
-
Soner, Mete (Ed.)We develop a backward-in-time machine learning algorithm that uses a sequence of neural networks to solve optimal switching problems in energy production, where electricity and fossil fuel prices are subject to stochastic jumps. We then apply this algorithm to a variety of energy scheduling problems, including novel high-dimensional energy production problems. Our experimental results demonstrate that the algorithm performs with accuracy and experiences linear to sub-linear slowdowns as dimension increases, demonstrating the value of the algorithm for solving high-dimensional switching problem. Keywords. Deep neural networks, forward-backward systems of stochastic differential equations, optimal switching, Monte Carlo algorithm, optimal investment in power generation, planning problemsmore » « less
-
Abstract Recent remarkable progress in artificial intelligence (AI) has garnered tremendous attention from researchers, industry leaders, and the general public, who are increasingly aware of AI's growing impact on everyday life. The advancements of AI and deep learning have also significantly influenced the field of nanophotonics. On the one hand, deep learning facilitates data‐driven strategies for optimizing and solving forward and inverse problems of nanophotonic devices. On the other hand, photonic devices offer promising optical platforms for implementing deep neural networks. This review explores both AI for photonic design and photonic implementation of AI. Various deep learning models and their roles in the design of photonic devices are introduced, analyzing the strengths and challenges of these data‐driven methodologies from the perspective of computational cost. Additionally, the potential of optical hardware accelerators for neural networks is discussed by presenting a variety of photonic devices capable of performing linear and nonlinear operations, essential building blocks of neural networks. It is believed that the bidirectional interactions between nanophotonics and AI will drive the coevolution of these two research fields.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1108/HFF-11-2020-0684
