skip to main content


Title: Global Exponential Stability of a Neural Network for Inverse Variational Inequalities
We investigate the convergence properties of a projected neural network for solving inverse variational inequalities. Under standard assumptions, we establish the exponential stability of the proposed neural network. A discrete version of the proposed neural network is considered, leading to a new projection method for solving inverse variational inequalities, for which we obtain the linear convergence. We illustrate the effectiveness of the proposed neural network and its explicit discretization by considering applications in the road pricing problem arising in transportation science. The results obtained in this paper provide a positive answer to a recent open question and improve several recent results in the literature.  more » « less
Award ID(s):
2047793
NSF-PAR ID:
10318501
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Journal of optimization theory and applications
Volume:
190
ISSN:
1573-2878
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In recent years, large convolutional neural networks have been widely used as tools for image deblurring, because of their ability in restoring images very precisely. It is well known that image deblurring is mathematically modeled as an ill-posed inverse problem and its solution is difficult to approximate when noise affects the data. Really, one limitation of neural networks for deblurring is their sensitivity to noise and other perturbations, which can lead to instability and produce poor reconstructions. In addition, networks do not necessarily take into account the numerical formulation of the underlying imaging problem when trained end-to-end. In this paper, we propose some strategies to improve stability without losing too much accuracy to deblur images with deep-learning-based methods. First, we suggest a very small neural architecture, which reduces the execution time for training, satisfying a green AI need, and does not extremely amplify noise in the computed image. Second, we introduce a unified framework where a pre-processing step balances the lack of stability of the following neural-network-based step. Two different pre-processors are presented. The former implements a strong parameter-free denoiser, and the latter is a variational-model-based regularized formulation of the latent imaging problem. This framework is also formally characterized by mathematical analysis. Numerical experiments are performed to verify the accuracy and stability of the proposed approaches for image deblurring when unknown or not-quantified noise is present; the results confirm that they improve the network stability with respect to noise. In particular, the model-based framework represents the most reliable trade-off between visual precision and robustness. 
    more » « less
  2. The plug-and-play priors (PnP) and regularization by denoising (RED) methods have become widely used for solving inverse problems by leveraging pre-trained deep denoisers as image priors. While the empirical imaging performance and the theoretical convergence properties of these algorithms have been widely investigated, their recovery properties have not previously been theoretically analyzed. We address this gap by showing how to establish theoretical recovery guarantees for PnP/RED by assuming that the solution of these methods lies near the fixed-points of a deep neural network. We also present numerical results comparing the recovery performance of PnP/RED in compressive sensing against that of recent compressive sensing algorithms based on generative models. Our numerical results suggest that PnP with a pre-trained artifact removal network provides significantly better results compared to the existing state-of-the-art methods. 
    more » « less
  3. Recent advancements in physics-informed machine learning have contributed to solving partial differential equations through means of a neural network. Following this, several physics-informed neural network works have followed to solve inverse problems arising in structural health monitoring. Other works involving physics-informed neural networks solve the wave equation with partial data and modeling wavefield data generator for efficient sound data generation. While a lot of work has been done to show that partial differential equations can be solved and identified using a neural network, little work has been done the same with more basic machine learning (ML) models. The advantage with basic ML models is that the parameters learned in a simpler model are both more interpretable and extensible. For applications such as ultrasonic nondestructive evaluation, this interpretability is essential for trustworthiness of the methods and characterization of the material system under test. In this work, we show an interpretable, physics-informed representation learning framework that can analyze data across multiple dimensions (e.g., two dimensions of space and one dimension of time). The algorithm comes with convergence guarantees. In addition, our algorithm provides interpretability of the learned model as the parameters correspond to the individual solutions extracted from data. We demonstrate how this algorithm functions with wavefield videos. 
    more » « less
  4. Solving a bilevel optimization problem is at the core of several machine learning problems such as hyperparameter tuning, data denoising, meta- and few-shot learning, and training-data poisoning. Different from simultaneous or multi-objective optimization, the steepest descent direction for minimizing the upper-level cost in a bilevel problem requires the inverse of the Hessian of the lower-level cost. In this work, we propose a novel algorithm for solving bilevel optimization problems based on the classical penalty function approach. Our method avoids computing the Hessian inverse and can handle constrained bilevel problems easily. We prove the convergence of the method under mild conditions and show that the exact hypergradient is obtained asymptotically. Our method's simplicity and small space and time complexities enable us to effectively solve large-scale bilevel problems involving deep neural networks. We present results on data denoising, few-shot learning, and training-data poisoning problems in a large-scale setting. Our results show that our approach outperforms or is comparable to previously proposed methods based on automatic differentiation and approximate inversion in terms of accuracy, run-time, and convergence speed. 
    more » « less
  5. Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. They are capable of solving standard inverse problems such as denoising and compressive sensing with excellent results by simply fitting a neural network model to measurements from a single image or signal without the need for any additional training data. For some applications, this critically requires additional regularization in the form of early stopping the optimization. For signal recovery from a few measurements, however, un-trained convolutional networks have an intriguing self-regularizing property: Even though the network can perfectly fit any image, the network recovers a natural image from few measurements when trained with gradient descent until convergence. In this paper, we provide numerical evidence for this property and study it theoretically. We show that---without any further regularization---an un-trained convolutional neural network can approximately reconstruct signals and images that are sufficiently structured, from a near minimal number of random measurements. 
    more » « less