More Like this

1. Approximating the Steady-State Periodic Solutions of Contractive Systems

   https://doi.org/10.1109/tac.2018.2838054  

   Coogan, Samuel ; Margaliot, Michael ( January 2019 , IEEE Transactions on Automatic Control)
2. Anderson acceleration for contractive and noncontractive operators

   https://doi.org/10.1093/imanum/draa095  

   Pollock, Sara ; Rebholz, Leo G ( January 2021 , IMA Journal of Numerical Analysis)

   Abstract A one-step analysis of Anderson acceleration with general algorithmic depths is presented. The resulting residual bounds within both contractive and noncontractive settings reveal the balance between the contributions from the higher and lower order terms, which are both dependent on the success of the optimization problem solved at each step of the algorithm. The new residual bounds show the additional terms introduced by the extrapolation produce terms that are of a higher order than was previously understood. In the contractive setting these bounds sharpen previous convergence and acceleration results. The bounds rely on sufficient linear independence of the differences between consecutive residuals, rather than assumptions on the boundedness of the optimization coefficients, allowing the introduction of a theoretically sound safeguarding strategy. Several numerical tests illustrate the analysis primarily in the noncontractive setting, and demonstrate the use of the method, the safeguarding strategy and theory-based guidance on dynamic selection of the algorithmic depth, on a p-Laplace equation, a nonlinear Helmholtz equation and the steady Navier–Stokes equations with high Reynolds number in three spatial dimensions. 

   more » « less
3. Anderson acceleration based on the H−s Sobolev norm for contractive and noncontractive fixed-point operators

   https://doi.org/10.1016/j.cam.2021.113844  

   Yang, Yunan ; Townsend, Alex ; Appelö, Daniel ( March 2022 , Journal of Computational and Applied Mathematics)
4. Robust training and verification of implicit neural networks: a non-Euclidean contractive approach

   Jafarpour, S ; Davydov, A ; Abate, M ; Bullo, F ; Coogan, S ( July 2022 , ICML workshop on Formal Verification of Machine Learning (WFVML 2022))
5. Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach

   Jafarpour, S ; Abate, M. ; Davydov A. ; Bullo, F. ; Coogan S. ( May 2022 , Proceedings of The 4th Annual Learning for Dynamics and Control Conference, PMLR)

   Implicit neural networks are a general class of learning models that replace the layers in traditional feedforward models with implicit algebraic equations. Compared to traditional learning models, implicit networks offer competitive performance and reduced memory consumption. However, they can remain brittle with respect to input adversarial perturbations. This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks; our framework blends together mixed monotone systems theory and contraction theory. First, given an implicit neural network, we introduce a related embedded network and show that, given an infinity-norm box constraint on the input, the embedded network provides an infinity-norm box overapproximation for the output of the original network. Second, using infinity-matrix measures, we propose sufficient conditions for well-posedness of both the original and embedded system and design an iterative algorithm to compute the infinity-norm box robustness margins for reachability and classification problems. Third, of independent value, we show that employing a suitable relative classifier variable in our analysis will lead to tighter bounds on the certified adversarial robustness in classification problems. Finally, we perform numerical simulations on a Non-Euclidean Monotone Operator Network (NEMON) trained on the MNIST dataset. In these simulations, we compare the accuracy and run time of our mixed monotone contractive approach with the existing robustness verification approaches in the literature for estimating the certified adversarial robustness. 

   more » « less