More Like this

1. EikoNet: Solving the Eikonal Equation With Deep Neural Networks

   https://doi.org/10.1109/tgrs.2020.3039165  

   Smith, Jonathan D. ; Azizzadenesheli, Kamyar ; Ross, Zachary E. ( December 2020 , IEEE Transactions on Geoscience and Remote Sensing)

   null (Ed.)

   The recent deep learning revolution has created enormous opportunities for accelerating compute capabilities in the context of physics-based simulations. In this article, we propose EikoNet, a deep learning approach to solving the Eikonal equation, which characterizes the first-arrival-time field in heterogeneous 3-D velocity structures. Our grid-free approach allows for rapid determination of the travel time between any two points within a continuous 3-D domain. These travel time solutions are allowed to violate the differential equation--which casts the problem as one of optimization--with the goal of finding network parameters that minimize the degree to which the equation is violated. In doing so, the method exploits the differentiability of neural networks to calculate the spatial gradients analytically, meaning that the network can be trained on its own without ever needing solutions from a finite-difference algorithm. EikoNet is rigorously tested on several velocity models and sampling methods to demonstrate robustness and versatility. Training and inference are highly parallelized, making the approach well-suited for GPUs. EikoNet has low memory overhead and further avoids the need for travel-time lookup tables. The developed approach has important applications to earthquake hypocenter inversion, ray multipathing, and tomographic modeling, as well as to other fields beyond seismology where ray tracing is essential. 

   more » « less
2. ∇-Prox: Differentiable Proximal Algorithm Modeling for Large-Scale Optimization

   https://doi.org/10.1145/3592144  

   Lai, Zeqiang ; Wei, Kaixuan ; Fu, Ying ; Härtel, Philipp ; Heide, Felix ( August 2023 , ACM Transactions on Graphics)

   Tasks across diverse application domains can be posed as large-scale optimization problems, these include graphics, vision, machine learning, imaging, health, scheduling, planning, and energy system forecasting. Independently of the application domain, proximal algorithms have emerged as a formal optimization method that successfully solves a wide array of existing problems, often exploiting problem-specific structures in the optimization. Although model-based formal optimization provides a principled approach to problem modeling with convergence guarantees, at first glance, this seems to be at odds with black-box deep learning methods. A recent line of work shows that, when combined with learning-based ingredients, model-based optimization methods are effective, interpretable, and allow for generalization to a wide spectrum of applications with little or no extra training data. However, experimenting with such hybrid approaches for different tasks by hand requires domain expertise in both proximal optimization and deep learning, which is often error-prone and time-consuming. Moreover, naively unrolling these iterative methods produces lengthy compute graphs, which when differentiated via autograd techniques results in exploding memory consumption, making batch-based training challenging. In this work, we introduce ∇-Prox, a domain-specific modeling language and compiler for large-scale optimization problems using differentiable proximal algorithms. ∇-Prox allows users to specify optimization objective functions of unknowns concisely at a high level, and intelligently compiles the problem into compute and memory-efficient differentiable solvers. One of the core features of ∇-Prox is its full differentiability, which supports hybrid model- and learning-based solvers integrating proximal optimization with neural network pipelines. Example applications of this methodology include learning-based priors and/or sample-dependent inner-loop optimization schedulers, learned with deep equilibrium learning or deep reinforcement learning. With a few lines of code, we show ∇-Prox can generate performant solvers for a range of image optimization problems, including end-to-end computational optics, image deraining, and compressive magnetic resonance imaging. We also demonstrate ∇-Prox can be used in a completely orthogonal application domain of energy system planning, an essential task in the energy crisis and the clean energy transition, where it outperforms state-of-the-art CVXPY and commercial Gurobi solvers. 

   more » « less
3. Automatic Differentiable Procedural Modeling

   https://doi.org/10.1111/cgf.14475  

   Gaillard, Mathieu ; Krs, Vojtěch ; Gori, Giorgio ; Měch, Radomír ; Benes, Bedrich ( May 2022 , Computer Graphics Forum)

   Procedural modeling allows for an automatic generation of large amounts of similar assets, but there is limited control over the generated output. We address this problem by introducing Automatic Differentiable Procedural Modeling (ADPM). The forward procedural model generates a final editable model. The user modifies the output interactively, and the modifications are transferred back to the procedural model as its parameters by solving an inverse procedural modeling problem. We present an auto‐differentiable representation of the procedural model that significantly accelerates optimization. In ADPM the procedural model is always available, all changes are non‐destructive, and the user can interactively model the 3D object while keeping the procedural representation. ADPM provides the user with precise control over the resulting model comparable to non‐procedural interactive modeling. ADPM is node‐based, and it generates hierarchical 3D scene geometry converted to a differentiable computational graph. Our formulation focuses on the differentiability of high‐level primitives and bounding volumes of components of the procedural model rather than the detailed mesh geometry. Although this high‐level formulation limits the expressiveness of user edits, it allows for efficient derivative computation and enables interactivity. We designed a new optimizer to solve for inverse procedural modeling. It can detect that an edit is under‐determined and has degrees of freedom. Leveraging cheap derivative evaluation, it can explore the region of optimality of edits and suggest various configurations, all of which achieve the requested edit differently. We show our system's efficiency on several examples, and we validate it by a user study.

    

   more » « less
4. Optimal Control Strategies for Systems with Input Delay using the PIE Framework

   https://doi.org/10.1016/j.ifacol.2022.11.339  

   Peet, Matthew M. ( January 2022 , IFAC-PapersOnLine)

   The Partial Integral Equation (PIE) framework provides a unified algebraic representation for use in analysis, control, and estimation of infinite-dimensional systems. However, the presence of input delays results in a PIE representation with dependence on the derivative of the control input, u˙. This dependence complicates the problem of optimal state-feedback control for systems with input delay – resulting in a bilinear optimization problem. In this paper, we present two strategies for convexification of the H∞-optimal state-feedback control problem for systems with input delay. In the first strategy, we use a generalization of Young's inequality to formulate a convex optimization problem, albeit with some conservatism. In the second strategy, we filter the actuator signal – introducing additional dynamics, but resulting in a convex optimization problem without conservatism. We compare these two optimal control strategies on four example problems, solving the optimization problem using the latest release of the PIETOOLS software package for analysis, control and simulation of PIEs. 

   more » « less
5. Optimal Control of Velocity and Nonlocal Interactions in the Mean-Field Kuramoto Model

   https://doi.org/10.23919/ACC53348.2022.9867715  

   Sinigaglia, Carlo ; Braghin, Francesco ; Berman, Spring ( June 2022 , 2022 American Control Conference (ACC))

   In this paper, we investigate how the self-synchronization property of a swarm of Kuramoto oscillators can be controlled and exploited to achieve target densities and target phase coherence. In the limit of an infinite number of oscillators, the collective dynamics of the agents’ density is described by a mean-field model in the form of a nonlocal PDE, where the nonlocality arises from the synchronization mechanism. In this mean-field setting, we introduce two space-time dependent control inputs to affect the density of the oscillators: an angular velocity field that corresponds to a state feedback law for individual agents, and a control parameter that modulates the strength of agent interactions over space and time, i.e., a multiplicative control with respect to the integral nonlocal term. We frame the density tracking problem as a PDE-constrained optimization problem. The controlled synchronization and phase-locking are measured with classical polar order metrics. After establishing the mass conservation property of the mean-field model and bounds on its nonlocal term, a system of first-order necessary conditions for optimality is recovered using a Lagrangian method. The optimality system, comprising a nonlocal PDE for the state dynamics equation, the respective nonlocal adjoint dynamics, and the Euler equation, is solved iteratively following a standard Optimize-then-Discretize approach and an efficient numerical solver based on spectral methods. We demonstrate our approach for each of the two control inputs in simulation. 

   more » « less