More Like this

1. Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEs

   https://doi.org/10.24963/ijcai.2022/405  

   Djeumou, Franck; Neary, Cyrus; Goubault, Eric; Putot, Sylvie; Topcu, Ufuk (July 2022, Thirty-First International Joint Conference on Artificial Intelligence)

   Neural ordinary differential equations (NODEs) -- parametrizations of differential equations using neural networks -- have shown tremendous promise in learning models of unknown continuous-time dynamical systems from data. However, every forward evaluation of a NODE requires numerical integration of the neural network used to capture the system dynamics, making their training prohibitively expensive. Existing works rely on off-the-shelf adaptive step-size numerical integration schemes, which often require an excessive number of evaluations of the underlying dynamics network to obtain sufficient accuracy for training. By contrast, we accelerate the evaluation and the training of NODEs by proposing a data-driven approach to their numerical integration. The proposed Taylor-Lagrange NODEs (TL-NODEs) use a fixed-order Taylor expansion for numerical integration, while also learning to estimate the expansion's approximation error. As a result, the proposed approach achieves the same accuracy as adaptive step-size schemes while employing only low-order Taylor expansions, thus greatly reducing the computational cost necessary to integrate the NODE. A suite of numerical experiments, including modeling dynamical systems, image classification, and density estimation, demonstrate that TL-NODEs can be trained more than an order of magnitude faster than state-of-the-art approaches, without any loss in performance. 

   more » « less
2. Inducing persistence of excitation through sensor motion in the adaptive estimation of spatial fields

   https://doi.org/10.23919/ACC53348.2022.9867448  

   Demetriou, Michael A. (June 2022, 2022 American Control Conference)

   An alternative to regression-based estimation of spatial fields is adaptive-based estimation. Harnessing a widely used assumption on the series expansion of an unknown spatial field, the on-line estimation of the spatial field enables the integration of the real-time estimation of the field with any other tasks required of sensing agents. Parameter convergence in the adaptive estimation case requires the property of persistence of excitation. This condition reduces to imposing the integral of the outer product of a regressor vector be uniformly positive definite. With a single sensor measurement this is impossible to achieve unless the measurements are mobile. In this work, it is shown that in the adaptive estimation of a spatial field, a single mobile sensor is capable of inducing persistence of excitation and hence provide the sought after parameter convergence. Thus, the motion of a single sensor is a necessary condition for parameter convergence. It is shown with the appropriate control design for the platform carrying onboard the sensor, it also is a sufficient condition for persistence of excitation. Numerical results examining the time-variation of the regressor vector to induce a persistence of excitation along with user-defined guidance for the adaptive estimation of spatial fields are included to demonstrate the effects of mobile sensors in inducing persistence of excitation. 

   more » « less
3. AC-BioSD: A Biomolecular Signal Differentiator Module With Enhanced Performance

   https://doi.org/10.1109/LCSYS.2024.3394810  

   Alexis, Emmanouil; Avalos, José L; Cardelli, Luca; Papachristodoulou, Antonis (January 2024, IEEE Control Systems Letters)

   Temporal gradient estimation is a pervasive phenomenon in natural biological systems and holds great promise for synthetic counterparts with broad-reaching applications. Here, we advance the concept of BioSD (Biomolecular Signal Differentiators) by introducing a novel biomolecular topology, termed Autocatalytic-BioSD or AC-BioSD. Its structure allows for insensitivity to input signal changes and high precision in terms of signal differentiation, even when operating far from nominal conditions. Concurrently, disruptive high-frequency signal components are effectively attenuated. In addition, the usefulness of our topology in biological regulation is highlighted via a PID (Proportional-Integral-Derivative) bio-control scheme with set point weighting and filtered derivative action in both the deterministic and stochastic domains. 

   more » « less
4. Adaptive RKHS-based functional estimation of structurally perturbed second order infinite dimensional systems

   https://doi.org/10.1109/CDC51059.2022.9992577  

   Demetriou, Michael A. (December 2022, 2022 IEEE 61st Conference on Decision and Control (CDC))

   . (Ed.)

   This paper proposes a new approach for the adaptive functional estimation of second order infinite dimensional systems with structured perturbations. First, the proposed observer is formulated in the natural second order setting thus ensuring the time derivative of the estimated position is the estimated velocity, and therefore called natural adaptive observer. Assuming that the system does not yield a positive real system when placed in first order form, then the next step in deriving parameter adaptive laws is to assume a form of input-output collocation. Finally, to estimate structured perturbations taking the form of functions of the position and/or velocity outputs, the parameter space is not identified by a finite dimensional Euclidean space but instead is considered in a Reproducing Kernel Hilbert Space. Such a setting allows one not to be restricted by a priori assumptions on the dimension of the parameter spaces. Convergence of the position and velocity errors in their respective norms is established via the use of a parameter-dependent Lyapunov function, specifically formulated for second order infinite dimensional systems that include appropriately defined norms of the functional errors in the reproducing kernel Hilbert spaces. Boundedness of the functional estimates immediately follow and via an appropriate definition of a persistence of excitation condition for functional estimation, a functional convergence follows. When the system is governed by vector second order dynamics, all abstract spaces for the state evolution collapse to a Euclidean space and the natural adaptive observer results simplify. Numerical results of a second order PDE and a multi-degree of freedom finite dimensional mechanical system are presented. 

   more » « less
5. Projected Nesterov's proximal-gradient signal recovery from compressive poisson measurements

   https://doi.org/10.1109/acssc.2015.7421393  

   Gu, Renliang; Dogandzic, Aleksandar (November 2015, Proc. 49th Asilomar Conference on Signals, Systems and Computers)

   We develop a projected Nesterov’s proximal-gradient (PNPG) scheme for reconstructing sparse signals from compressive Poisson-distributed measurements with the mean signal intensity that follows an affine model with known intercept. The objective function to be minimized is a sum of convex data fidelity (negative log-likelihood (NLL)) and regularization terms. We apply sparse signal regularization where the signal belongs to a nonempty closed convex set within the domain of the NLL and signal sparsity is imposed using total-variation (TV) penalty. We present analytical upper bounds on the regularization tuning constant. The proposed PNPG method employs projected Nesterov’s acceleration step, function restart, and an adaptive step-size selection scheme that aims at obtaining a good local majorizing function of the NLL and reducing the time spent backtracking. We establish O(k⁻²) convergence of the PNPG method with step-size backtracking only and no restart. Numerical examples demonstrate the performance of the PNPG method. 

   more » « less