skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Approximate Controllability of Continuity Equation of Transformers
Building on the recent work by Geshkovski et al. (2023) which provides an interacting particle system interpretation of Transformers with a continuous-time evolution, we study the controllability attributes of the corresponding continuity equation across the landscape of probability space curves. In particular, we consider the parameters of the Transformer’s continuous-time evolution as control inputs. We prove that given an absolutely continuous probability measure and a non-local Lipschitz velocity field that satisfy a continuity equation, there exist control inputs such that the measure and the non-local velocity field of the Transformer’s continuous-time evolution approximate them, respectively, in the p-Wasserstein and Lp-sense, where 1 ≤ p < ∞.  more » « less
Award ID(s):
2211146
PAR ID:
10598493
Author(s) / Creator(s):
;
Corporate Creator(s):
Editor(s):
NA
Publisher / Repository:
IEEE
Date Published:
Journal Name:
IEEE Control Systems Letters
Volume:
8
ISSN:
2475-1456
Page Range / eLocation ID:
964 to 969
Subject(s) / Keyword(s):
Distributed parameter systems, machine learning, neural networks.
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, Transactions of the American Mathematical Society)
    In this paper we construct short time classical solutions to a class of master equations in the presence of non-degenerate individual noise arising in the theory of mean field games. The considered Hamiltonians are non-separable andlocalfunctions of the measure variable, therefore the equation is restricted to absolutely continuous measures whose densities lie in suitable Sobolev spaces. Our results hold for smooth enough Hamiltonians, without any additional structural conditions as convexity or monotonicity. 
    more » « less
  2. ; (, Mathematische Nachrichten)
    Abstract This paper aims at proving the local boundedness and continuity of solutions of the heat equation in the context of Dirichlet spaces under some rather weak additional assumptions. We consider symmetric local regular Dirichlet forms, which satisfy mild assumptions concerning (1) the existence of cut‐off functions, (2) a local ultracontractivity hypothesis, and (3) a weak off‐diagonal upper bound. In this setting, local weak solutions of the heat equation, and their time derivatives, are shown to be locally bounded; they are further locally continuous, if the semigroup admits a locally continuous density function. Applications of the results are provided including discussions on the existence of locally bounded heat kernel; structure results for ancient (local weak) solutions of the heat equation. 
    more » « less
  3. ; ; ; ; (, IEEE)
    Given starting and ending positions and velocities, L2 bounds on the acceleration and velocity, and the restriction to no more than two constant control inputs, this paper provides routines to compute the minimal-time path. Closed form solutions are provided for reaching a position in minimum time with and without a velocity bound, and for stopping at the goal position. A numeric solver is used to reach a goal position and velocity with no more than two constant control inputs. If a cruising phase at the terminal velocity is needed, this requires solving a non-linear equation with a single parameter. Code is provided on GitHub 1 , extended paper version at [1]. [1] https://github.com/RoboticSwarmControl/MinTimeL2pathsConstraints/ 
    more » « less
  4. ; ; (, 2022 American Control Conference (ACC))
    In this paper, we investigate how the self-synchronization property of a swarm of Kuramoto oscillators can be controlled and exploited to achieve target densities and target phase coherence. In the limit of an infinite number of oscillators, the collective dynamics of the agents’ density is described by a mean-field model in the form of a nonlocal PDE, where the nonlocality arises from the synchronization mechanism. In this mean-field setting, we introduce two space-time dependent control inputs to affect the density of the oscillators: an angular velocity field that corresponds to a state feedback law for individual agents, and a control parameter that modulates the strength of agent interactions over space and time, i.e., a multiplicative control with respect to the integral nonlocal term. We frame the density tracking problem as a PDE-constrained optimization problem. The controlled synchronization and phase-locking are measured with classical polar order metrics. After establishing the mass conservation property of the mean-field model and bounds on its nonlocal term, a system of first-order necessary conditions for optimality is recovered using a Lagrangian method. The optimality system, comprising a nonlocal PDE for the state dynamics equation, the respective nonlocal adjoint dynamics, and the Euler equation, is solved iteratively following a standard Optimize-then-Discretize approach and an efficient numerical solver based on spectral methods. We demonstrate our approach for each of the two control inputs in simulation. 
    more » « less
  5. ; (, Calculus of Variations and Partial Differential Equations)
    Abstract The goal of this paper is to prove a comparison principle for viscosity solutions of semilinear Hamilton–Jacobi equations in the space of probability measures. The method involves leveraging differentiability properties of the 2-Wasserstein distance in the doubling of variables argument, which is done by introducing a further entropy penalization that ensures that the relevant optima are achieved at positive, Lipschitz continuous densities with finite Fischer information. This allows to prove uniqueness and stability of viscosity solutions in the class of bounded Lipschitz continuous (with respect to the 1-Wasserstein distance) functions. The result does not appeal to a mean field control formulation of the equation, and, as such, applies to equations with nonconvex Hamiltonians and measure-dependent volatility. For convex Hamiltonians that derive from a potential, we prove that the value function associated with a suitable mean-field optimal control problem with nondegenerate idiosyncratic noise is indeed the unique viscosity solution. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email