skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


This content will become publicly available on August 1, 2026

Title: End-to-End Learning Framework for Solving Non-Markovian Optimal Control
Integer-order calculus fails to capture the long-range dependence (LRD) and memory effects found in many complex systems. Fractional calculus addresses these gaps through fractional-order integrals and derivatives, but fractional-order dynamical systems pose substantial challenges in system identification and optimal control tasks. In this paper, we theoretically derive the optimal control via linear quadratic regulator (LQR) for fractional-order linear time-invariant (FOLTI) systems and develop an end-to-end deep learning framework based on this theoretical foundation. Our approach establishes a rigorous mathematical model, derives analytical solutions, and incorporates deep learning to achieve data-driven optimal control of FOLTI systems. Our key contributions include: (i) proposing a novel method for system identification and optimal control strategy in FOLTI systems, (ii) developing the first end-to-end data-driven learning framework, Fractional-Order Learning for Optimal Control (FOLOC), that learns control policies from observed trajectories, and (iii) deriving theoretical bounds on the sample complexity for learning accurate control policies under fractional-order dynamics. Experimental results indicate that our method accurately approximates fractional-order system behaviors without relying on Gaussian noise assumptions, pointing to promising avenues for advanced optimal control.  more » « less
Award ID(s):
1936775
PAR ID:
10626297
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
International Conference on Machine Learning (ICML)
Date Published:
Format(s):
Medium: X
Location:
Vancouver, Canada
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Abstract Recent advances in network science, control theory, and fractional calculus provide us with mathematical tools necessary for modeling and controlling complex dynamical networks (CDNs) that exhibit long-term memory. Selecting the minimum number of driven nodes such that the network is steered to a prescribed state is a key problem to guarantee that complex networks have a desirable behavior. Therefore, in this paper, we study the effects of long-term memory and of the topological properties on the minimum number of driven nodes and the required control energy. To this end, we introduce Gramian-based methods for optimal driven node selection for complex dynamical networks with long-term memory and by leveraging the structure of the cost function, we design a greedy algorithm to obtain near-optimal approximations in a computationally efficiently manner. We investigate how the memory and topological properties influence the control effort by considering Erdős–Rényi, Barabási–Albert and Watts–Strogatz networks whose temporal dynamics follow a fractional order state equation. We provide evidence that scale-free and small-world networks are easier to control in terms of both the number of required actuators and the average control energy. Additionally, we show how our method could be applied to control complex networks originating from the human brain and we discover that certain brain cortex regions have a stronger impact on the controllability of network than others. 
    more » « less
  2. Abstract We present a theoretical and computational framework based on fractional calculus for the analysis of the nonlocal static response of cylindrical shell panels. The differ-integral nature of fractional derivatives allows an efficient and accurate methodology to account for the effect of long-range (nonlocal) interactions in curved structures. More specifically, the use of frame-invariant fractional-order kinematic relations enables a physically, mathematically, and thermodynamically consistent formulation to model the nonlocal elastic interactions. To evaluate the response of these nonlocal shells under practical scenarios involving generalized loads and boundary conditions, the fractional-finite element method (f-FEM) is extended to incorporate shell elements based on the first-order shear-deformable displacement theory. Finally, numerical studies are performed exploring both the linear and the geometrically nonlinear static response of nonlocal cylindrical shell panels. This study is intended to provide a general foundation to investigate the nonlocal behavior of curved structures by means of fractional-order models. 
    more » « less
  3. The learning-to-rank problem aims at ranking items to maximize exposure of those most relevant to a user query. A desirable property of such ranking systems is to guarantee some notion of fairness among specified item groups. While fairness has recently been considered in the context of learning-to-rank systems, current methods cannot provide guarantees on the fairness of the predicted rankings. This paper addresses this gap and introduces Smart Predict and Optimize for Fair Ranking (SPOFR), an integrated optimization and learning framework for fairness-constrained learning to rank. The end-to-end SPOFR framework includes a constrained optimization sub-model and produces ranking policies that are guaranteed to satisfy fairness constraints, while allowing for fine control of the fairness-utility tradeoff. SPOFR is shown to significantly improve on current state-of-the-art fair learning-to-rank systems with respect to established performance metrics. 
    more » « less
  4. Existing approaches for autonomous control of pan-tilt-zoom (PTZ) cameras use multiple stages where object detection and localization are performed separately from the control of the PTZ mechanisms. These approaches require manual labels and suffer from performance bottlenecks due to error propagation across the multi-stage flow of information. The large size of object detection neural networks also makes prior solutions infeasible for real-time deployment in resource-constrained devices. We present an end-to-end deep reinforcement learning (RL) solution called Eagle1 to train a neural network policy that directly takes images as input to control the PTZ camera. Training reinforcement learning is cumbersome in the real world due to labeling effort, runtime environment stochasticity, and fragile experimental setups. We introduce a photo-realistic simulation framework for training and evaluation of PTZ camera control policies. Eagle achieves superior camera control performance by maintaining the object of interest close to the center of captured images at high resolution and has up to 17% more tracking duration than the state-of-the-art. Eagle policies are lightweight (90x fewer parameters than Yolo5s) and can run on embedded camera platforms such as Raspberry PI (33 FPS) and Jetson Nano (38 FPS), facilitating real-time PTZ tracking for resource-constrained environments. With domain randomization, Eagle policies trained in our simulator can be transferred directly to real-world scenarios2. 
    more » « less
  5. Data-driven approaches are increasingly popular for identifying dynamical systems due to improved accuracy and availability of sensor data. However, relying solely on data for identification does not guarantee that the identified systems will maintain their physical properties or that the predicted models will generalize well. In this paper, we propose a novel method for data-driven system identification by integrating a neural network as the first-order derivative of the learned dynamics in a Taylor series instead of learning the dynamical function directly. In addition, for dynamical systems with known monotonic properties, our approach can ensure monotonicity by constraining the neural network derivative to be non-positive or non-negative to the respective inputs, resulting in Monotonic Taylor Neural Networks (MTNN). Such constraints are enforced by either a specialized neural network architecture or regularization in the loss function for training. The proposed method demonstrates better performance compared to methods without the physics-based monotonicity constraints when tested on experimental data from an HVAC system and a temperature control testbed. Furthermore, MTNN shows good performance in a control application of a model predictive controller for a nonlinear MIMO system, illustrating the practical application of our method. 
    more » « less