skip to main content


Title: Conversion of a Class of Stochastic Control Problems to Fundamental-Solution Deterministic Control Problems
A new optimal control based representation for stationary action trajectories is constructed by exploiting connections between semiconvexity, semiconcavity, and stationarity. This new representation is used to verify a known two-point boundary value problem characterization of stationary action.  more » « less
Award ID(s):
1908918
NSF-PAR ID:
10170563
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the American Control Conference
ISSN:
0743-1619
Page Range / eLocation ID:
1779-1784
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    A new optimal control based representation for stationary action trajectories is constructed by exploiting connections between semiconvexity, semiconcavity, and stationarity. This new representation is used to verify a known two-point boundary value problem characterization of stationary action. 
    more » « less
  2. Abstract

    Bikebot (i.e., bicycle-based robot) is a class of underactuated balance robotic systems that require simultaneous trajectory tracking and balance control tasks. We present a tracking and balance control design of an autonomous bikebot. The external-internal convertible structure of the bikebot dynamics is used to design a causal feedback control to achieve both the tracking and balance tasks. A balance equilibrium manifold is used to define and capture the platform balance profiles and coupled interaction with the trajectory tracking performance. To achieve fully autonomous navigation, a gyrobalancer actuation is integrated with the steering and velocity control for stationary platform balance and stationary-moving switching. Stability and convergence analyses are presented to guarantee the control performance. Extensive experiments are presented to validate and demonstrate the autonomous control design. We also compare the autonomous control performance with human riding experiments and similar action strategies are found between them.

     
    more » « less
  3. We consider the linear third order (in time) PDE known as the SMGTJ-equation, defined on a bounded domain, under the action of either Dirichlet or Neumann boundary control \begin{document}$ g $\end{document}. Optimal interior and boundary regularity results were given in [1], after [41], when \begin{document}$ g \in L^2(0, T;L^2(\Gamma)) \equiv L^2(\Sigma) $\end{document}, which, moreover, in the canonical case \begin{document}$ \gamma = 0 $\end{document}, were expressed by the well-known explicit representation formulae of the wave equation in terms of cosine/sine operators [19], [17], [24,Vol Ⅱ]. The interior or boundary regularity theory is however the same, whether \begin{document}$ \gamma = 0 $\end{document} or \begin{document}$ 0 \neq \gamma \in L^{\infty}(\Omega) $\end{document}, since \begin{document}$ \gamma \neq 0 $\end{document} is responsible only for lower order terms. Here we exploit such cosine operator based-explicit representation formulae to provide optimal interior and boundary regularity results with \begin{document}$ g $\end{document} "smoother" than \begin{document}$ L^2(\Sigma) $\end{document}, qualitatively by one unit, two units, etc. in the Dirichlet boundary case. To this end, we invoke the corresponding results for wave equations, as in [17]. Similarly for the Neumann boundary case, by invoking the corresponding results for the wave equation as in [22], [23], [37] for control smoother than \begin{document}$ L^2(0, T;L^2(\Gamma)) $\end{document}, and [44] for control less regular in space than \begin{document}$ L^2(\Gamma) $\end{document}. In addition, we provide optimal interior and boundary regularity results when the SMGTJ equation is subject to interior point control, by invoking the corresponding wave equations results [42], [24,Section 9.8.2].

     
    more » « less
  4. Krause, Andreas and (Ed.)
    General function approximation is a powerful tool to handle large state and action spaces in a broad range of reinforcement learning (RL) scenarios. However, theoretical understanding of non-stationary MDPs with general function approximation is still limited. In this paper, we make the first such an attempt. We first propose a new complexity metric called dynamic Bellman Eluder (DBE) dimension for non-stationary MDPs, which subsumes majority of existing tractable RL problems in static MDPs as well as non-stationary MDPs. Based on the proposed complexity metric, we propose a novel confidence-set based model-free algorithm called SW-OPEA, which features a sliding window mechanism and a new confidence set design for non-stationary MDPs. We then establish an upper bound on the dynamic regret for the proposed algorithm, and show that SW-OPEA is provably efficient as long as the variation budget is not significantly large. We further demonstrate via examples of non-stationary linear and tabular MDPs that our algorithm performs better in small variation budget scenario than the existing UCB-type algorithms. To the best of our knowledge, this is the first dynamic regret analysis in non-stationary MDPs with general function approximation. 
    more » « less
  5. Abstract

    The Community Earth System Model 2 (CESM2) is the latest Earth System Model developed by the National Center for Atmospheric Research in collaboration with the university community and is significantly advanced in most components compared to its predecessor (CESM1). Here, CESM2's representation of the large‐scale atmospheric circulation and its variability is assessed. Further context is providedthrough comparison to the CESM1 large ensemble and other models from the Coupled Model Intercomparison Project (CMIP5 and CMIP6). This includes an assessment of the representation of jet streams and storm tracks, stationary waves, the global divergent circulation, the annular modes, the North Atlantic Oscillation, and blocking. Compared to CESM1, CESM2 is substantially improved in the representation of the storm tracks, Northern Hemisphere (NH) stationary waves, NH winter blocking and the global divergent circulation. It ranks within the top 10% of CMIP class models in many of these features. Some features of the Southern Hemisphere (SH) circulation have degraded, such as the SH jet strength, stationary waves, and blocking, although the SH jet stream is placed at approximately the correct location. This analysis also highlights systematic deficiencies in these features across the new CMIP6 archive, such as the continued tendency for the SH jet stream to be placed too far equatorward, the North Atlantic westerlies to be too strong over Europe, the storm tracks as measured by low‐level meridional wind variance to be too weak and a lack of blocking in the North Atlantic sector.

     
    more » « less