We study the sample complexity of offline learning for a class of structured Markov decision processes (MDPs) describing the inventory control system with fixed ordering cost/setup cost, a fundamental problem in supply chains. We find that a naive plug-in sampling-based approach applied to the inventory MDPs leads to strictly lower sample complexity bounds compared to the optimal bounds recently obtained for the general MDPs. More specifically, in the infinite-horizon discounted cost setting, we obtain an sample complexity bound, where corresponds to the number of state-action pairs in a generic MDP with state space and action space . As such, improves on the optimal generic reinforcement learning (RL) bound (when directly applying here) by a factor of , and is able to completely remove the dependence on state and action cardinality. In the infinite-horizon average cost setting, we obtain an bound, improving on the generic optimal RL bound (when directly applying here) by a factor of , and hence removing the mixing time dependence. By carefully leveraging the structural properties of the inventory dynamics in various settings, we are able to improve on those “best-possible” bounds developed in the RL literature. Our results demonstrate the drawbacks one could face by blindly following RL algorithms and the necessity of designing sample efficient algorithms that properly incorporate the special structures of the inventory systems.
more »
« less
This content will become publicly available on January 27, 2027
Generalizations of the quadratic bound optimization principle
The quadratic bound (QB) principle proposed by Böhning and Lindsay in 1988 is an important special case of the majorization–minimization or minorization-maximization optimization principle. The quadratic upper-bound (QUB) principle is pertinent to minimization; the analogous quadratic lower-bound principle is pertinent to maximization. Unfortunately, in minimizing a loss , the QUB principle is limited by the difficulty of finding a constant positive definite matrix such that is positive semidefinite for all . This paper proposes a generalization of the QB principle that avoids this limitation. In particular, we construct QUB algorithms by replacing the matrix by a continuous matrix-valued function that dominates the Hessian and depends on the both the current iterate and the next potential iterate . In practice, we require to be diagonal with its diagonal entries separated in . In other words, the th diagonal entry of depends on only through its th entry . Theoretical analysis confirms that this class of generalized QB algorithms enjoys global convergence in favorable circumstances. For the scalar case, the tangency condition that the second derivative equals the bound function at the current point promotes superlinear convergence. Several numerical experiments implemented in Julia illustrate the power of the generalized QB principle.
more »
« less
- PAR ID:
- 10702848
- Publisher / Repository:
- National Academy of Sciences
- Date Published:
- Journal Name:
- Proceedings of the National Academy of Sciences
- Volume:
- 123
- Issue:
- 4
- ISSN:
- 0027-8424
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
LetNandpbe prime numbers with such that . In a previous paper, we showed that there is a cuspformfof weight 2 and level whoseℓ-th Fourier coefficient is congruent to modulo a prime abovepfor all primesℓ. In this paper, we prove that this formfis unique up to Galois conjugacy, and the extension of generated by the coefficients offis exactly . We also prove similar results when a higher power ofpdivides .more » « less
-
Abstract Many attempts to introduce fundamental nonlocality into quantum (or classical) field theory are based on the assumption that exponentials of the d’Alembertian are positive-definite, so that these operators can be employed without engendering the Ostrogradskian instability associated with higher derivative Lagrangians.This assumption is false.Working in the simple context of a 1-dimensional, point particle , I demonstrate that the equation has an infinite number of rapidly oscillating, exponentially rising and falling solutions. This infinite kernel is in one-to-one correspondence with the ability to specify ‘initial value data’arbitrarilyoveranyfinite interval .more » « less
-
Abstract We studyℓ∞norms ofℓ2-normalized eigenfunctions of quantum cat maps. For maps with short quantum periods (constructed by Bonechi and de Biévre in F Bonechi and S De Bièvre (2000,Communications in Mathematical Physics,211, 659–686)) we show that there exists a sequence of eigenfunctionsuwith . For general eigenfunctions we show the upper bound . Here the semiclassical parameter is . Our upper bound is analogous to the one proved by Bérard in P Bérard (1977,Mathematische Zeitschrift,155, 249-276) for compact Riemannian manifolds without conjugate points.more » « less
-
Abstract Polyatomic molecules have been identified as sensitive probes of charge-parity violating and parity violating physics beyond the Standard Model (BSM). For example, many linear triatomic molecules are both laser-coolable and have parity doublets in the ground electronic state arising from the bending vibration, both features that can greatly aid BSM searches. Understanding the state is a crucial prerequisite to precision measurements with linear polyatomic molecules. Here, we characterize the fundamental bending vibration of YbOH using high-resolution optical spectroscopy on the nominally forbidden transition at 588 nm. We assign 39 transitions originating from the lowest rotational levels of the state, and accurately model the state’s structure with an effective Hamiltonian using best-fit parameters. Additionally, we perform Stark and Zeeman spectroscopy on the state and fit the molecule-frame dipole moment to Dand the effective electrong-factor to . Further, we use an empirical model to explain observed anomalous line intensities in terms of interference from spin–orbit and vibronic perturbations in the excited state. Our work is an essential step toward searches for BSM physics in YbOH and other linear polyatomic molecules.more » « less
An official website of the United States government
