This content will become publicly available on April 1, 2027

Title: Don’t Follow Reinforcement Learning Blindly: Lower Sample Complexity of Learning Optimal Inventory Control Policies with Fixed Ordering Cost
We study the sample complexity of offline learning for a class of structured Markov decision processes (MDPs) describing the inventory control system with fixed ordering cost/setup cost, a fundamental problem in supply chains. We find that a naive plug-in sampling-based approach applied to the inventory MDPs leads to strictly lower sample complexity bounds compared to the optimal bounds recently obtained for the general MDPs. More specifically, in the infinite-horizon discounted cost setting, we obtain an O~ (min { (S¯−s_)2 (1−γ)2ϵ2 , 1(1−γ)4ϵ2 } ) sample complexity bound, where (S¯−s_) 2 corresponds to the number of state-action pairs in a generic MDP with state space S and action space A . As such, O~ ( (S¯−s_)2 (1−γ)2ϵ2 ) improves on the optimal generic reinforcement learning (RL) bound Θ~ ( (S¯−s_)2 (1−γ)3ϵ2 ) (when directly applying Θ~ ( |S||A| (1−γ)3ϵ2 ) here) by a factor of (1−γ)−1 , and O~ ( 1(1−γ)4ϵ2 ) is able to completely remove the dependence on state and action cardinality. In the infinite-horizon average cost setting, we obtain an O~ ( (S¯−s_)2 ϵ2 ) bound, improving on the generic optimal RL bound Θ~ ( (S¯−s_)2 tmix ϵ2 ) (when directly applying Θ~ ( |S||A| tmix ϵ2 ) here) by a factor of tmix , and hence removing the mixing time dependence. By carefully leveraging the structural properties of the inventory dynamics in various settings, we are able to improve on those “best-possible” bounds developed in the RL literature. Our results demonstrate the drawbacks one could face by blindly following RL algorithms and the necessity of designing sample efficient algorithms that properly incorporate the special structures of the inventory systems.  more » « less
Award ID(s):
2312205
PAR ID:
10708761
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
POMS
Date Published:
Journal Name:
Production and Operations Management
Volume:
35
Issue:
4
ISSN:
1059-1478
Page Range / eLocation ID:
1449 to 1468
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The quadratic bound (QB) principle proposed by Böhning and Lindsay in 1988 is an important special case of the majorization–minimization or minorization-maximization optimization principle. The quadratic upper-bound (QUB) principle is pertinent to minimization; the analogous quadratic lower-bound principle is pertinent to maximization. Unfortunately, in minimizing a loss f(θ) , the QUB principle is limited by the difficulty of finding a constant positive definite matrix B such that B−d2f (θ) is positive semidefinite for all θ . This paper proposes a generalization of the QB principle that avoids this limitation. In particular, we construct QUB algorithms by replacing the matrix B by a continuous matrix-valued function Bn (θ) that dominates the Hessian d2f (θ) and depends on the both the current iterate θn and the next potential iterate θ . In practice, we require Bn (θ) to be diagonal with its diagonal entries separated in θ . In other words, the i th diagonal entry of Bn (θ) depends on θ only through its i th entry θi . Theoretical analysis confirms that this class of generalized QB algorithms enjoys global convergence in favorable circumstances. For the scalar case, the tangency condition that the second derivative equals the bound function at the current point promotes superlinear convergence. Several numerical experiments implemented in Julia illustrate the power of the generalized QB principle. 
    more » « less
  2. The ratio of branching fractions R(D*)=B(B¯→D*τ− ν¯τ )/B(B¯→D*ℓ− ν¯ℓ ) , where ℓ is an electron or muon, is measured using a Belle II data sample with an integrated luminosity of 189  fb−1 at the SuperKEKB asymmetric-energy e+e− collider. Data is collected at the ϒ(4S) resonance, and one B meson in the ϒ(4S)→BB¯ decay is fully reconstructed in hadronic decay modes. The accompanying signal B meson is reconstructed as B¯→D*τ− ν¯τ using leptonic τ decays. The normalization decay, B¯→D*ℓ− ν¯ℓ , produces the same observable final-state particles. The ratio of branching fractions is extracted in a simultaneous fit to two signal-discriminating variables in both channels and yields R(D*)= 0.262−0.039+0.041 (stat )−0.032+0.035 (syst) . This result is consistent with the current world average and with Standard Model predictions. Published by the American Physical Society2024 
    more » « less
  3. Abstract Polyatomic molecules have been identified as sensitive probes of charge-parity violating and parity violating physics beyond the Standard Model (BSM). For example, many linear triatomic molecules are both laser-coolable and have parity doublets in the ground electronic X˜2Σ+(010) state arising from the bending vibration, both features that can greatly aid BSM searches. Understanding the X˜2Σ+(010) state is a crucial prerequisite to precision measurements with linear polyatomic molecules. Here, we characterize the fundamental bending vibration of 174 YbOH using high-resolution optical spectroscopy on the nominally forbidden X˜2Σ+(010) →A˜2 Π 1/2 (000) transition at 588 nm. We assign 39 transitions originating from the lowest rotational levels of the X˜2Σ+(010) state, and accurately model the state’s structure with an effective Hamiltonian using best-fit parameters. Additionally, we perform Stark and Zeeman spectroscopy on the X˜2Σ+(010) state and fit the molecule-frame dipole moment to Dmol =2.16(1) Dand the effective electrong-factor to gS=2.07(2) . Further, we use an empirical model to explain observed anomalous line intensities in terms of interference from spin–orbit and vibronic perturbations in the excited A˜2 Π 1/2 (000) state. Our work is an essential step toward searches for BSM physics in YbOH and other linear polyatomic molecules. 
    more » « less
  4. The first four-dimensional amplitude analysis of the B+→ψ(2S)KS0π+ decay is performed with proton-proton collision data collected by the LHCb experiment at s=13  TeV , corresponding to an integrated luminosity of 5.4  fb−1 . The data cannot be fully explained by B+→ψ(2S)K*+ contributions alone. A significantly better description of the data is obtained by adding a T cc¯ + contribution decaying to ψ(2S)π+ . The properties of the T cc¯ + structure are consistent with the exotic state T cc¯1 (4430)+ reported in the isospin-related B¯0→ψ(2S)K−π+ decay. Effects of a possible T cc¯1 (4430)+→ D¯1* (2600)0D+ decay mode on the T cc¯1 (4430)+→ψ(2S)π+ mass distribution are investigated through a Flatté parametrization, providing constraints on the relative decay strength. A description of the T cc¯1 (4430)+ structure using the triangle singularity mechanism is studied and also found to be consistent with the data. 
    more » « less
  5. A search for the rare decay B+→ Λ¯ pμ+μ− is performed using proton-proton collision data recorded by the LHCb experiment at a center-of-mass energy of s =13  TeV , corresponding to an integrated luminosity of 5.4  fb−1 . An excess of events is found with respect to the background-only expectation, with a signal significance of 3.5 standard deviations, in the low invariant-mass region of m( Λ¯ p)<2.8  GeV/c2 . The branching fraction is measured to be Blow (B+→ Λ¯ pμ+μ− )= (1.7 0−0.56+0.65 (stat)±0.17(syst)±0.14(ext)) ×10−8 , where the last uncertainty is due to external inputs on B(B+→ J/ψ Λ¯ p )×B(J/ψ→μ+μ−) . With no significant signal observed in the high m(Λ¯p) region above 2.8  GeV/c2 , an upper limit is set to be Bhigh (B+→ Λ¯ pμ+μ− )<2.8(3.7)×10−9 at the 90% (95%) confidence level. 
    more » « less