Small‐to‐medium‐sized enterprises (SMEs), including many startup firms, need to manage interrelated flows of cash and inventories of goods. In this study, we model a firm that can finance its inventory (ordered or manufactured) with loans in order to meet random demand which in general may not be time stationary. The firm earns interest on its cash on hand and pays interest on its debt. The objective is to maximize the expected value of the firm's capital at the end of a finite planning horizon. The firm's state at the beginning of each period is characterized by the inventory level and the capital level measured in units of the product, whose sum represents the “net worth” of the firm. Our study shows that the optimal ordering policy is characterized by a pair of threshold parameters as follows. (i) If the net worth is less than the lower threshold, then the firm employs a base stock order up to the lower threshold. (ii) If the net worth is between the two thresholds, then the firm orders exactly as many units as it can afford, without borrowing. (iii) If the net worth is above the upper threshold, then the firm employs a base stock order up to the upper threshold. Further, upper and lower bounds for the threshold values are developed using two simple‐to‐compute myopic ordering policies which yield lower bounds for the value function. We also derive an upper bound for the value function by considering a sell‐back policy. Subsequently, it is shown that policies of similar structure are optimal when the loan and deposit interest rates are piecewise linear functions, when there is a maximal loan limit and when unsatisfied demand is backordered. Finally, further managerial insights are provided with extensive numerical studies.
Looking Upstream: Optimal Policies for a Class of Capacitated Multi-Stage Inventory Systems
We consider a multi-stage inventory system with stochastic demand and processing capacity constraints at each stage, for both finite-horizon and infinite-horizon, discounted-cost settings. For a class of such systems characterized by having the smallest capacity at the most downstream stage and system utilization above a certain threshold, we identify the structure of the optimal policy, which represents a novel variation of the order-up-to policy. We find the explicit functional form of the optimal order-up-to levels, and show that they depend (only) on upstream echelon inventories. We establish that, above the threshold utilization, this optimal policy achieves the decomposition of the multidimensional objective cost function for the system into a sum of single-dimensional convex functions. This decomposition eliminates the curse of dimensionality and allows us to numerically solve the problem. We provide a fast algorithm to determine a (tight) upper bound on this threshold utilization for capacity-constrained inventory problems with an arbitrary number of stages. We make use of this algorithm to quantify upper bounds on the threshold utilization for three-, four-, and five-stage capacitated systems over a range of model parameters, and discuss insights that emerge.
more »
« less
- Award ID(s):
- 1644935
- NSF-PAR ID:
- 10041616
- Date Published:
- Journal Name:
- Production and operations management
- ISSN:
- 1059-1478
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
Abstract Achieving sample efficiency in online episodic reinforcement learning (RL) requires optimally balancing exploration and exploitation. When it comes to a finite-horizon episodic Markov decision process with $S$ states, $A$ actions and horizon length $H$, substantial progress has been achieved toward characterizing the minimax-optimal regret, which scales on the order of $\sqrt{H^2SAT}$ (modulo log factors) with $T$ the total number of samples. While several competing solution paradigms have been proposed to minimize regret, they are either memory-inefficient, or fall short of optimality unless the sample size exceeds an enormous threshold (e.g. $S^6A^4 \,\mathrm{poly}(H)$ for existing model-free methods). To overcome such a large sample size barrier to efficient RL, we design a novel model-free algorithm, with space complexity $O(SAH)$, that achieves near-optimal regret as soon as the sample size exceeds the order of $SA\,\mathrm{poly}(H)$. In terms of this sample size requirement (also referred to the initial burn-in cost), our method improves—by at least a factor of $S^5A^3$—upon any prior memory-efficient algorithm that is asymptotically regret-optimal. Leveraging the recently introduced variance reduction strategy (also called reference-advantage decomposition), the proposed algorithm employs an early-settled reference update rule, with the aid of two Q-learning sequences with upper and lower confidence bounds. The design principle of our early-settled variance reduction method might be of independent interest to other RL settings that involve intricate exploration–exploitation trade-offs.more » « less
-
null (Ed.)Although urban transit systems (UTS) often have fixed vehicle capacity and relatively constant departure headways, they may need to accommodate dramatically fluctuating passenger demands over space and time, resulting in either excessive passenger waiting or vehicle capacity and energy waste. Therefore, on the one hand, optimal operations of UTS rely on accurate modeling of passenger queuing dynamics, which is particularly complex on a multistop transit corridor. On the other hand, capacities of transit vehicles can be made variable and adaptive to time-variant passenger demand so as to minimize energy waste, especially with the emergence of modular vehicle technologies. This paper investigates operations of a multistop transit corridor in which vehicles may have different capacities across dispatches. We specify skewed time coordinates to simplify the problem structure while incorporating traffic congestion. Then, we propose a mixed integer linear programming model that determines the optimal dynamic headways and vehicle capacities over the study time horizon to minimize the overall system cost for the transit corridor. In particular, the proposed model considers a realistic multistop first-in, first-out (MSFIFO) rule that gives the same boarding priority to passengers arriving at a station in the same time interval yet with different destinations. A customized dynamic programming (DP) algorithm is proposed to solve this model efficiently. To circumvent the rapid increase of the state space of a typical DP algorithm, we analyze the theoretical properties of the investigated problem and identify upper and lower bounds to a feasible solution. The bounds largely reduce the state space during the DP iterations and greatly improve the efficiency of the proposed DP algorithm. The state dimensions are also reduced with the MSFIFO rule such that all queues with different destinations at the same origin can be tracked with a single boarding position state variable at each stage. A hypothetical example and a real-world case study show that the proposed DP algorithm greatly outperforms a state-of-the-art commercial solver (Gurobi) in both solution quality and time.more » « less
-
more » « less
Abstract We present a stochastic optimization model for allocating and sharing a critical resource in the case of a pandemic. The demand for different entities peaks at different times, and an initial inventory for a central agency are to be allocated. The entities (states) may share the critical resource with a different state under a risk‐averse condition. The model is applied to study the allocation of ventilator inventory in the COVID‐19 pandemic by FEMA to different U.S. states. Findings suggest that if less than 60% of the ventilator inventory is available for non‐COVID‐19 patients, FEMA's stockpile of 20 000 ventilators (as of March 23, 2020) would be nearly adequate to meet the projected needs in slightly above average demand scenarios. However, when more than 75% of the available ventilator inventory must be reserved for non‐COVID‐19 patients, various degrees of shortfall are expected. In a severe case, where the demand is concentrated in the top‐most quartile of the forecast confidence interval and states are not willing to share their stockpile of ventilators, the total shortfall over the planning horizon (until May 31, 2020) is about 232 000 ventilator days, with a peak shortfall of 17 200 ventilators on April 19, 2020. Results are also reported for a worst‐case where the demand is at the upper limit of the 95% confidence interval. An important finding of this study is that a central agency (FEMA) can act as a coordinator for sharing critical resources that are in short supply over time to add efficiency in the system. Moreover, through properly managing risk‐aversion of different entities (states) additional efficiency can be gained. An additional implication is that ramping up production early in the planning cycle allows to reduce shortfall significantly. An optimal timing of this production ramp‐up consideration can be based on a cost‐benefit analysis.
-
Reverse logistics has been gaining recognition in practice (and theory) for helping companies better match supply with demand, and thus reduce costs in their supply chains. In this paper, we study reverse logistics from the perspective of a supply chain in which each location can initiate multiple flows of product. Our first objective is to jointly optimize ordering decisions pertaining to regular, reverse and expedited flows of product in a stochastic, multi-stage inventory model of a logistics supply chain, in which the physical transformation of the product is completed at the most upstream location in the system. Due to those multiple flows of product, the feasible region for the problem acquires multi-dimensional boundaries that lead to the curse of dimensionality. To address this challenge, we develop a different solution method that allows us to reduce the dimensionality of the feasible region and, subsequently, identify the structure of the optimal policy. We refer to this policy as a nested echelon base-stock policy, as decisions for different product flows are sequentially nested within each other. We show that this policy renders the model analytically and numerically tractable. Our results provide actionable policies for firms to jointly manage three different product flows in their supply chains, and allow us arrive at insights regarding the main drivers of the value of reverse logistics. One of our key findings is that, when it comes to the value generated by reverse logistics, demand variability (i.e., demand uncertainty across periods) matters more than demand volatility (i.e., demand uncertainty within each period). This is because, in the absence of demand variability, it is effectively never optimal to return product upstream, regardless of the level of inherent demand volatility. Our second objective is to extend our analysis to product transforming-supply chains, in which product transformation is allowed to occur at each location. In such a system, it becomes necessary to keep track of both the location and stage of completion of each unit of inventory, so that the number of state and decisions variables increases with the square of the number of locations in the system. To analyze such a supply chain, we first identify a policy that provides a lower bound on the total cost. Then, we establish a special decomposition of the objective cost function that allows us to propose a novel heuristic policy. We find that the performance gap of our heuristic policy relative to the lower-bounding policy averages less than 5% across a range of parameters and supply chain lengths.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
