Search for: All records

We study the construction of a confidence interval (CI) for a simulation output performance measure that accounts for input uncertainty when the input models are estimated from finite data. In particular, we focus on performance measures that can be expressed as a ratio of two dependent simulation outputs’ means. We adopt the parametric bootstrap method to mimic input data sampling and construct the percentile bootstrap CI after estimating the ratio at each bootstrap sample. The standard estimator, which takes the ratio of two sample means, tends to exhibit large finite-sample bias and variance, leading to overcoverage of the percentile bootstrap CI. To address this, we propose two new ratio estimators that replace the sample means with pooled mean estimators via the k-nearest neighbor (kNN) regression: the kNN estimator and the kLR estimator. The kNN estimator performs well in low dimensions, but its estimation error converges more slowly as the dimension increases. The kLR estimator combines the likelihood ratio (LR) method with the kNN regression, leveraging the strengths of both while mitigating their weaknesses; the LR method removes dependence of the error convergence rate on the dimension, whereas the kNN method controls the variance of the kLR estimator to be asymptotically bounded. From the asymptotic analyses and finite-sample heuristics, we propose an experiment design for the ratio estimators and demonstrate their superior empirical performances over the standard ratio estimator using three examples, including one in the enterprise risk management application. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: This work was supported by the National Science Foundation [Grants CAREER CMMI-2246281 and CMMI-2417616] and the Natural Sciences and Engineering Research Council of Canada [Grant RGPIN-2018-03755]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0914 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0914 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ . 

In practice, simulation parameters are estimated from finite data, which introduces model uncertainty. In “Selection of the Most Probable Best,” Taeho Kim, Kyoung-Kuk Kim, and Eunhye Song propose a new estimator of the optimum, the most probable best (MPB), for ranking and selection (R&S) when the model uncertainty is characterized by a posterior distribution on the parameter given data. Defined as the solution with the largest posterior probability of optimality, selecting the MPB requires simulating at all parameter-solution combinations. The authors focus on when the posterior has a finite support and derive the convergence rate of the probability of correctly selecting the MPB as a function of sampling fractions of all solution-parameter pairs. They propose a lower bound on the rate function that allows easier characterization of the optimal sampling fractions and devise a sequential sampling algorithm from it. The algorithm shows superior empirical performance over benchmarks.