Search for: All records

Abstract This paper proposes two sequential metamodel‐based methods for level‐set estimation (LSE) that leverage the uniform bound built on stochastic kriging: predictive variance reduction (PVR) and expected classification improvement (ECI). We show that PVR and ECI possess desirable theoretical performance guarantees and provide closed‐form expressions for their respective sequential sampling criteria to seek the next design point for performing simulation runs, allowing computationally efficient one‐iteration look‐ahead updates. To enhance understanding, we reveal the connection between PVR and ECI's sequential sampling criteria. Additionally, we propose integrating a budget allocation feature with PVR and ECI, which improves computational efficiency and potentially enhances robustness to the impacts of heteroscedasticity. Numerical studies demonstrate the superior performance of the proposed methods compared to state‐of‐the‐art benchmarking approaches when given a fixed simulation budget, highlighting their effectiveness in addressing LSE problems. 

Abstract In this paper we provide a thorough investigation of the cluster sampling scheme for Morris' elementary effects method (MM), a popular model‐free factor screening method originated in the setting of design and analysis of computational experiments. We first study the sampling mechanism underpinning the two sampling schemes of MM (i.e., cluster sampling and noncluster sampling) and unveil its nature as a two‐level nested sampling process. This in‐depth understanding sets up a foundation for tackling two important aspects of cluster sampling: budget allocation and sampling plan. On the one hand, we study the budget allocation problem for cluster sampling under the analysis of variance framework and derive optimal budget allocations for efficient estimation of the importance measures. On the other hand, we devise an efficient cluster sampling algorithm with two variants to achieve enhanced statistical properties. The numerical evaluations demonstrate the superiority of the proposed cluster sampling algorithm and the budget allocations derived (when used both separately and in conjunction) to existing cluster and noncluster sampling schemes. 

In this work, we propose a method to construct a uniform error bound for the SK predictor. In investigating the asymptotic properties of the proposed uniform error bound, we examine the convergence rate of SK’s predictive variance under the supremum norm in both fixed and random design settings. Our analyses reveal that the large-sample properties of SK prediction depend on the design-point sampling scheme and the budget allocation scheme adopted. Appropriately controlling the order of noise variances through budget allocation is crucial for achieving a desirable convergence rate of SK’s approximation error, as quantified by the uniform error bound, and for maintaining SK’s numerical stability. Moreover, we investigate the impact of noise variance estimation on the uniform error bound’s performance theoretically and numerically. We demonstrate the superiority of the proposed uniform bound to the Bonferroni correction-based simultaneous confidence interval under various experimental settings through numerical evaluations. 

This paper proposes two fully sequential procedures for selecting the best system with a guaranteed probability of correct selection (PCS). The main features of the proposed procedures include the following: (1) adopting a Bonferroni-free model that overcomes the conservativeness of the Bonferroni correction and delivers the exact probabilistic guarantee without overshooting; (2) conducting always valid and fully sequential hypothesis tests that enable continuous monitoring of each candidate system and control the type I error rate (or equivalently, PCS) at a prescribed level; and (3) assuming an indifference-zone-flexible formulation, which means that the indifference-zone parameter is not indispensable but could be helpful if provided. We establish statistical validity and asymptotic efficiency for the proposed procedures under normality settings with and without the knowledge of true variances. Numerical studies conducted under various configurations corroborate the theoretical findings and demonstrate the superiority of the proposed procedures. Funding: W. Wang and H. Wan were supported in part by CollinStar Capital Pty Ltd. X. Chen was supported in part by the National Science Foundation [Grant IIS-1849300 and CAREER CMMI-1846663]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/opre.2023.2447 .