More Like this

1. Human Inductive Biases in Design Decision Making

   https://doi.org/10.1115/DETC2020-22252  

   Shergadwala, Murtuza N.; Panchal, Jitesh H. (August 2020, ASME 2020 International Design Engineering Technical Conferences and Computers and Information Science in Engineering)

   null (Ed.)

   Abstract Designers make information acquisition decisions, such as where to search and when to stop the search. Such decisions are typically made sequentially, such that at every search step designers gain information by learning about the design space. However, when designers begin acquiring information, their decisions are primarily based on their prior knowledge. Prior knowledge influences the initial set of assumptions that designers use to learn about the design space. These assumptions are collectively termed as inductive biases. Identifying such biases can help us better understand how designers use their prior knowledge to solve problems in the light of uncertainty. Thus, in this study, we identify inductive biases in humans in sequential information acquisition tasks. To do so, we analyze experimental data from a set of behavioral experiments conducted in the past [1–5]. All of these experiments were designed to study various factors that influence sequential information acquisition behaviors. Across these studies, we identify similar decision making behaviors in the participants in their very first decision to “choose x”. We find that their choices of “x” are not uniformly distributed in the design space. Since such experiments are abstractions of real design scenarios, it implies that further contextualization of such experiments would only increase the influence of these biases. Thus, we highlight the need to study the influence of such biases to better understand designer behaviors. We conclude that in the context of Bayesian modeling of designers’ behaviors, utilizing the identified inductive biases would enable us to better model designer’s priors for design search contexts as compared to using non-informative priors. 

   more » « less
2. A Value‐Based Model Selection Approach for Environmental Random Variables

   https://doi.org/10.1029/2018WR023000  

   Müller, M. F.; Thompson, S. E. (January 2019, Water Resources Research)

   Abstract Environmental decisions with substantial social and environmental implications are regularly informed by model predictions, incurring inevitable uncertainty. The selection of a set of model predictions to inform a decision is usually based on model performance, measured by goodness‐of‐fit metrics. Yet goodness‐of‐fit metrics have a questionable relationship to a model's value to end users, particularly when validation data are themselves uncertain. For example, decisions based on flow frequency models are not necessarily improved by adopting models with the best overall goodness of fit. We propose an alternative model evaluation approach based on the conditional value of sample information, first defined in 1961, which has found extensive use in sampling design optimization but which has not previously been used for model evaluation. The metric uses observations from a validation set to estimate the expected monetary costs associated with model prediction uncertainties. A model is only considered superior to alternatives if (i) its predictions reduce these costs and (ii) sufficient validation data are available to distinguish its performance from alternative models. By describing prediction uncertainties in monetary terms, the metric facilitates the communication of prediction uncertainty by end users, supporting the inclusion of uncertainty analysis in decision making. 

   more » « less
3. A Principal-Agent Model of Systems Engineering Processes with Application to Satellite Design

   Safarkhani, Salar; Kattakuri, Vikranth; Bilionis, I.; Panchal, Jitesh H. (July 2018, Council of Engineering Systems Universities (CESUN) Global Conference)

   We present a principal-agent model of a one-shot, shallow, systems engineering process. The process is "one-shot" in the sense that decisions are made during a one-time step and that they are final. The term "shallow" refers to a one-layer hierarchy of the process. Specifically, we assume that the systems engineer has already decomposed the problem in subsystems and that each subsystem is assigned to a different subsystem engineer. Each subsystem engineer works independently to maximize their own expected payoff. The goal of the systems engineer is to maximize the system-level payoff by incentivizing the subsystem engineers. We restrict our attention to requirements-based system-level payoffs, i.e., the systems engineer makes a profit only if all the design requirements are met. We illustrate the model using the design of an Earth-orbiting satellite system where the systems engineer determines the optimum incentive structures and requirements for two subsystems: the propulsion subsystem and the power subsystem. The model enables the analysis of a systems engineer's decisions about optimal passed-down requirements and incentives for sub-system engineers under different levels of task difficulty and associated costs. Sample results, for the case of risk-neutral systems and subsystems engineers, show that it is not always in the best interest of the systems engineer to pass down the true requirements. As expected, the model predicts that for small to moderate task uncertainties the optimal requirements are higher than the true ones, effectively eliminating the probability of failure for the systems engineer. In contrast, the model predicts that for large task uncertainties the optimal requirements should be smaller than the true ones in order to lure the subsystem engineers into participation. 

   more » « less
4. Approximation-Aware Bayesian Optimization

   Maus, Natalie; Kim, Kyurae; Pleiss, Geoff; Eriksson, David; Cnningham, John P; Gardner, Jacob R (December 2024, Neural Information Processing Systems (NeurIPS 2024))

   High-dimensional Bayesian optimization (BO) tasks such as molecular design often require > 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design. 

   more » « less
5. Cost–benefit analysis of a distracted pedestrian intervention

   https://doi.org/10.1136/ip-2022-044740  

   Rahim, Md Jillur; Schwebel, David C; Hasan, Ragib; Griffin, Russell; Sen, Bisakha (November 2022, Injury Prevention)

   Objective Cellphone ubiquity has increased distracted pedestrian behaviour and contributed to growing pedestrian injury rates. A major barrier to large-scale implementation of prevention programmes is unavailable information on potential monetary benefits. We evaluated net economic societal benefits of StreetBit, a programme that reduces distracted pedestrian behaviour by sending warnings from intersection-installed Bluetooth beacons to distracted pedestrians’ smartphones. Methods Three data sources were used as follows: (1) fatal, severe, non-severe pedestrian injury rates from Alabama’s electronic crash reporting system; (2) expected costs per fatal, severe, non-severe pedestrian injury—including medical cost, value of statistical life, work-loss cost, quality-of-life cost—from CDC and (3) prevalence of distracted walking from extant literature. We computed and compared estimated monetary costs of distracted walking in Alabama and monetary benefits from implementing StreetBit to reduce pedestrian injuries at intersections. Results Over 2019–2021, Alabama recorded an annual average of 31 fatal, 83 severe and 115 non-severe pedestrian injuries in intersections. Expected costs/injury were US$11 million, US$339 535 and US$93 877, respectively. The estimated distracted walking prevalence is 25%–40%, and StreetBit demonstrates 19.1% (95% CI 1.6% to 36.0%) reduction. These figures demonstrate potential annual cost savings from using interventions like StreetBit statewide ranging from US$18.1 to US$29 million. Potential costs range from US$3 208 600 (beacons at every-fourth urban intersection) to US$6 359 200 (every other intersection). Conclusions Even under the most parsimonious scenario (25% distracted pedestrians; densest beacon placement), StreetBit yields US$11.8 million estimated net annual benefit to society. Existing data sources can be leveraged to predict net monetary benefits of distracted pedestrian interventions like StreetBit and facilitate large-scale intervention adoption. 

   more » « less