skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Optimization Under Uncertainty Versus Algebraic Heuristics: A Research Method for Comparing Computational Design Methods
In this paper, we introduce a research method for comparing computational design methods. This research method addresses the challenge of measuring the difference in performance of different design methods in a way that is fair and unbiased with respect to differences in modeling abstraction, accuracy and uncertainty representation. The method can be used to identify the conditions under which each design method is most beneficial. To illustrate the research method, we compare two design methods for the design of a pressure vessel: 1) an algebraic approach, based on the ASME pressure vessel code, which accounts for uncertainty implicitly through safety factors, and 2) an optimization-based, expected-utility maximization approach which accounts for uncertainty explicitly. The computational experiments initially show that under some conditions the algebraic heuristic surprisingly outperforms the optimization-based approach. Further analysis reveals that an optimization-based approach does perform best as long as the designer applies good judgment during uncertainty elicitation. An ignorant or overly confident designer is better off using safety factors.  more » « less
Award ID(s):
1645316
PAR ID:
10026049
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In this work, we introduce an interval formulation that accounts for uncertainty in supporting conditions of structural systems. Uncertainty in structural systems has been the focus of a wide range of research. Different models of uncertain parameters have been used. Conventional treatment of uncertainty involves probability theory, in which uncertain parameters are modeled as random variables. Due to specific limitation of probabilistic approaches, such as the need of a prior knowledge on the distributions, lack of complete information, and in addition to their intensive computational cost, the rationale behind their results is under debate. Alternative approaches such as fuzzy sets, evidence theory, and intervals have been developed. In this work, it is assumed that only bounds on uncertain parameters are available and intervals are used to model uncertainty. Here, we present a new approach to treat uncertainty in supporting conditions. Within the context of Interval Finite Element Method (IFEM), all uncertain parameters are modeled as intervals. However, supporting conditions are considered in idealized types and described by deterministic values without accounting for any form of uncertainty. In the current developed approach, uncertainty in supporting conditions is modeled as bounded range of values, i.e., interval value that capture any possible variation in supporting condition within a given interval. Extreme interval bounds can be obtained by analyzing the considered system under the conditions of the presence and absence of the specific supporting condition. A set of numerical examples is presented to illustrate and verify the accuracy of the proposed approach. 
    more » « less
  2. Computational modeling of the human sequential design process and successful prediction of future design decisions are fundamental to design knowledge extraction, transfer, and the development of artificial design agents. However, it is often difficult to obtain designer-related attributes (static data) in design practices, and the research based on combining static and dynamic data (design action sequences) in engineering design is still underexplored. This paper presents an approach that combines both static and dynamic data for human design decision prediction using two different methods. The first method directly combines the sequential design actions with static data in a recurrent neural network (RNN) model, while the second method integrates a feed-forward neural network that handles static data separately, yet in parallel with RNN. This study contributes to the field from three aspects: (a) we developed a method of utilizing designers’ cluster information as a surrogate static feature to combine with a design action sequence in order to tackle the challenge of obtaining designer-related attributes; (b) we devised a method that integrates the function–behavior–structure design process model with the one-hot vectorization in RNN to transform design action data to design process stages where the insights into design thinking can be drawn; (c) to the best of our knowledge, it is the first time that two methods of combining static and dynamic data in RNN are compared, which provides new knowledge about the utility of different combination methods in studying sequential design decisions. The approach is demonstrated in two case studies on solar energy system design. The results indicate that with appropriate kernel models, the RNN with both static and dynamic data outperforms traditional models that only rely on design action sequences, thereby better supporting design research where static features, such as human characteristics, often play an important role. 
    more » « less
  3. null (Ed.)
    Abstract Engineers design for an inherently uncertain world. In the early stages of design processes, they commonly account for such uncertainty either by manually choosing a specific worst-case and multiplying uncertain parameters with safety factors or by using Monte Carlo simulations to estimate the probabilistic boundaries in which their design is feasible. The safety factors of this first practice are determined by industry and organizational standards, providing a limited account of uncertainty; the second practice is time intensive, requiring the development of separate testing infrastructure. In theory, robust optimization provides an alternative, allowing set-based conceptualizations of uncertainty to be represented during model development as optimizable design parameters. How these theoretical benefits translate to design practice has not previously been studied. In this work, we analyzed the present use of geometric programs as design models in the aerospace industry to determine the current state-of-the-art, then conducted a human-subjects experiment to investigate how various mathematical representations of uncertainty affect design space exploration. We found that robust optimization led to far more efficient explorations of possible designs with only small differences in an experimental participant’s understanding of their model. Specifically, the Pareto frontier of a typical participant using robust optimization left less performance “on the table” across various levels of risk than the very best frontiers of participants using industry-standard practices. 
    more » « less
  4. null (Ed.)
    Engineers design for an inherently uncertain world. In the early stages of design processes, they commonly account for such uncertainty either by manually choosing a specific worstcase and multiplying uncertain parameters with safety factors, or by using Monte Carlo simulations to estimate the probabilistic boundaries in which their design is feasible. The safety factors of this first practice are determined by industry and organizational standards, providing an inexpressive account of uncertainty; the second practice is time intensive, requiring the development of separate testing infrastructure. In theory, robust optimization provides an alternative, allowing set based conceptualizations of uncertainty to be represented during model development as optimizable design parameters. How these theoretical benefits translate to design practice has not previously been studied. In this work, we analyzed present use of geometric programs as design models in the aerospace industry to determine the current state-of-the-art, then conducted a human-subjects experiment to investigate how various mathematical representations of uncertainty affect design space exploration. We found that robust optimization led to far more efficient explorations of possible designs with only small differences in experimental participant’s understandings of their model. Specifically, the Pareto frontier of a typical participant using robust optimization left less performance “on the table” across various levels of risk than the very best frontiers of participants using industry-standard practices. 
    more » « less
  5. Parametric optimization techniques allow building designers to pursue multiple performance objectives, which can benefit the overall design. However, the strategies used by architecture and engineering graduate students when working with optimization tools are unclear, and ineffective computational design procedures may limit their success as future designers. In response, this re-search identifies several designerly behaviors of graduate students when responding to a conceptual building design optimization task. It uses eye-tracking, screen recording, and empirical methods to code their behaviors following the situated FBS framework. From these data streams, three different types of design iterations emerge: one by the designer alone, one by the optimizer alone, and one by the designer incorporating feedback from the optimizer. Based on the timing and frequency of these loops, student participants were characterized as completing partial, crude, or complete optimization cycles while developing their designs. This organization of optimization techniques establishes reoccurring strategies employed by developing designers, which can encourage future pedagogical approaches that empower students to incorporate complete optimization cycles while improving their designs. It can also be used in future research studies to establish clear links between types of design optimization behavior and design quality. 
    more » « less