An official website of the United States government
Abstract Existing literature on information sharing in contests has established that sharing contest-specific information influences contestant behaviors, and thereby, the outcomes of a contest. However, in the context of engineering design contests, there is a gap in knowledge about how contest-specific information such as competitors’ historical performance influences designers’ actions and the resulting design outcomes. To address this gap, the objective of this study is to quantify the influence of information about competitors’ past performance on designers’ belief about the outcomes of a contest, which influences their design decisions, and the resulting design outcomes. We focus on a single-stage design competition where an objective figure of merit is available to the contestants for assessing the performance of their design. Our approach includes (i) developing a behavioral model of sequential decision making that accounts for information about competitors’ historical performance and (ii) using the model in conjunction with a human-subject experiment where participants make design decisions given controlled strong or weak performance records of past competitors. Our results indicate that participants spend greater efforts when they know that the contest history reflects that past competitors had a strong performance record than when it reflects a weak performance record. Moreover, we quantify cognitive underpinnings of such informational influence via our model parameters. Based on the parametric inferences about participants’ cognition, we suggest that contest designers are better off not providing historical performance records if past contest outcomes do not match their expectations setup for a given design contest.
more »
« less
Sequential Design Decision Making Under the Influence of Competition: A Protocol Analysis
Abstract In this study, we focus on crowdsourcing contests for engineering design problems where contestants search for design alternatives. Our stakeholder is a designer of such a contest who requires support to make decisions, such as whether to share opponent-specific information with the contestants. There is a significant gap in our understanding of how sharing opponent-specific information influences a contestant’s information acquisition decision such as whether to stop searching for design alternatives. Such decisions in turn affect the outcomes of a design contest. To address this gap, the objective of this study is to investigate how participants’ decision to stop searching for a design solution is influenced by the knowledge about their opponent’s past performance. The objective is achieved by conducting a protocol study where participants are interviewed at the end of a behavioral experiment. In the experiment, participants compete against opponents with strong (or poor) performance records. We find that individuals make decisions to stop acquiring information based on various thresholds such as a target design quality, the number of resources they want to spend, and the amount of design objective improvement they seek in sequential search. The threshold values for such stopping criteria are influenced by the contestant’s perception about the competitiveness of their opponent. Such insights can enable contest designers to make decisions about sharing opponent-specific information with participants, such as the resources utilized by the opponent towards purposefully improving the outcomes of an engineering design contest.
more »
« less
- Award ID(s):
- 1662230
- PAR ID:
- 10281765
- Date Published:
- Journal Name:
- ASME 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Designers make information acquisition decisions, such as where to search and when to stop the search. Such decisions are typically made sequentially, such that at every search step designers gain information by learning about the design space. However, when designers begin acquiring information, their decisions are primarily based on their prior knowledge. Prior knowledge influences the initial set of assumptions that designers use to learn about the design space. These assumptions are collectively termed as inductive biases. Identifying such biases can help us better understand how designers use their prior knowledge to solve problems in the light of uncertainty. Thus, in this study, we identify inductive biases in humans in sequential information acquisition tasks. To do so, we analyze experimental data from a set of behavioral experiments conducted in the past [1–5]. All of these experiments were designed to study various factors that influence sequential information acquisition behaviors. Across these studies, we identify similar decision making behaviors in the participants in their very first decision to “choose x”. We find that their choices of “x” are not uniformly distributed in the design space. Since such experiments are abstractions of real design scenarios, it implies that further contextualization of such experiments would only increase the influence of these biases. Thus, we highlight the need to study the influence of such biases to better understand designer behaviors. We conclude that in the context of Bayesian modeling of designers’ behaviors, utilizing the identified inductive biases would enable us to better model designer’s priors for design search contexts as compared to using non-informative priors.more » « less
-
Effective coordination of design teams must account for the influence of costs incurred while searching for the best design solutions. This article introduces a cost-aware multi-agent system (MAS), a theoretical model to (1) explain how individuals in a team should search, assuming that they are all rational utility-maximizing decision-makers and (2) study the impact of cost on the search performance of both individual agents and the system. First, we develop a new multi-agent Bayesian optimization framework accounting for information exchange among agents to support their decisions on where to sample in search. Second, we employ a reinforcement learning approach based on the multi-agent deep deterministic policy gradient for training MAS to identify where agents cannot sample due to design constraints. Third, we propose a new cost-aware stopping criterion for each agent to determine when costs outweigh potential gains in search as a criterion to stop. Our results indicate that cost has a more significant impact on MAS communication in complex design problems than in simple ones. For example, when searching in complex design spaces, some agents could initially have low-performance gains, thus stopping prematurely due to negative payoffs, even if those agents could perform better in the later stage of the search. Therefore, global-local communication becomes more critical in such situations for the entire system to converge. The proposed model can serve as a benchmark for empirical studies to quantitatively gauge how humans would rationally make design decisions in a team.more » « less
-
Abstract The objective of this study is to investigate students’ decision-making during the information gathering activities of a design process. Existing literature in engineering education has shown that students face difficulties while gathering information in various activities of a design process such as brainstorming and CAD modeling. Decision-making is an important aspect of these activities. While gathering information, students make several decisions such as what information to acquire and how to acquire that information. There lies a research gap in understanding how students make decisions while gathering information in a product design process. To address this gap, we conduct semi-structured interviews and surveys in a product design course. We analyze the students’ decision-making activities from the lens of a sequential information acquisition and decision-making (SIADM) framework. We find that the students recognize the need to acquire information about the physics and dynamics of their design artifact during the CAD modeling activity of the product design process. However, they do not acquire such information from their CAD models primarily due to the lack of the project requirements, their ability, and the time to do so. Instead, they acquire such information from the prototyping activity as their physical prototype does not satisfy their design objectives. However, the students do not get the opportunity to iterate their prototype with the given cost and time constraints. Consequently, they rely on improvising during prototyping. Based on our observations, we discuss the need for designing course project activities such that it facilitates students’ product design decisions.more » « less
-
The main objective of this project is to help students learn to make decisions that lead to academic success. Our first goal is to map curriculum pathways, which begins by studying overpersistence (when a student persists in a particular major but does not make timely progress toward a degree). We seek to identify curriculum-specific indicators of overpersistence and corresponding alternative paths that could lead to success. Our second goal is to improve the structure of the Decision-Making Competency Inventory (DMCI) so that it can explain student's decision-making competency in more detail and in congruence with the Self-Regulation Model of Decision-Making. This instrument will be used to map decision-making competency to academic choices and outcomes. The third goal is to develop an Academic Dashboard as a means for sharing relevant research results with students. This will allow students to have access to the strategies, information, and stories needed to make and implement adaptive decisions. This paper highlights our progress in the fifth year of the project and our plans going forward.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1115/DETC2020-22254
