Abstract Existing literature on information sharing in contests has established that sharing contest-specific information influences contestant behaviors, and thereby, the outcomes of a contest. However, in the context of engineering design contests, there is a gap in knowledge about how contest-specific information such as competitors’ historical performance influences designers’ actions and the resulting design outcomes. To address this gap, the objective of this study is to quantify the influence of information about competitors’ past performance on designers’ belief about the outcomes of a contest, which influences their design decisions, and the resulting design outcomes. We focus on a single-stage design competition where an objective figure of merit is available to the contestants for assessing the performance of their design. Our approach includes (i) developing a behavioral model of sequential decision making that accounts for information about competitors’ historical performance and (ii) using the model in conjunction with a human-subject experiment where participants make design decisions given controlled strong or weak performance records of past competitors. Our results indicate that participants spend greater efforts when they know that the contest history reflects that past competitors had a strong performance record than when it reflects a weak performance record. Moreover, we quantify cognitive underpinnings of such informational influence via our model parameters. Based on the parametric inferences about participants’ cognition, we suggest that contest designers are better off not providing historical performance records if past contest outcomes do not match their expectations setup for a given design contest.
more »
« less
Human Inductive Biases in Design Decision Making
Designers make information acquisition decisions, such as where to search and when to stop the search. Such decisions are typically made sequentially, such that at every search step designers gain information by learning about the design space. However, when designers begin acquiring information, their decisions are primarily based on their prior knowledge. Prior knowledge influences the initial set of assumptions that designers use to learn about the design space. These assumptions are collectively termed as inductive biases. Identifying such biases can help us better understand how designers use their prior knowledge to solve problems in the light of uncertainty. Thus, in this study, we identify inductive biases in humans in sequential information acquisition tasks. To do so, we analyze experimental data from a set of behavioral experiments conducted in the past [1–5]. All of these experiments were designed to study various factors that influence sequential information acquisition behaviors. Across these studies, we identify similar decision making behaviors in the participants in their very first decision to “choose x”. We find that their choices of “x” are not uniformly distributed in the design space. Since such experiments are abstractions of real design scenarios, it implies that further contextualization of such experiments would only increase the influence of these biases. Thus, we highlight the need to study the influence of such biases to better understand designer behaviors. We conclude that in the context of Bayesian modeling of designers’ behaviors, utilizing the identified inductive biases would enable us to better model designer’s priors for design search contexts as compared to using non-informative priors.
more » « less- Award ID(s):
- 1662230
- NSF-PAR ID:
- 10281766
- Date Published:
- Journal Name:
- ASME 2020 International Design Engineering Technical Conferences and Computers and Information Science in Engineering
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)more » « less
Abstract In this study, we focus on crowdsourcing contests for engineering design problems where contestants search for design alternatives. Our stakeholder is a designer of such a contest who requires support to make decisions, such as whether to share opponent-specific information with the contestants. There is a significant gap in our understanding of how sharing opponent-specific information influences a contestant’s information acquisition decision such as whether to stop searching for design alternatives. Such decisions in turn affect the outcomes of a design contest. To address this gap, the objective of this study is to investigate how participants’ decision to stop searching for a design solution is influenced by the knowledge about their opponent’s past performance. The objective is achieved by conducting a protocol study where participants are interviewed at the end of a behavioral experiment. In the experiment, participants compete against opponents with strong (or poor) performance records. We find that individuals make decisions to stop acquiring information based on various thresholds such as a target design quality, the number of resources they want to spend, and the amount of design objective improvement they seek in sequential search. The threshold values for such stopping criteria are influenced by the contestant’s perception about the competitiveness of their opponent. Such insights can enable contest designers to make decisions about sharing opponent-specific information with participants, such as the resources utilized by the opponent towards purposefully improving the outcomes of an engineering design contest.
-
Introduction Integrated water management (IWM) involves a range of policies, actions, and organizational processes that go beyond traditional hydrology to consider multifaceted aspects of complex water resource systems. Due to its transdisciplinary nature, IWM comprises input from diverse stakeholders, each with unique perceptions, values, and experiences. However, stakeholders from differing backgrounds may disagree on best practices and collective paths forward. As such, successful IWM must address key governance principles (e.g., information flow, collective decision-making, and power relations) across social and institutional scales. Here, we sought to demonstrate how network structure impacts shared decision-making within IWM. Methods We explored a case study in Houston, Texas, USA, where decision-making stakeholders from various sectors and levels of governance engaged in a participatory modeling workshop to improve adoption of nature-based solutions (NBS) through IWM. The stakeholders used fuzzy cognitive mapping (FCM) to define an IWM model comprising multifaceted elements and their interrelationships, which influenced the adoption of NBS in Houston. We applied grounded theory and inductive reasoning to categorize tacit belief schemas regarding how stakeholders viewed themselves within the management system. We then used FCM-based modeling to explore how unique NBS policies would translate into more (or less) NBS adoption. Finally, we calculated specific network metrics (e.g., density, hierarchy, and centrality indices) to better understand the structure of human-water relations embedded within the IWM model. We compared the tacit assumptions about stakeholder roles in IWM against the quantitative degrees of influence and collectivism embedded within the stakeholder-defined model. Results and discussion Our findings revealed a mismatch between stakeholders' external belief statements about IWM and their internal assumptions through cognitive mapping and participatory modeling. The case study network was characterized by a limited degree of internal coordination (low density index), high democratic potential (low hierarchy index), and high-efficiency management opportunities (high centrality index), which transcended across socio-institutional scales. These findings contrasted with several of the belief schemas described by stakeholders during the group workshop. We describe how ongoing partnership with the stakeholders resulted in an opportunity for adaptive learning, where the NBS planning paradigm began to shift toward trans-scale collaboration aimed at high-leverage management opportunities. We emphasize how network analytics allowed us to better understand the extent to which key governance principles drove the behavior of the IWM model, which we leveraged to form deeper stakeholder partnerships by identifying hidden opportunities for governance transformation.more » « less
-
Designers can benefit from inspirational stimuli when presented during the design process. Encountering external stimuli can also lead designers to negative design outcomes by limiting exploration of the design space and idea generation. Prior work has investigated how specific features of inspirational stimuli can be beneficial or harmful to designers. However, the processes designers use to search for and discover inspirational stimuli leading to these outcomes are less known. The objective of this work is thus to better understand how designers search for inspirational design stimuli. Specifically, we investigate how factors such as designer expertise and search modality (e.g., text vs. visual-based) impact both explicit and implicit features during the search for design stimuli. A cognitive study was completed by novice and expert designers (seven students and eight professionals), who searched for design stimuli using a novel multi-modal search platform while following a think-aloud protocol. The multi-modal search platform enabled search using text and nontext inputs, and provided design stimuli in the form of 3D-model parts. This work presents methods to describe search processes in terms of three levels: activities, behaviors, and pathways, as defined in this paper. Our findings determine that design expertise and search modality influence search behavior. Illustrative examples are presented and discussed of search processes leading designers to both negative and beneficial outcomes, such as designers fixating on specific results or benefiting unexpectedly from unintentional inspirational stimuli. Overall, this work contributes to an improved understanding of how designers search for inspiration, and key factors influencing these behaviors.more » « less
-
null (Ed.)Heuristics are essential for addressing the complexities of engineering design processes. The goodness of heuristics is context-dependent. Appropriately tailored heuristics can enable designers to find good solutions efficiently, and inappropriate heuristics can result in cognitive biases and inferior design outcomes. While there have been several efforts at understanding which heuristics are used by designers, there is a lack of normative understanding about when different heuristics are suitable. Towards addressing this gap, this paper presents a reinforcement learning-based approach to evaluate the goodness of heuristics for three sub-problems commonly faced by designers while carrying out design under resource constraints: (i) learning the mapping between the design space and the performance space, (ii) sequential information acquisition in design, and (iii) decision to stop information acquisition. Using a multi-armed bandit formulation and simulation studies, we learn the heuristics that are suitable for these sub-problems under different resource constraints and problem complexities. The results of our simulation study indicate that the proposed reinforcement learning-based approach can be effective for determining the quality of heuristics for different sub-problems, and how the effectiveness of the heuristics changes as a function of the designer's preference (e.g., performance versus cost), the complexity of the problem, and the resources available.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1115/DETC2020-22252
