skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Best Fits and Dark Horses: Can Design Teams Tell the Difference?
Abstract Design teams are often asked to produce solutions of a certain type in response to design challenges. Depending on the circumstances, they may be tasked with generating a solution that clearly follows the given specifications and constraints of a problem (i.e., a Best Fit solution), or they may be encouraged to provide a higher risk solution that challenges those constraints, but offers other potential rewards (i.e., a Dark Horse solution). In the current research, we investigate: what happens when design teams are asked to generate solutions of both types at the same time? How does this request for dual and conflicting modes of thinking impact a team’s design solutions? In addition, as concept generation proceeds, are design teams able to discern which solution fits best in each category? Rarely, in design research, do we prompt design teams for “normal” designs or ask them to think about both types of solutions (boundary preserving and boundary challenging) at the same time. This leaves us with the additional question: can design teams tell the difference between Best Fit solutions and Dark Horse solutions? In this paper, we present the results of an exploratory study with 17 design teams from five different organizations. Each team was asked to generate both a Best Fit solution and a Dark Horse solution in response to the same design prompt. We analyzed these solutions using rubrics based on familiar design metrics (feasibility, usefulness, and novelty) to investigate their characteristics. Our assumption was that teams’ Dark Horse solutions would be more novel, less feasible, but equally useful when compared with their Best Fit solutions. Our analysis revealed statistically significant results showing that teams generally produced Best Fit solutions that were more useful (met client needs) than Dark Horse solutions, and Dark Horse solutions that were more novel than Best Fit solutions. When looking at each team individually, however, we found that Dark Horse concepts were not always more novel than Best Fit concepts for every team, despite the general trend in that direction. Some teams created equally novel Best Fit and Dark Horse solutions, and a few teams generated Best Fit solutions that were more novel than their Dark Horse solutions. In terms of feasibility, Best Fit and Dark Horse solutions did not show significant differences. These findings have implications for both design educators and design practitioners as they frame design prompts and tasks for their teams of interest.  more » « less
Award ID(s):
1635437
PAR ID:
10301780
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
ASME 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
Volume:
3
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Proceedings of the ASME 2021 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference)
    null (Ed.)
    Nearly 60 years ago, Thomas Kuhn revolutionized how we think of scientific discovery and innovation when he identified that scientific change can occur in incremental developments that improve upon existing solutions, or it can occur as drastic change in the form of a paradigm shift. In engineering design, both types of scientific change are critical when exploring the solution space. However, most methods of examining design outputs look at whether an idea is creative or not and not the type of creativity that is deployed or if we can predict what types of individuals or teams is more likely to develop a paradigm-shifting idea. Without knowing how to identify who will generate ideas that fit a certain paradigm, we do not know how to build teams that can develop ideas that better explore the solution space. This study provides the first attempt at answering this question through an empirical study with 60 engineering design student teams over the course of a 4- and 8-week design project. Specifically, we sought to identify the role of cognitive style using KAI score, derived from Kirton’s Adaption-Innovation (A-I) theory, on the paradigm-relatedness of ideas generated by individuals and teams. We also sought to investigate the role of crowdsourcing for measuring the paradigm-relatedness of design solutions. The results showed that KAI was positively related to a greater likelihood of an individual’s idea being categorized as paradigm-breaking. In addition, the team KAI diversity was also linked to a greater likelihood of teams’ ideas being categorized as paradigm-challenging. Finally, the results support the use of crowdsourcing for measuring the paradigm-relatedness of design solutions. 
    more » « less
  2. (, Proc. of the ASEE 2018 Annual Conference & Exposition)
    In this pilot study, we used the Interaction Dynamics Notation (IDN), originally designed for use with engineering design teams, to explore the dynamic interactions of five NSF I-Corps™ teams engaged in a simple design activity. Our aim was to relate these interaction data to selected cognitive characteristics of the team members, as well as team design outcomes and individual perceptions related to the experience. The individual cognitive characteristics we assessed focused on cognitive style, as measured by the Kirton Adaption-Innovation inventory (KAI), while team outcomes included the novelty, usefulness, and feasibility of each team’s design solutions, as well as their success within and beyond the NSF I-Corps™ program. Our findings show that the Interaction Dynamics Notation (IDN) can be readily extended to the study of entrepreneurial teams, with important insights gained from the combined study of interaction dynamics, individual cognitive characteristics as measured by KAI, and team outcomes. The results of this study demonstrate the feasibility and value of this approach for investigating the dynamic interactions of NSF I-Corps™ teams, as well as product-focused design teams in general. 
    more » « less
  3. ; ; ; ; ; (, TransAI 2019)
    The aggregation of individual personality tests to predict team performance is widely accepted in management theory but has significant limitations: the isolated nature of individual personality surveys fails to capture much of the team dynamics that drive real-world team performance. Artificial Swarm Intelligence (ASI), a technology that enables networked teams to think together in real-time and answer questions as a unified system, promises a solution to these limitations by enabling teams to take personality tests together, whereby the team uses ASI to converge upon answers that best represent the group’s disposition. In the present study, the group personality of 94 small teams was assessed by having teams take a standard Big Five Inventory (BFI) test both as individuals, and as a real-time system enabled by an ASI technology known as Swarm AI. The predictive accuracy of each personality assessment method was assessed by correlating the BFI personality traits to a range of real-world performance metrics. The results showed that assessments of personality generated using Swarm AI were far more predictive of team performance than the traditional survey-based method, showing a significant improvement in correlation with at least 25% of performance metrics, and in no case showing a significant decrease in predictive performance. This suggests that Swarm AI technology may be used as a highly effective team personality assessment tool that more accurately predicts future team performance than traditional survey approaches. 
    more » « less
  4. ; ; ; ; ; (, NeurIPS23)
    Recent algorithms have achieved superhuman performance at a number of twoplayer zero-sum games such as poker and go. However, many real-world situations are multi-player games. Zero-sum two-team games, such as bridge and football, involve two teams where each member of the team shares the same reward with every other member of that team, and each team has the negative of the reward of the other team. A popular solution concept in this setting, called TMECor, assumes that teams can jointly correlate their strategies before play, but are not able to communicate during play. This setting is harder than two-player zerosum games because each player on a team has different information and must use their public actions to signal to other members of the team. Prior works either have game-theoretic guarantees but only work in very small games, or are able to scale to large games but do not have game-theoretic guarantees. In this paper we introduce two algorithms: Team-PSRO, an extension of PSRO from twoplayer games to team games, and Team-PSRO Mix-and-Match which improves upon Team PSRO by better using population policies. In Team-PSRO, in every iteration both teams learn a joint best response to the opponent’s meta-strategy via reinforcement learning. As the reinforcement learning joint best response approaches the optimal best response, Team-PSRO is guaranteed to converge to a TMECor. In experiments on Kuhn poker and Liar’s Dice, we show that a tabular version of Team-PSRO converges to TMECor, and a version of Team PSRO using deep cooperative reinforcement learning beats self-play reinforcement learning in the large game of Google Research Football. 
    more » « less
  5. ; ; ; (, 2022 ASEE Annual Conference & Exposition Proceedings)
    Problem-solving is a critical skill in the workplace, but recent college graduates are often deficient in problem-solving skills. Introductory STEM courses present engineering students with well-structured problems with single-path solutions that do not prepare students with the problem-solving skills they will need to solve complex problems within authentic engineering contexts. When presented with complex problems in authentic contexts, engineering students find it difficult to transfer the scientific knowledge learned in their STEM courses to solve these integrated and ill structured problems. By integrating physics laboratories with engineering design problems, students are taught to apply physics principles to solve ill-structured and complex engineering problems. The integration of engineering design processes to physics labs is meant to help students transfer physics learning to engineering problems, as well as to transfer the design skills learned in their engineering courses to the physics lab. We hypothesize this integration will help students become better problem solvers when they go out to industry after graduation. The purpose of this study is to examine how students transfer their understanding of physics concepts to solve ill-structured engineering problems by means of an engineering design project in a physics laboratory. We use a case-study methodology to examine two cases and analyze the cases using a lens of co-regulated learning and transfer between physics and engineering contexts. Observations were conducted using transfer lenses. That is, we observed groups during the physics labs for evidence of transfer. The research question for this study was, to what extent do students relate physics concepts with concepts from other materials (classes) through an engineering design project incorporated in a physics laboratory? Teams were observed over the course of 6 weeks as they completed the second design challenge. The cases presented in this study were selected using observations from the lab instructors of the team’s work in the first design project. Two teams, one who performed well, and one that performed poorly, were selected to be observed to provide insight on how students use physics concepts to engage in the design process. The second design challenge asked students to design an eco-friendly way of delivering packages of food to an island located in the middle of the river, which is home to critically endangered species. They are given constraints that the solution cannot disrupt the habitat in any way, nor can the animals come into contact directly with humans or loud noises. Preliminary results indicate that both teams successfully demonstrated transfer between physics and engineering contexts, and integrated physics concepts from multiple labs to complete the design project. Teams that struggle seem to be less connected with the design process at the beginning of the project and are less organized. In contrast, teams that are successful demonstrate greater co-regulated learning (communication, reflection, etc.) and focus on making connections between the physics concepts and principles of engineering design from their engineering course work. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email