skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: The effect of sketch and render quality and design experience on concept evaluation in engineering design
Abstract Illusory correlation (IC) is a cognitive bias that appears when decisions are based on false perception of patterns from limited data and can prevent subjects from detecting present correlations. It appears in design or psychology studies as a secondary bias, but has not been studied in-depth in engineering design. This research examines the presence of IC during concept evaluation of the engineering design process and how current engineering design education may mitigate the appearance of IC. To examine IC, different products at four different sketch quality levels and render quality levels, from quick hand drawing to shaded product render, were presented to participants through a survey-based data collection instrument. The four sketch and/or render quality levels simulate the variety of presented drawings when a design engineer is evaluating colleagues’ concepts during group work. Participants, 70 undergraduate (novice) and 21 graduate (advanced) engineering students at a major southeast US institution, were asked to rank these products based on a series of function-based and preference-based attributes. The collected data were analyzed to see if sketch and/or render quality impacted participants’ ability to gage functionality of the presented products. Results indicated no statistically significant linear correlation between better rankings of products and the sketch and/or render quality level of the provided depictions for function-based questions; however, a non-linear relationship was present for preference-based questions where participants gave higher rankings to products drawn at intermediate quality levels. No statistically significant differences were found in the strength of correlations between rankings and sketch and/or render quality levels in the comparison of novice and advanced student designers, but advanced student designers’ perception of product functionality was more strongly correlated to pre-determined baseline answers based on user ratings of the selected products. This may indicate less vulnerability to IC bias with more design engineering education due to a stronger intuition for and understanding of product functionality based on visual inspection.  more » « less
Award ID(s):
2207448
PAR ID:
10584477
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
Research in Engineering Design
Volume:
36
Issue:
2
ISSN:
0934-9839
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The fuzzy front end of engineering design can present a difficult challenge, and as such, recent engineering design research has focused on guiding and influencing the way a designer ideates. Early ideation can be especially difficult when attempting to integrate specific design objectives in product design, called Design for X (DfX). Some examples of DfX are Design for Manufacturing (DfM), Design for Assembly (DfA), Design for Function (DfF), and Design for Safety (DfS). This paper will present two experiments exploring the efficacy of a structured Design for the Environment (DfE) design method called the GREEn Quiz (Guidelines and Regulations for Early design for the Environment) that provides designers with sustainable design knowledge during the conceptual design phase. The GREEn Quiz operates on a web-based platform and queries the designer about their design concepts; an end-of-quiz report provides abstract DfE knowledge to designers. While this abstract knowledge was able to be applied by designers in a former study, we hypothesize that providing targeted, specific design strategies during conceptual design may enable better integration in concept generation by novice designers. In this study, we created these DfE strategies, embedded these in the GREEn Quiz, and studied the efficacy of these strategies when presented to designers at both the expert and novice levels. Experimental results suggest that respondents with access to the strategy-based GREEn Quiz produced concepts with evidence of more sustainable design decisions and higher solution quality scores when compared to previous respondents and the control groups. This research encourages the consideration of downstream environmental impact knowledge during conceptual design, resulting in lower-impact products regardless of the previous DfE expertise of the designer. 
    more » « less
  2. null (Ed.)
    Engineering designers currently use downstream information about product and component functions to facilitate ideation and concept generation of analogous products. These processes, often called Function-Based Design, can be reliant on designer definitions of product function, which are inconsistent from designer to designer. In this paper, we employ supervised learning algorithms to reduce the variety of component functions that are available to designers in a design repository, thus enabling designers to focus their function-based design efforts on more accurate, reduced sets of potential functions. To do this, we generate decisions trees and rules that define the functions of components based on the identity of neighboring components. The resultant decision trees and rulesets reduce the number of feasible functions for components within a product, which is of particular interest for use by novice designers, as reducing the feasible functional space can help focus the design activities of the designer. This reduction was evident in both case studies: one exploring a component that is known to the designer, and the other looking at defining function of an unrecognizable component. The work presented here contributes to the recent popularity of using product data in data-driven design methodologies, especially those focused on supplementing designer cognition. Importantly, we found that this methodology is reliant on repository data quality, and the results indicate a need to continue the development of design repository data schemas with improved data consistency and fidelity. This research is a necessary precursor for the development of function-based design tools, including automated functional modeling. 
    more » « less
  3. Generative AI has enabled novice designers to quickly create professional-looking visual representations for product concepts. However, novices have limited domain knowledge that could constrain their ability to write prompts that effectively explore a product design space. To understand how experts explore and communicate about design spaces, we conducted a formative study with 12 experienced product designers and found that experts — and their less-versed clients — often use visual references to guide co-design discussions rather than written descriptions. These insights inspired DesignWeaver, an interface that helps novices generate prompts for a text-to-image model by surfacing key product design dimensions from generated images into a palette for quick selection. In a study with 52 novices, DesignWeaver enabled participants to craft longer prompts with more domain-specific vocabularies, resulting in more diverse, innovative product designs. However, the nuanced prompts heightened participants’ expectations beyond what current text-to-image models could deliver. We discuss implications for AI-based product design support tools. 
    more » « less
  4. Effective communication is an integral part of engineering design and leads to successful design outcomes. While there have been extensive calls to equip novice designers with effective communication skills, there is only a limited body of work that has attempted to characterize the communication patterns of novice designers, particularly when engaging with external audiences. This work seeks to characterize how the project type, or the nature of design problem, shapes the communicative patterns of novice designers when communicating design outcomes to external audiences. Presentations of design solutions from 46 teams were collected at the end of a semester-long capstone design program. These presentations were then characterized as industry- and human-centered projects. Analysis was conducted using topic modeling and Linguistic Inquiry and Word Count analysis to identify differences in linguistic patterns of novice designers between the two project types. Contrary to prior findings, no significant differences were found, implying that the communication patterns of novice designers are not affected by the type of project (or design problem being solved). 
    more » « less
  5. Algorithmic decision-making using rankings— prevalent in areas from hiring and bail to university admissions— raises concerns of potential bias. In this paper, we explore the alignment between people’s perceptions of fairness and two popular fairness metrics designed for rankings. In a crowdsourced experiment with 480 participants, people rated the perceived fairness of a hypothetical scholarship distribution scenario. Results suggest a strong inclination towards relying on explicit score values. There is also evidence of people’s preference for one fairness metric, NDKL, over the other metric, ARP. Qualitative results paint a more complex picture: some participants endorse meritocratic award schemes and express concerns about fairness metrics being used to modify rankings; while other participants acknowledge socio-economic factors in score-based rankings as justification for adjusting rankings. In summary, we find that operationalizing algorithmic fairness in practice is a balancing act between mitigating harms towards marginalized groups and societal conventions of leveraging traditional performance scores such as grades in decision-making contexts. 
    more » « less