skip to main content


Title: Investigating Priming Effects of Sketch Evaluation Instructions on Idea Generation Productivity
Abstract

Engineering design involves intensive visual-spatial reasoning, and engineers depend upon external representation to develop concepts during idea generation. Previous research has not explored how our visual representation skills influence our idea generation effectiveness. A designer’s deficit in sketching skills could create a need for increased focus on the task of visual representation reducing cognitive resources available for the task at hand — generating concept. Further, this effect could be compounded if designers believed that their sketching skill would be evaluated or judged by their peers. This evaluation apprehension could cause additional mental workload distracting from the production of idea generation.

The goal of this study is to investigate and better understand the relationship between designers’ sketching skills and idea generation abilities. In this paper, we present preliminary results of the relationship between independent measures of sketching skill and idea generation ability from an entry-level engineering design and graphics course. During data collection, task instructions were given in two ways to independent groups: one group was instructed upfront that sketching would be evaluated, while the second group was kept blind to the sketch evaluation. In this paper, we also examine the potential priming effects of sketch quality evaluation apprehension on idea generation productivity. The results show that sketching quality and idea quantity are largely independent, and that the priming effects of sketch evaluation instructions are small to negligible on idea generation productivity.

 
more » « less
Award ID(s):
2013612
NSF-PAR ID:
10405702
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the ASME 2022 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC-CIE2022
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Engineering design involves intensive visual-spatial reasoning, and engineers depend upon external representation to develop concepts during idea generation. Previous research has not explored how our visual representation skills influence our idea generation effectiveness. A designer’s deficit in sketching skills could create a need for increased focus on the task of visual representation reducing cognitive resources available for the task at hand – generating concept. Further, this effect could be compounded if designers believed that their sketching skill would be evaluated or judged by their peers. This evaluation apprehension could cause additional mental workload distracting from the production of idea generation. The goal of this study is to investigate and better understand the relationship between designers’ sketching skills and idea generation abilities. In this paper, we present preliminary results of the relationship between independent measures of sketching skill and idea generation ability from an entry-level engineering design and graphics course. During data collection, task instructions were given in two ways to independent groups: one group was instructed upfront that sketching would be evaluated, while the second group was kept blind to the sketch evaluation. In this paper, we also examine the potential priming effects of sketch quality evaluation apprehension on idea generation productivity. The results show that sketching quality and idea quantity are largely independent, and that the priming effects of sketch evaluation instructions are small to negligible on idea generation productivity. 
    more » « less
  2. null (Ed.)
    Mobile devices are becoming a more common part of the education experience. Students can access their devices at any time to perform assignments or review material. Mobile apps can have the added advantage of being able to automatically grade student work and provide instantaneous feedback. However, numerous challenges remain in implementing effective mobile educational apps. One challenge is the small screen size of smartphones, which was a concern for a spatial visualization training app where students sketch isometric and orthographic drawings. This app was originally developed for iPads, but the wide prevalence of smartphones led to porting the software to iPhone and Android phones. The sketching assignments on a smartphone screen required more frequent zooming and panning, and one of the hypotheses of this study was that the educational effectiveness on smartphones was the same as on the larger screen sizes using iPad tablets. The spatial visualization mobile sketching app was implemented in a college freshman engineering graphics course to teach students how to sketch orthographic and isometric assignments. The app provides automatic grading and hint feedback to help students when they are stuck. Students in this pilot were assigned sketching problems as homework using their personal devices. Students were administered a pre- and post- spatial visualization test (PSVT-R, a reliable, well-validated instrument) to assess learning gains. The trial analysis focuses on students who entered the course with limited spatial visualization experience as identified based on a score of ≤70% on the PSVT:R since students entering college with low PSVT:R scores are at higher risk of dropping out of STEM majors. Among these low-performing students, those who used the app showed significant progress: (71%) raised their test scores above 70% bringing them out of the at-risk range for dropping out of engineering. While the PSVT:R test has been well validated, there are benefits to developing alternative methods of assessing spatial visualization skills. We developed an assembly pre- and post- test based upon a timed Lego™ exercise. At the start of the quarter, students were timed to see how long it would take them to build small lego sets using only visual instructions. Students were timed again on a different lego set after completion of the spatial visualization app. One benefit of the test was that it illustrated to the engineering students a skill that could be perceived as more relevant to their careers, and thus possibly increased their motivation for spatial visualization training. In addition, it may be possible to adapt the assembly test to elementary school grade levels where the PSVT:R test would not be suitable. Preliminary results show that the average lego build times decreased significantly after using the mobile app, indicating an improvement in students’ spatial reasoning skills. A comparison will also be done between normalized completion times on the assembly test and the PSVT:R tests in order to see how the assembly test compares to the “gold standard”. In addition to the PSVT-R instrument, a survey was conducted to evaluate student usage and their impressions of the app. Students found the app engaging, easy to use, and something they would do whenever they had “a free moment”. 95% of the students recommended the app to a friend if they are struggling with spatial visualization skills. This paper will describe the implementation of the mobile spatial visualization sketching app in a large college classroom, and highlight the app’s impact in increasing self-efficacy in spatial visualization and sketching 
    more » « less
  3. The success of a design is not determined solely by its technical aspects. A design must work for the people who will be using it and in the context in which it will be used. Human-centered design approaches suggest strategies to remind engineering designers of the people impacted by their design decisions. While many of these strategies can be used within and across multiple design phases, during the act of sketching concepts during concept generation, there are few explicit strategies for centering people. We investigate possible impact from a simple intervention during a concept generation task through a between-subjects experiment. Working alone is a single design session, half of a group of mechanical engineering students were asked to explicitly “represent people” within their conceptual sketches. Afterwards, all students reviewed each of their concepts to answer, “Who is this idea for? Who do you imagine would use it?” Those who received the intervention requiring representation of people within concept sketches produced significantly longer reflections with greater depth compared to the control group. Adding drawings of people to sketches resulted in more consideration of the social and physical context of use and of the user's personal preferences and values. Depicting people in generated concepts substantially reduced claims a design is "for everyone,” suggesting explicit representation of potential users produced more thoughtful consideration of diversity among potential users. 
    more » « less
  4. In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developed an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system. 
    more » « less
  5. In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developed an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system. 
    more » « less