An official website of the United States government
Heuristics are cognitive strategies used to efficiently achieve an outcome and are used in the daily practice of many disciplines. Understanding the heuristics used by experts can help researchers and practitioners to better understand an activity and develop systems to support the efficacy and development of novices. While heuristics have been well-documented in psychology, industrial design, and engineering disciplines, they are not as thoroughly understood in the field of instructional design, or engineering course design in particular. This study sought to address that gap by using thematic analysis to identify and group the heuristics used by nine educators redesigning a second-year embedded systems course for electrical, computer, and software engineering students. We collected a variety of data, including audio recordings and written notes from team meetings, design artifacts (including final course materials), interviews with team members, and semi-weekly reflections from the course instructor, to explore heuristics from multiple lenses. We identified 22 heuristics, which were further grouped into 6 categories. The paper describes these heuristics and provides concrete examples of how they were used in practice. These findings indicate the prevalence of heuristics for engineering course design and suggests that additional heuristics can be identified across different educational settings.
more »
« less
An Exploration of Course Design Heuristics Identified from Design Meetings, Design Artifacts, and Educator Interviews
This research paper investigates differences between course design heuristics that have been identified from three distinct data sources: course design team meetings, educator interviews, and course design papers. The study of heuristics used by experts in a discipline can have several practical benefits. They can (1) be employed as tools to scaffold expert behavior among novices, (2) be translated into processes to make challenging tasks more efficient, and (3) provide deeper insights into the nature of a domain, task, or discipline. While the study of heuristics remains robust across domains, they have demonstrated differences in format and have been identified through a variety of data types. The purpose of this study is to unpack differences in heuristics independently identified through different data types in order to better understand the role these types of data can play in understanding of heuristics for course design, especially as related to engineering courses. We utilized thematic analysis to explore the patterns of differences between heuristics identified from the three settings in three related, but distinct studies. Datasets includes audio-recordings from a four-month team course redesign process, five approximately hour-long educator interviews, and 183 peer-reviewed course design papers. We identified four themes representing differences across the datasets: (1) differences in volume/frequency of heuristics, (2) differences in breadth, specificity, and conceptualizations evidenced by categories of heuristics, (3) individual heuristic specificity, and (4) locus of clarity in heuristic examples. These results inform a set of four considerations for selecting data sources for studies of heuristics within engineering course design and other domains.
more »
« less
- Award ID(s):
- 1623125
- PAR ID:
- 10337950
- Date Published:
- Journal Name:
- 2019 ASEE Annual Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This study works toward addressing a knowledge gap in understanding how heuristics are developed, retrieved, employed, and modified by designers. Having a better awareness of one’s own set of heuristics can be beneficial for relaying to other team members, improving a team’s training processes, and aiding others on their path to design expertise. The ability to understand and justify the use of a heuristic should lead to more effective decision-making in systems design. To do this, the heuristics and their characteristics must be extracted using a repeatable scientific research methodology. This study describes a unique extraction and characterization process compared to prior literature. It includes some of the first work towards documenting heuristics for both designers and operators in a hybrid manufacturing setting. Eight participants performed a series of two design journals, two interviews, and one survey. Heuristics were extracted and refined between each method and then verified by participants in the survey. The surveys produced novel statistically significant findings in regard to heuristic characterizations, impacting how participants view how often a heuristic is used, the reliability of the heuristic, and the evolution of the heuristic. Lastly, an alternate perspective of heuristics as an error management bias is highlighted and discussed.more » « less
-
Abstract Design heuristics are traditionally used as qualitative principles to guide the design process, but they have also been used to improve the efficiency of design optimization. Using design heuristics as soft constraints or search operators has been shown for some problems to reduce the number of function evaluations needed to achieve a certain level of convergence. However, in other cases, enforcing heuristics can reduce diversity and slow down convergence. This paper studies the question of when and how a given set of design heuristics represented in different forms (soft constraints, repair operators, and biased sampling) can be utilized in an automated way to improve efficiency for a given design problem. An approach is presented for identifying promising heuristics for a given problem by estimating the overall impact of a heuristic based on an exploratory screening study. Two impact indices are formulated: weighted influence index and hypervolume difference index. Using this approach, the promising heuristics for four design problems are identified and the efficacy of selectively enforcing only these promising heuristics over both enforcement of all available heuristics and not enforcing any heuristics is benchmarked. In all problems, it is found that enforcing only the promising heuristics as repair operators enables finding good designs faster than by enforcing all available heuristics or not enforcing any heuristics. Enforcing heuristics as soft constraints or biased sampling functions results in improvements in efficiency for some of the problems. Based on these results, guidelines for designers to leverage heuristics effectively in design optimization are presented.more » « less
-
Substantial research over the past few decades has documented the challenges women experience both as students in engineering programs and as professionals in engineering workplaces. Few studies, however, have followed women from one context to the other to explore the ways in which school experiences, and particularly capstone experiences designed explicitly to facilitate this transition, do and do not prepare women for their work as practicing engineers. To address this gap, we draw on data from a larger multi-institution study to address the question, “How do women engineers experience the transition from school to work?” The sample for this study includes 23 participants from four different universities (three mechanical engineering programs and one engineering science program). All participants identified as “female” on a screen questionnaire that included options for transgender and gender-nonconforming, as well as an option to skip the question. The data set includes interviews with the participants conducted at the end of their capstone design course, responses to open-ended questions sent each week during their first 12 weeks of work, and interviews conducted after their first three months of work. The capstone interviews explored participants’ experiences in their capstone design course, including project role, significant challenges and accomplishments, and perceived learning, as well as their plans for and expectations of their post-graduation work. The weekly open-ended questions asked participants to describe their most significant challenge over the past week and to explain how they addressed the challenge. Finally, the three-month interviews explored participants’ work experiences, including significant challenges as well as similarities and differences between capstone experiences and work, along with their evolving definitions of engineering. To answer the research question, we will employ thematic analysis to first identify emergent codes from the data set and subsequently synthesize those codes into themes. Preliminary review of the data suggests several potential themes that include overt experiences of gender discrimination, perceptions of (lack of) belonging or competence, and cultural shifts that may not have been effectively addresses in participants’ capstone courses or broader experiences.more » « less
-
One approach to teaching game design to students with a wide variety of disciplinary backgrounds is through team game projects that span multiple weeks, up to an entire term. However, open-ended, creative projects introduce a gamut of challenges to novice programmers. Our goal is to assist game design students with the planning stage of their projects. This paper describes our data collection process through three course interventions and student interviews, and subsequent analysis in which we learned students had difficulty expressing their creative vision and connecting the game mechanics to the intended player experience. We present these results as a step towards the goal of scaffolding the planning process for student game projects, supporting more creative ideas, clearer communication among team members, and a stronger understanding of human-centered design in software development.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18260/1-2--32060
