skip to main content


Title: Tools for Assessing the Creative Person, Process, and Product in Engineering Education
WIP: Assessing the Creative Person, Process and Product in Engineering Education This evidence-based practice paper provides guidance in assessing creativity in engineering education. In the last decade, a number of vision statements on the future of engineering education (e.g. Educating the Engineer of 2020, the ASCE Body of Knowledge) point to the fact that creativity is essential to engineering innovation; it is regarded as an important attribute in the education of engineers in order to meet the most urgent national challenges and to drive economic growth in the new millennium. Yet studies suggest that engineering students’ creative skills are being left underdeveloped or diminish over the course of their studies, or worse, that students who consider themselves to be creative are being driven away from engineering as a chosen field. On the surface, creativity skills are perceived as difficult to utilize in the engineering classroom, primarily due to the didactic nature of science and engineering instruction. Assessing the product of open ended or ill-structured assignments remains a difficult task as well. This study examines available assessments for creativity that are founded in three of the Four Ps of creativity: person, process, product (the fourth P, press, is not considered in this work.) The intent is to identify verified metrics that can be used to quantify creativity with a particular look to whether the metrics are appropriate for creativity, particularly as they pertain to the science and engineering domains. These metrics are examined for applicability to science and engineering, ease of administration and completion, expertise required to score, cost to administer, and time required to administer. Rather than determining the “best” metrics, this examination will provide guidelines for engineering educators and researchers interested in creativity for selecting appropriate metrics to be used in classrooms and research studies based on metric attributes.  more » « less
Award ID(s):
1712195
NSF-PAR ID:
10104575
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ASEE Annual Conference proceedings
ISSN:
1524-4644
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Design for manufacturing provides engineers with a structure for accommodating the limitations of traditional manufacturing processes. However, little emphasis is typically given to the capabilities of processes that enable novel design geometries, which are often a point of focus when designing products to be made with additive manufacturing (AM) technologies. In addition, limited research has been conducted to understand how knowledge of both the capabilities (i.e., opportunistic) and limitations (i.e., restrictive aspects) of AM affects design outcomes. This study aims to address this gap by investigating the effect of no, restrictive, and both, opportunistic and restrictive (dual) design for additive manufacturing (DfAM) education on engineering students’ creative process. Based on the componential model of creativity [1], these effects were measured through changes in (1) motivation and interest in AM, (2) DfAM self-efficacy, and (3) the emphasis given to DfAM in the design process. These metrics were chosen as they represent the cognitive components of ‘task-motivation’ and ‘domain relevant skills’, which in turn influence the learning and usage of domain knowledge in creative production. The results of the study show that while the short (45 minute) DfAM intervention did not significantly change student motivation and interest towards AM, students showed high levels of motivation and interest towards AM, before the intervention. Teaching students different aspects of DfAM also resulted in an increase in their self-efficacy in the respective topics. However, despite showing a greater increase in self-efficacy in their respective areas of training, the students did not show differences in the emphasis they gave to these DfAM concepts, in the design process. Further, students from all three education groups showed higher use of restrictive concepts, in comparison to opportunistic DfAM. 
    more » « less
  2. null (Ed.)
    Investigations of creativity have been an intriguing topic for a long time, but assessing creativity is extremely complex. Creativity is a cornerstone of engineering disciplines, so understanding creativity and how to enhance creative abilities through engineering education has received substantial attention. Fields outside of engineering are no stranger to neuro-investigations of creativity and although some neuro-response studies have been conducted to understand creativity in engineering, these studies need to map the engineering design and concept generation processes better. Using neuroimaging techniques alongside engineering design and concept generation processes is necessary for understanding how to improve creative idea generation and creativity studies in engineering. In this paper, a survey is provided of the literature for the different neurological approaches that have been used to study the engineering design process and creative processes. Also presented are proposed strategies to apply these neurological approaches to engineering design to understand the creative process in greater detail. Furthermore, results from a pilot study investigating neuro-responses of engineers are presented. 
    more » « less
  3. Previous studies have convincingly shown that traditional, content-centered, and didactic teaching methods are not effective for developing a deep understanding and knowledge transfer. Nor does it adequately address the development of critical problem-solving skills. Active and collaborative instruction, coupled with effective means to encourage student engagement, invariably leads to better student learning outcomes irrespective of academic discipline. Despite these findings, the existing construction engineering programs, for the most part, consist of a series of fragmented courses that mainly focus on procedural skills rather than on the fundamental and conceptual knowledge that helps students become innovative problem-solvers. In addition, these courses are heavily dependent on traditional lecture-based teaching methods focused on well-structured and closed-ended problems that prepare students to plug variables into equations to get the answer. Existing programs rarely offer a systematic approach to allow students to develop a deep understanding of the engineering core concepts and discover systematic solutions for fundamental problems. Without properly understanding these core concepts, contextualized in domain-specific settings, students are not able to develop a holistic view that will help them to recognize the big picture and think outside the box to come up with creative solutions for arising problems. The long history of empirical learning in the field of construction engineering shows the significant potential of cognitive development through direct experience and reflection on what works in particular situations. Of course, the complex nature of the construction industry in the twenty-first century cannot afford an education through trial and error in the real environment. However, recent advances in computer science can help educators develop virtual environments and gamification platforms that allow students to explore various scenarios and learn from their experiences. This study aims to address this need by assessing the effectiveness of guided active exploration in a digital game environment on students’ ability to discover systematic solutions for fundamental problems in construction engineering. To address this objective, through a research project funded by the NSF Division of Engineering Education and Centers (EEC), we designed and developed a scenario-based interactive digital game, called Zebel, to guide students solve fundamental problems in construction scheduling. The proposed gamified pedagogical approach was designed based on the Constructivism learning theory and a framework that consists of six essential elements: (1) modeling; (2) reflection; (3) strategy formation; (4) scaffolded exploration; (5) debriefing; and (6) articulation. We also designed a series of pre- and post-assessment instruments for empirical data collection to assess the effectiveness of the proposed approach. The proposed gamified method was implemented in a graduate-level construction planning and scheduling course. The outcomes indicated that students with no prior knowledge of construction scheduling methods were able to discover systematic solutions for fundamental scheduling problems through their experience with the proposed gamified learning method. 
    more » « less
  4. Who and by what means do we ensure that engineering education evolves to meet the ever changing needs of our society? This and other papers presented by our research team at this conference offer our initial set of findings from an NSF sponsored collaborative study on engineering education reform. Organized around the notion of higher education governance and the practice of educational reform, our open-ended study is based on conducting semi-structured interviews at over three dozen universities and engineering professional societies and organizations, along with a handful of scholars engaged in engineering education research. Organized as a multi-site, multi-scale study, our goal is to document differences in perspectives and interest the exist across organizational levels and institutions, and to describe the coordination that occurs (or fails to occur) in engineering education given the distributed structure of the engineering profession. This paper offers for all engineering educators and administrators a qualitative and retrospective analysis of ABET EC 2000 and its implementation. The paper opens with a historical background on the Engineers Council for Professional Development (ECPD) and engineering accreditation; the rise of quantitative standards during the 1950s as a result of the push to implement an engineering science curriculum appropriate to the Cold War era; EC 2000 and its call for greater emphasis on professional skill sets amidst concerns about US manufacturing productivity and national competitiveness; the development of outcomes assessment and its implementation; and the successive negotiations about assessment practice and the training of both of program evaluators and assessment coordinators for the degree programs undergoing evaluation. It was these negotiations and the evolving practice of assessment that resulted in the latest set of changes in ABET engineering accreditation criteria (“1-7” versus “a-k”). To provide an insight into the origins of EC 2000, the “Gang of Six,” consisting of a group of individuals loyal to ABET who used the pressure exerted by external organizations, along with a shared rhetoric of national competitiveness to forge a common vision organized around the expanded emphasis on professional skill sets. It was also significant that the Gang of Six was aware of the fact that the regional accreditation agencies were already contemplating a shift towards outcomes assessment; several also had a background in industrial engineering. However, this resulted in an assessment protocol for EC 2000 that remained ambiguous about whether the stated learning outcomes (Criterion 3) was something faculty had to demonstrate for all of their students, or whether EC 2000’s main emphasis was continuous improvement. When it proved difficult to demonstrate learning outcomes on the part of all students, ABET itself began to place greater emphasis on total quality management and continuous process improvement (TQM/CPI). This gave institutions an opening to begin using increasingly limited and proximate measures for the “a-k” student outcomes as evidence of effort and improvement. In what social scientific terms would be described as “tactical” resistance to perceived oppressive structures, this enabled ABET coordinators and the faculty in charge of degree programs, many of whom had their own internal improvement processes, to begin referring to the a-k criteria as “difficult to achieve” and “ambiguous,” which they sometimes were. Inconsistencies in evaluation outcomes enabled those most discontented with the a-k student outcomes to use ABET’s own organizational processes to drive the latest revisions to EAC accreditation criteria, although the organization’s own process for member and stakeholder input ultimately restored much of the professional skill sets found in the original EC 2000 criteria. Other refinements were also made to the standard, including a new emphasis on diversity. This said, many within our interview population believe that EC 2000 had already achieved much of the changes it set out to achieve, especially with regards to broader professional skills such as communication, teamwork, and design. Regular faculty review of curricula is now also a more routine part of the engineering education landscape. While programs vary in their engagement with ABET, there are many who are skeptical about whether the new criteria will produce further improvements to their programs, with many arguing that their own internal processes are now the primary drivers for change. 
    more » « less
  5. This Complete Research paper investigates the holistic assessment of creativity in design solutions in engineering education. Design is a key element in contemporary engineering education, given the emphasis on its development through the ABET criteria. As such, design projects play a central role in many first-year engineering courses. Creativity is a vital component of design capability which can influence design performance; however, it is difficult to measure through traditional assessment rubrics and holistic assessment approaches may be more suitable to assess creativity of design solutions. One such holistic assessment approach is Adaptive Comparative Judgement (ACJ). In this system, student designs are presented to judges in pairs, and they are asked to select the item of work that they deem to have demonstrated the greatest level of a specific criterion or set of criteria. Each judge is asked to make multiple judgements where the work they are presented with is adaptively paired in order to create a ranked order of all items in the sample. The use of this assessment approach in technology education has demonstrated high levels of reliability among judges (~0.9) irrespective of whether the judges are students or faculty. This research aimed to investigate the use of ACJ to holistically assess the creativity of first-year engineering students design solutions. The research also sought to explore the differences, if any, that would exist between the rank order produced by first-year engineering students and the faculty who regularly teach first-year students. Forty-six first-year engineering students and 23 faculty participated in this research. A separate ACJ session was carried out with each of these groups; however, both groups were asked to assess the same items of work. Participants were instructed to assess the creativity of 101 solutions to a design task, a “Ping Pong problem,” where undergraduate engineering students had been asked to design a ping pong ball launcher to meet specific criteria. In both ACJ sessions each item of work was included in at least 11 pairwise comparisons, with the maximum number of comparisons for a single item being 29 in the faculty ACJ session and 50 in the student ACJ session. The data from the ACJ sessions were analyzed to determine the reliability of using ACJ to assess creativity of design solutions in first-year engineering education, and to explore whether the rankings produced from the first-year engineering students ACJ session differed significantly from those of the faculty. The results indicate a reasonably high level of reliability in both sessions as measured by the Scale Separation Reliability (SSR) coefficient, SSRfaculty = 0.65 ± 0.02, SSRstudents = 0.71 ± 0.02. Further a strong correlation was observed between the ACJ ranks produced by the students and faculty both when considered in terms of the relative differences between items of work, r = .533, p < .001, and their absolute rank position, σ = .553, p < .001. These findings indicate that ACJ is a promising tool for holistically assessing design solutions in engineering education. Additionally, given the strong correlation between ranks of students and faculty, ACJ could be used to include students in their own assessment to reduce the faculty grading burden or to develop a shared construct of capability which could increase the alignment of teaching and learning. 
    more » « less