Abstract There are three approaches to studying designers – through their cognitive profile, design behaviors, and design artifacts (e.g., quality). However, past work has rarely considered all three data domains together. Here we introduce and describe a framework for a comprehensive approach to engineering design, and discuss how the insights may benefit engineering design research and education. To demonstrate the proposed framework, we conducted an empirical study with a solar energy system design problem. Forty-six engineering students engaged in a week-long computer-aided design challenge that assessed their design behavior and artifacts, and completed a set of psychological tests to measure cognitive competencies. Using a machine learning approach consisting of k-means, hierarchical, and spectral clustering, designers were grouped by similarities on the psychological tests. Significant differences were revealed between designer groups in their sequential design behavior, suggesting that a designer's cognitive profile is related to how they engage in the design process.
more »
« less
This content will become publicly available on May 23, 2025
Beyond analytics: Using computer‐aided methods in educational research to extend qualitative data analysis
This study proposes and demonstrates how computer‐aided methods can be used to extend qualitative data analysis by quantifying qualitative data, and then through exploration, categorization, grouping, and validation. Computer‐aided approaches to inquiry have gained important ground in educational research, mostly through data analytics and large data set processing. We argue that qualitative data analysis methods can also be supported and extended by computer‐aided methods. In particular, we posit that computing capacities rationally applied can expand the innate human ability to recognize patterns and group qualitative information based on similarities. We propose a principled approach to using machine learning in qualitative education research based on the three interrelated elements of the assessment triangle: cognition, observation, and interpretation. Through the lens of the assessment triangle, the study presents three examples of qualitative studies in engineering education that have used computer‐aided methods for visualization and grouping. The first study focuses on characterizing students' written explanations of programming code, using tile plots and hierarchical clustering with binary distances to identify the different approaches that students used to self‐explain. The second study looks into students' modeling and simulation process and elicits the types of knowledge that they used in each step through a think‐aloud protocol. For this purpose, we used a bubble plot and a k‐means clustering algorithm. The third and final study explores engineering faculty's conceptions of teaching, using data from semi‐structured interviews. We grouped these conceptions based on coding similarities, using Jaccard's similarity coefficient, and visualized them using a treemap. We conclude this manuscript by discussing some implications for engineering education qualitative research.
more »
« less
- Award ID(s):
- 2219271
- PAR ID:
- 10521448
- Publisher / Repository:
- Wiley
- Date Published:
- Journal Name:
- Computer Applications in Engineering Education
- ISSN:
- 1061-3773
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This is a research study that investigates the range of conceptions of prototyping in engineering design courses through exploring the conceptions and implementations from the instructors’ perspective. Prototyping is certainly an activity central to engineering design. The context of prototyping to support engineering education and practice has a range of implementations in an undergraduate engineering curriculum, from first-year engineering to capstone engineering design experiences. Understanding faculty conceptions’ of the reason, purpose, and place of prototyping can help illustrate how teaching and learning of the engineering design process is realistically implemented across a curriculum and how students are prepared for work practice. We seek to understand, and consequently improve, engineering design teaching and learning, through transformations of practice that are based on engineering education research. In this exploratory study, we interviewed three faculty members who teach engineering design in project-based learning courses across the curriculum of an undergraduate engineering program. This builds on related work done by the authors that previously investigated undergraduate engineering students’ conceptions of prototyping activities and process. With our instructor participants, a similar interview protocol was followed through semi-structured qualitative interviews. Data analysis has been undertaken through an emerging thematic analysis of these interview transcripts. Early findings characterize the focus on teaching the design process; the kind of feedback that the educators provide on students’ prototypes; students’ behavior while working on design projects; and educators’ perspectives on the design course. Understanding faculty conceptions with students’ conceptions of prototyping can shed light on the efficacy of using prototyping as an authentic experience in design teaching and learning. In project-based learning courses, particular issues of authenticity and assessment are under consideration, especially across the curriculum. More specifically, “proportions of problems” inform “problem solving” as one of the key characteristics in design thinking, teaching and learning. More attention to prototyping as part of the study of problem-solving processes can be useful to enhance understanding of the impact of instructional design. Challenges for teaching engineering design exist, and may be due to difficulties in framing design problems, recognizing what expertise students possess, and assessing their expertise to help them reach their goals, all at an appropriate place and ambiguity with student learning goals. Initial findings show that prototyping activities can help students become more reflective on their design. Scaffolded activities in prototyping can support self-regulated learning by students. The range of support and facilities, such as campus makerspaces, may also help students and instructors alike develop industry-ready engineering students.more » « less
-
null (Ed.)This NSF EAGER research paper investigates how undergraduate STEM and engineering students’ learning trajectories evolve over time, from 1st to senior year, along a novice to expert spectrum. We borrow the idea of “learning trajectories” from mathematics education that can paint the evolution of students’ knowledge and skills over time over a set of learning experiences. Curricula for undergraduate engineering programs can reflect an intended pathway of knowledge construction within a discipline. We intend our study of individual students within undergraduate STEM and engineering programs can highlight how this may happen in situ and how it may be similar or might differ from a given, prescribed programs of study among disciplines. We use a theoretical framework based in adaptive expertise and design thinking adaptive expertise to develop a design learning continuum further. Envisioned routes through disciplinary undergraduate curricula and student conceptions of their design process are explored through qualitative, semi-structured interviews with undergraduate 1st year and senior year students across STEM, engineering and non-STEM field such as computer science, mechanical engineering, general engineering, mathematics, science, English, and art. We also conduct similar interviews with faculty in these fields who are responsible and knowledgeable for undergraduate programs about their perceived benefits for the structure of their program’s curriculum. Additional information is collected from noticing the organizational and pedagogical structures of the relative undergraduate curriculum. Initial findings/outcomes suggest that traditions to knowledge construction both differ across disciplinary approaches and have similarities across non-obvious disciplinary relationships. Faculty have a firm understanding of how one class chains from one to another; students have less of a field of view for how mindful chunks of knowledge combine together.more » « less
-
This Work-in-Progress paper in the Research Category explores the unique challenges and opportunities of interdisciplinary education in computational modeling for life sciences student researchers at emerging research institutions (ERIs), specifically in predominantly undergraduate institutions (PUIs), and minority serving institutions (MSIs). Engineering approaches such as computational modeling have underappreciated potential for capacity building for the biomedical research enterprises of ERIs. We perform a bibliometric analysis to assess the prevailing use of computational modeling in life sciences research at MSIs, and PUIs. Additionally, we apply Social and Cognitive Theory to identify unique attitudinal, social and structural barriers for student researchers in learning and using computational modeling approaches at each of these types of institutions. Specifically, we use quantitative retrospective pre- and post-survey data and qualitative interviews of students who have attended a short-format computational modeling training course. We supplement these data with qualitative interviews of the students' faculty sponsors. Upon completion, this study will provide deeper understanding of issues related to computer science and engineering education at non-Research I institutions.more » « less
-
null (Ed.)As technology advances, data-driven work is becoming increasingly important across all disciplines. Data science is an emerging field that encompasses a large array of topics including data collection, data preprocessing, data visualization, and data analysis using statistical and machine learning methods. As undergraduates enter the workforce in the future, they will need to “benefit from a fundamental awareness of and competence in data science”[9]. This project has formed a research-practice partnership that brings together STEM+C instructors and researchers from three universities and education research and consulting groups. We aim to use high-frequency monitoring data collected from real-world systems to develop and implement an interdisciplinary approach to enable undergraduate students to develop an understanding of data science concepts through individual STEM disciplines that include engineering, computer science, environmental science, and biology. In this paper, we perform an initial exploratory analysis on how data science topics are introduced into the different courses, with the ultimate goal of understanding how instructional modules and accompanying assessments can be developed for multidisciplinary use. We analyze information collected from instructor interviews and surveys, student surveys, and assessments from five undergraduate courses (243 students) at the three universities to understand aspects of data science curricula that are common across disciplines. Using a qualitative approach, we find commonalities in data science instruction and assessment components across the disciplines. This includes topical content, data sources, pedagogical approaches, and assessment design. Preliminary analyses of instructor interviews also suggest factors that affect the content taught and the assessment material across the five courses. These factors include class size, students’ year of study, students’ reasons for taking class, and students’ background expertise and knowledge. These findings indicate the challenges in developing data modules for multidisciplinary use. We hope that the analysis and reflections on our initial offerings have improved our understanding of these challenges, and how we may address them when designing future data science teaching modules. These are the first steps in a design-based approach to developing data science modules that may be offered across multiple courses.more » « less