Computer-aided design (CAD) programs are essential to engineering as they allow for better designs through low-cost iterations. While CAD programs are typically taught to undergraduate students as a job skill, such software can also help students learn engineering concepts. A current limitation of CAD programs (even those that are specifically designed for educational purposes) is that they are not capable of providing automated real-time help to students. To encourage CAD programs to build in assistance to students, we used data generated from students using a free, open-source CAD software called Aladdin to demonstrate how student data combined with machine learning techniques can predict how well a particular student will perform in a design task. We challenged students to design a house that consumed zero net energy as part of an introductory engineering technology undergraduate course. Using data from 128 students, along with the scikit-learn Python machine learning library, we tested our models using both total counts of design actions and sequences of design actions as inputs. We found that our models using early design sequence actions are particularly valuable for prediction. Our logistic regression model achieved a >60% chance of predicting if a student would succeed in designing a zero net energy house. Our results suggest that it would be feasible for Aladdin to provide useful feedback to students when they are approximately halfway through their design. Further improvements to these models could lead to earlier predictions and thus provide students feedback sooner to enhance their learning.
more »
« less
Automatic Assessment of Students’ Engineering Design Performance Using a Bayesian Network Model
Integrating engineering design into K-12 curricula is increasingly important as engineering has been incorporated into many STEM education standards. However, the ill-structured and open-ended nature of engineering design makes it difficult for an instructor to keep track of the design processes of all students simultaneously and provide personalized feedback on a timely basis. This study proposes a Bayesian network model to dynamically and automatically assess students’ engagement with engineering design tasks and to support formative feedback. Specifically, we applied a Bayesian network to 111 ninth-grade students’ process data logged by a computer-aided design software program that students used to solve an engineering design challenge. Evidence was extracted from the log files and fed into the Bayesian network to perform inferential reasoning and provide a barometer of their performance in the form of posterior probabilities. Results showed that the Bayesian network model was competent at predicting a student’s task performance. It performed well in both identifying students of a particular group (recall) and ensuring identified students were correctly labeled (precision). This study also suggests that Bayesian networks can be used to pinpoint a student’s strengths and weaknesses for applying relevant science knowledge to engineering design tasks. Future work of implementing this tool within the computer-aided design software will provide instructors a powerful tool to facilitate engineering design through automatically generating personalized feedback to students in real time.
more »
« less
- Award ID(s):
- 1503196
- NSF-PAR ID:
- 10193602
- Date Published:
- Journal Name:
- Journal of Educational Computing Research
- ISSN:
- 0735-6331
- Page Range / eLocation ID:
- 073563312096042
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Computer-aided Design for Manufacturing (DFM) systems play an essential role in reducing the time taken for product development by providing manufacturability feedback to the designer before the manufacturing phase. Traditionally, DFM rules are hand-crafted and used to accelerate the engineering product design process by integrating manufacturability analysis during design. Recently, the feasibility of using a machine learning-based DFM tool in intelligently applying the DFM rules have been studied. These tools use a voxelized representation of the design and then use a 3D-Convolutional Neural Network (3D-CNN), to provide manufacturability feedback. Although these frameworks work effectively, there are some limitations to the voxelized representation of the design. In this paper, we introduce a new representation of the computer-aided design (CAD) model using orthogonal distance fields (ODF). We provide a GPU-accelerated algorithm to convert standard boundary representation (B-rep) CAD models into ODF representation. Using the ODF representation, we build a machine learning framework, similar to earlier approaches, to create a machine learning-based DFM system to provide manufacturability feedback. As proof of concept, we apply this framework to assess the manufacturability of drilled holes. The framework has an accuracy of more than 84% correctly classifying the manufacturable and non-manufacturable models using the new representation.
-
This NSF Grantees poster discusses an early phase Revolutionizing Engineering Departments (RED) project which is designed to address preparing engineering students to address large scale societal problems, the solutions of which integrate multiple disciplinary perspectives. These types of problems are often termed “convergent problems”. The idea of convergence captures how different domains of expertise contribute to solving a problem, but also the value of the network of connections between areas of knowledge that is built in undertaking such activities. While most existing efforts at convergence focus at the graduate and post-graduate levels, this project supports student development of capabilities to address convergent problems in an undergraduate disciplinary-based degree program in electrical and computer engineering. This poster discusses some of the challenges faced in implementing such learning including how to decouple engineering topics from societal concerns in ways that are relevant to undergraduate students yet retain aspects of convergence, negotiations between faculty on ways to balance discipline-specific skills with the breadth required for systemic understanding, and challenges in integrating relevant projects into courses with different faculty and instructional learning goals. One of the features of the project is that it builds on ideas from Communities of Transformation by basing activities on a coherent philosophical model that guides theories of change. The project has adopted Amartya Sen’s Development as Freedom or capabilities framework as the organizing philosophy. In this model the freedom for individuals to develop capabilities they value is viewed as both the means and end of development. The overarching goal of the project is then for students to build personalized frameworks based on their value systems which allow them to later address complex, convergent problems. Framework development by individual students is supported in the project through several activities: modifying grading practices to provide detailed feedback on skills that support convergence, eliciting self-narratives from students about their pathways through courses and projects with the goal of developing reflection, and carefully integrating educational software solutions that can reduce some aspects of faculty workload which is hypothesized to enable faculty to focus efforts on integrating convergent projects throughout the curriculum. The poster will present initial results on the interventions to the program including grading, software integration, projects, and narratives. The work presented will also cover an ethnographic study of faculty practices which serves as an early-stage baseline to calibrate longer-term changes.more » « less
-
Computing theory is often perceived as challenging by students, and verifying the correctness of a student’s automaton or grammar is time-consuming for instructors. Aiming to provide benefits to both students and instructors, we designed an automated feedback tool for assignments where students construct automata or grammars. Our tool, built as an extension to the widely popular JFLAP software, determines if a submission is correct, and for incorrect submissions it provides a “witness” string demonstrating the incorrectness. We studied the usage and benefits of our tool in two terms, Fall 2019 and Spring 2021. Each term, students in one section of the Introduction to Computer Science Theory course were required to use our tool for sample homework questions targeting DFAs, NFAs, RegExs, CFGs, and PDAs. In Fall 2019, this was a regular section of the course.We also collected comparison data from another section that did not use our tool but had the same instructor and homework assignments. In Spring 2021, a smaller honors section provided the perspective from this demographic. Overall, students who used the tool reported that it helped them to not only solve the homework questions (and they performed better than the comparison group) but also to better understand the underlying theory concept. They were engaged with the tool: almost all persisted with their attempts until their submission was correct despite not being able to random walk to a solution. This indicates that witness feedback, a succinct explanation of incorrectness, is effective. Additionally, it assisted instructors with assignment grading.more » « less
-
null (Ed.)Abstract Insufficient engineering analysis is a common weakness of student capstone design projects. Efforts made earlier in a curriculum to introduce analysis techniques should improve student confidence in applying these important skills toward design. To address student shortcomings in design, we implemented a new design project assignment for second-year undergraduate biomedical engineering students. The project involves the iterative design of a fracture fixation plate and is part of a broader effort to integrate relevant hands-on projects throughout our curriculum. Students are tasked with (1) using computer-aided design (CAD) software to make design changes to a fixation plate, (2) creating and executing finite element models to assess performance after each change, (3) iterating through three design changes, and (4) performing mechanical testing of the final device to verify model results. Quantitative and qualitative methods were used to assess student knowledge, confidence, and achievement in design. Students exhibited design knowledge gains and cognizance of prior coursework knowledge integration into their designs. Further, students self-reported confidence gains in approaching design, working with hardware and software, and communicating results. Finally, student self-assessments exceeded instructor assessment of student design reports, indicating that students have significant room for growth as they progress through the curriculum. Beyond the gains observed in design knowledge, confidence, and achievement, the fracture fixation project described here builds student experience with CAD, finite element analysis, 3D printing, mechanical testing, and design communication. These skills contribute to the growing toolbox that students ultimately bring to capstone design.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1177/0735633120960422
