skip to main content


Title: A Study on the Impact of a Sketch-based Tutoring System in Statics Instruction
Large class sizes in engineering programs often prevent instructors from providing detailed and meaningful feedback to students on their homework problems. While the literature shows that frequent and immediate formative feedback has several benefits in terms of knowledge gain and academic motivation, several instructors struggle to provide any feedback. Motivated by this inability, a sketch-based virtual tutoring system, named Mechanix, has been developed and implemented. Mechanix lets the students to sketch their freebody diagram on a virtual interface and the process involved is very close to using a pencil and paper. The system provides real-time feedback on the accuracy of their Freebody diagrams and the solution to the problem. This paper reports the implementation of Mechanix at two large public universities in the United States – Georgia Institute of Technology and Texas State University. Mechanix is used to solve specific assignments from each school that involve the use of freebody diagrams. Pre- and post- concept inventories are used to measure the improvements in the conceptual understanding of the students. The results show that students who solve their homework using Mechanix outperform their peers who do not in one school, whereas the results are similar across the two groups in the second school. The evaluation of the concept inventories shows that the students who used Mechanix has the same level of improvement in their conceptual knowledge compared to the control group.  more » « less
Award ID(s):
1725659
NSF-PAR ID:
10167629
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Proceedings of the 2020 ASEE Annual Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Large class sizes in engineering programs often prevent instructors from providing detailed and meaningful feedback to students on their homework problems. While the literature shows that frequent and immediate formative feedback has several benefits in terms of knowledge gain and academic motivation, several instructors struggle to provide any feedback. Motivated by this inability, a sketch-based virtual tutoring system, named Mechanix, has been developed and implemented. Mechanix lets the students to sketch their freebody diagram on a virtual interface and the process involved is very close to using a pencil and paper. The system provides real-time feedback on the accuracy of their Freebody diagrams and the solution to the problem. This paper reports the implementation of Mechanix at two large public universities in the United States – Georgia Institute of Technology and Texas State University. Mechanix is used to solve specific assignments from each school that involve the use of freebody diagrams. Pre- and post- concept inventories are used to measure the improvements in the conceptual understanding of the students. The results show that students who solve their homework using Mechanix outperform their peers who do not in one school, whereas the results are similar across the two groups in the second school. The evaluation of the concept inventories shows that the students who used Mechanix has the same level of improvement in their conceptual knowledge compared to the control group. 
    more » « less
  2. Large class sizes in engineering programs often prevent instructors from providing detailed and meaningful feedback to students on their homework problems. While the literature shows that frequent and immediate formative feedback has several benefits in terms of knowledge gain and academic motivation, several instructors struggle to provide any feedback. Motivated by this inability, a sketch-based virtual tutoring system, named Mechanix, has been developed and implemented. Mechanix lets the students to sketch their freebody diagram on a virtual interface and the process involved is very close to using a pencil and paper. The system provides real-time feedback on the accuracy of their Freebody diagrams and the solution to the problem. This paper reports the implementation of Mechanix at two large public universities in the United States – Georgia Institute of Technology and Texas State University. Mechanix is used to solve specific assignments from each school that involve the use of freebody diagrams. Pre- and post- concept inventories are used to measure the improvements in the conceptual understanding of the students. The results show that students who solve their homework using Mechanix outperform their peers who do not in one school, whereas the results are similar across the two groups in the second school. The evaluation of the concept inventories shows that the students who used Mechanix has the same level of improvement in their conceptual knowledge compared to the control group. 
    more » « less
  3. Introductory engineering courses within large universities often have annual enrollments exceeding several hundreds of students, while online classes have even larger enrollments. It is challenging to achieve differentiated instruction in classrooms with class sizes and student diversity of such great magnitude. In such classes, professors assess whether students have mastered a concept through multiple-choice questions, marking answers as right or wrong with little feedback, or using online text-only systems. However, in these scenarios the feedback is of a mostly binary nature (right or wrong) with limited constructive feedback to scaffold learning. A growing concern among engineering educators is that students are losing both the critical skill of sketched diagrams and the ability to take a real system and reduce it to an accurate but simplified free-body diagram (FBD). A sketch-recognition based tutoring system, called Mechanix, allows students to hand-draw solutions just as they would with pencil and paper, while also providing iterative real-time personalized feedback. Sketch recognition algorithms use artificial intelligence to identify the shapes, their relationships, and other features of the sketched student drawing. Other AI algorithms then determine if and why a student’s work is incorrect, enabling the tutoring system to return immediate and iterative personalized feedback facilitating student learning that is otherwise not possible in large classes. To observe the effectiveness of this system, it has been implemented into various courses at three universities, with two additional universities planning to use the system within the next year. Student knowledge is measured using Concept Inventories based in both Physics and Statics, common exam questions, and assignments turned in for class. Preliminary results using Mechanix, a sketch-based statics tutoring system built at Texas A&M University, suggest that a sketch-based tutoring system increases homework motivation in struggling students and is as effective as paper-and-pencil-based homework for teaching method of joints truss analysis. In focus groups, students believed the system enhanced their learning and increased engagement. Keywords: sketch recognition; intelligent user interfaces; physics education; engineering education 
    more » « less
  4. In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developed an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system. 
    more » « less
  5. In online or large in-person course sections, instructors often adopt an online homework tool to alleviate the burden of grading. While these systems can quickly tell students whether they got a problem correct for a multiple-choice or numeric answer, they are unable to provide feedback on students’ free body diagrams. As the process of sketching a free body diagram correctly is a foundational skill to solving engineering problems, the loss of feedback to the students in this area is a detriment to students. To address the need for rapid feedback on students’ free body diagram sketching, the research team developed an online, sketch-recognition system called Mechanix. This system allows students to sketch free body diagrams, including for trusses, and receive instant feedback on their sketches. The sketching feedback is ungraded. After the students have a correct sketch, they are then able to enter in the numeric answers for the problem and submit those for a grade. Thereby, the platform offers the grading convenience of other online homework systems but also helps the students develop their free body diagram sketching skills. To assess the efficacy of this experimental system, standard concept inventories were administered pre- and post-semester for both experimental and control groups. The unfamiliarity or difficulty of some advanced problems in the Statics Concept Inventory, however, appeared to discourage students, and many would stop putting in any effort after a few problems that were especially challenging to solve. This effect was especially pronounced with the Construction majors versus the Mechanical Engineering majors in the test group. To address this tendency and therefore collect more complete pre- and post-semester concept inventory data, the research group worked on reordering the Statics Concept Inventory questions from more familiar to more challenging, based upon the past performance of the initial students taking the survey. This paper describes the process and results of the effort to reorder this instrument in order to increase Construction student participation and, therefore, the researchers’ ability to measure the impact of the Mechanix system. 
    more » « less