Spatial visualization is the ability to mentally represent and manipulate two-dimensional (2D) and three-dimensional (3D) figures. These skills have been correlated with increased grade point averages in STEM-related fields including: math, engineering, computer programming, and science [ Sorby, 2009 ]. Spatial visualization skills are learnable [Sorby, 2009], however, most students do not receive formal instruction as spatial visualization concepts are not a focus of contemporary K-12 curriculum. Research has shown that a single unit spatial visualization course has increased retention in STEM majors, especially for women and other underrepresented minorities [Hill, 2010].
Typical spatial visualization training is a combination of multiple-choice and freehand sketching assignments [Sorby, 2009]. Multiple-choice based instruction is a more manageable system for instructors to incorporate into the classroom because of its ease of grading, ability to provide students with relatively quick feedback, and its ability to be administered both traditionally with paper-and-pencil and incorporated into an online learning management systems (LMS). However, multiple-choice assignments are subject to process of elimination and are not an accurate representation of how these skills would be applied in real-world application. Spatial visualization training using freehand sketching exercises has shown greater benefits over solely multiple-choice assignments for developing students' spatial visualization reasoning [Sorby, 2009], as well as improving students' technical communication ability and creativity [Do, 1996]. However, incorporating sketching assignments on paper requires instructor time for manually grading, and students do not receive immediate feedback on their performance.
To bridge the gap between the more practical multiple choice instruction and the more valuable freehand sketching training, an iPad application, called Spatial Vis, was developed in 2012 to address these concerns [Delson, 2015]. Spatial Vis automatically grades freehand sketching assignments, removing the need for human grading, providing instantaneous feedback to students, creating a more manageable solution for teachers and a more beneficial solution for students.
The initial pilot in 2014 yielded interesting results, in addition to freehand sketching playing a significant role in the development of spatial skill, the element of persistence also contributes [Delson, 2015]. The use of a touchscreen with stylus or finger sketching has led to the development of an alternative to multiple-choice learning technology. This paper will discuss application modifications focused on increasing student engagement and persistence made after the 2014 pilot that have led to the 2017 pilot results. The enhancements include; user interface modifications, assignment design modifications, and the introduction of gamification.
The Spatial Vis application has several key components, including; the assignment window, the sketching window, and the toolbar (see Figure 1). The assignment window describes the task students are to complete. The sketching window is where students draw their solution to the assignment. The sketching window and the assignment window each have a reference dot to prompt the student to know where in the sketching window they need to draw their solution in order for the grading algorithm to correctly score their solution. The toolbar features an eraser, pencil, and help button. When the user selects the save button in the top right corner, the grading algorithm is initiated. The grading algorithm produces a pop-up window that gives the student immediate feedback if their solution is correct or not. If the solution is correct, the student moves on to the next assignment. If the solution is incorrect, the student can either retry the assignment or peek at the solution. If the student chooses to peek at the solution, the sketching window will show which lines are correct or incorrect, and which lines are missing from their sketch for it to be graded as correct.
The 2014 pilot course was a one-unit credit, pass/no pass course at the University of California San Diego consisting of 52 students who met once a week for ten weeks. Prior to being introduced to the Spatial Vis application, students were given the Purdue Spatial Visualization Test: Rotations (PSVT:R) as a pre-assessment of their incoming spatial visualization skills. The PSVT:R is a timed, twenty minute assessment consisting of thirty multiple choice questions of three-dimensional rotation tasks (see Figure 2). The students then went through the one-unit course and completed the Spatial Vis application. Upon the conclusion of the course, the PSVT:R was administered as post-assessment allowing us to compare their scores. The primary focus was on students who scored 70% and below on the pre-test because they were considered most at risk for dropping out of a STEM major. Out of the 52 students that took the one-unit course, 13 of them scored 70% and below on the pre-test. Those 13 students were the focus of the analysis.
The results showed that the Low Pre-Test Group of students had an average test improvement of 18%. However, Table 1 shows a noticeable split between the individual students. 54% of the students had no or low improvement, while the other 46% of students had significant improvement of a 10% test improvement or more. Possible reasons why some students improved while others did not improve were further investigated. Multiple performance metrics were compared and the largest difference between the two groups was the amount of times they tried again without using the peek feature. When a student got an assignment incorrect, they were given two options, they could retry the assignment or peek at the solution. The students with significant improvement would retry assignments without peeking 74% more often than those who had low or no improvement [Delson, 2015]. Accordingly, the percentage of time a student would retry assignments without peeking was used to quantify persistence.
The 2014 version of the application used assignments from a traditional paper and pencil spatial visualization training curriculum [Delson, 2015]. Since the 2014 pilot, digital and touchscreen affordances have been leveraged by making several design changes with the goal of increasing student engagement, persistence, and spatial visualization skill. These alterations include, enhancements to the user interface, assignment design modifications, and the introduction of gamification.
Several user interface changes were made between Spatial Vis 2014 and Spatial Vis 2017 (see Figure 3). One major change was the addition of color. The color makes the application seem more game-like and less of an assignment. The changes in color create a more fun and engaging atmosphere for students.
The assignment window was also separated from the sketching window so that a larger assignment could be created that would stay in view if students wanted to zoom-in on the sketching window. By giving the assignment window more area on the interface, additional instruction about an assignment could be added.
Another improvement was the change in grid structure. Initially, both the orthographic and isometric grids that were used to guide the students in making accurate, proportioned sketches were individual black dots. This grid type is common in traditional hand-sketching application because it does not overpower the student's pencil lines, allowing graders and students to easily see their sketches. The digital interface is not restricted by color so the lack of contrast between black print and grey pencil lines was no longer a concern, allowing for the use of a full halftoned grey grid paired with a contrasting blue digital pen color. This subtle change not only helps students improve their free-hand sketching ability by giving them lines to follow, but also makes their potential solution more visible in the high contrast color and lineweight between the light-grey background grid and their heavier blue sketched lines.
In terms of assignment redesign, moving to pen and touch technology has many affordances over traditional, analog pen-and-paper. According to the 2014 student survey, students found sketching on the iPad about the same to sketching on paper [Delson, 2015]. Because of the survey results and studies showing the value of sketching assignments over multiple-choice assignments, the amount of sketching assignments in the application was increased. Table 2 shows the breakdown of assignment types within the two versions of the application. In addition to having primarily hand-sketching assignments, another focus of the Spatial Vis 2017 application was for the application to be able to stand-alone and not need additional instruction. This is ideal because the spatial visualization skills of incoming students varies and this method allows students to work at their own pace. Also, eliminating the need for instructions creates a more flexible system which gives the instructors more freedom.
In order to achieve this, the assignments started very simple and gradually progressed in difficulty (see Figure 4). Assignments could not progress too gradually or students would lose interest. There were several criteria for making an easy assignment. Easy assignments consist of simple shapes, multiple reference dots, labels, or a multiple choice format. To make an assignment more difficult the shape would increase in complexity and aids were removed such as extra reference dots and labels.
Similar to the user interface overhaul, color was strategically added to aid students in their understanding of the foundational concepts each individual lesson teaches. For example, traditional approaches to teaching multiview (orthographic) projection includes surface and vertices numbering and/or lettering. The app simplified this concept by using multiple colored dots to help prompt students to recognize the relationship between 2D orthographic projection drawing they are to create of the 3D isometric pictorial drawing given in the assignment window (see Figure 5).
An intermediate level of help was also added (see Figure 6). For 2014, when students got an answer incorrect they were given the option to retry the assignment or peek at the solution. A hint feature was added. The hint tells the user which parts of their drawing are correct by highlighting them in green and removing the incorrect lines. If a student used a hint and most of their submission remained in the workspace then the student is close to the solution. Alternatively, if the student uses a hint and most of their submission disappears then they know they need to rethink the problem, ask for help, or possibly use a peek. The final three assignments of each lesson were considered "Test Assignments". These assignments were of moderate difficulty and the hint and peek features were disabled. This served as an incentive for students to learn what the chapter was teaching, rather than rely on the hint and peek features. They were aware that they would have to eventually complete three assignments without help features before they can move on to the next chapter.
Lastly, a gamification aspect to the application was added to encourage persistence. A goal was to have students retry often and limit their use of the hint and peek features, so stars were added as incentive. The maximum stars a student could receive on an assignment was 3 stars. Students were allowed to retry as many times as they would like without a penalty (see Figure 7 left). If a student used the hint feature on an assignment, then the maximum stars they could receive was 2 stars (see Figure 7 middle). If a student used a peek, then the maximum stars they could receive was 1 star (see Figure 7 right).
After the changes were made, another one-unit pilot course study was conducted at the University of California San Diego. Like the 2014 pilot, students met once a week for ten weeks. Before the students were introduced to the application, the students completed the same Purdue Spatial Visualization Test. They then completed the course featuring the Spatial Vis application and took the Purdue Spatial Visualization Test again so that their test scores could be compared. Because the focus of the 2014 pilot was on the students who scored 70% and below on the pre-test, students were evaluated that met the same criteria in 2017. In the class, 11 of the 32 students during this pilot tested into the Low Pre-Test Group.
The results showed that the Low Pre-Test Group of students had an average test improvement of 39%. When you look at the students individually (see Table 3), there is no longer a noticeable split between students with no or low improvement and students with significant improvement. In addition to using the pre-test and post-test method to evaluate student performance, students also completed a survey; 26 of the 32 students that enrolled in the course took a survey after completing the course. Overall, 92% of students report noticing an improvement in their spatial visualization skills as a result from taking the course and 96% of students would recommend the course to their peers.
The design changes made to the user interface following the 2014 pilot has led to an improvement in the test scores of those with low incoming spatial visualization skills in 2017. The addition of color and gamification helped create a "low stress, interesting, and fun" (2017 Pilot Student) experience. Additionally, the focus on the assignments building difficulty with the use of scaffolding and a clear assignment progression allowed for the the students to work independently; creating a more flexible and manageable spatial visualization training method for the classroom.
While the combination of the design changes has led to better test scores, it is not clear if one specific design change had a greater impact on the results than the others. In addition, these were relatively small studies with n=13 in 2014 and n=11 in 2017. Additional studies with larger numbers of students will be needed to validate the results. It is interesting to note that although the focus was on students with low pre-test scores, students from all pre-test groups felt they have improved over the duration of the course. When surveying the entire class, 92% of students (n=26) that took the course and survey reported a noticeable improvement in their spatial visualization skills with some saying they "feel more confident in their visualizing skills" (2017 Pilot Student).
In addition to the previously discussed design changes impacting the post-test scores of students, we plan on further looking at the 2017 student submission data to determine if there is a difference in the student persistence metrics between 2014 and 2017. We will also look closely at the student data metrics surrounding specific assignments, which will influence the assignment designs.
We plan to continue our research around Spatial Vis at the University of California San Diego Engineering classes as well as making the applications available to other universities, high schools, camps, and afterschool programs. The Spatial Vis App is currently being ported to iPhone, iPad, Android Tablets, Android Phones, Chromebooks, and Windows Computers.
Touchscreen devices provide a unique environment which allow for an electronic stand-alone method of teaching a skill that has previously been reliant on analog methods and human grading. Not only can spatial visualization exercises be digitized, but the unique affordances of the digital platform allow for color, animations, personalized feedback, and gamification to play a role in student learning and motivation. The flexibility of these devices allows the application to be used in a classroom, independently, or a hybrid setting.
A prior study had indicated that sketching assignments can be used to quantify student persistence, which in turn was tied to the effectiveness in learning. This study shows how changes in the user interface designed to increase persistence has led to increases from 46% to 82% in the number of students with significant post-test gains.