Numerous learners find it challenging to master concepts where abstract topics cannot be explicitly seen and experienced in everyday life. For this reason, different aids have been used to make these concepts tangible. In physics education, tools such as visual representations (e.g., graphs, diagrams, icons) and physical manipulatives (e.g., blocks, magnets, compass) are frequently used as teaching aids (Zacharia & Olympiou, 2011;Savinainen et al, 2015;Suyatna et al, 2017). One issue with the usage of multiple representations is that learners must understand and integrate them (Kohl et al, 2007;Ainsworth 2008). For instance, Ainsworth (2008) has found that to use different representations effectively, learners must understand the relationship between the representation and the concept they are learning and understand what information the representation is carrying. This is especially problematic when different representations are presented in different places at different times (Bujak et al, 2013). Accordingly, using hand gestures (e.g., deictic and iconic) to connect multiple concepts together can reduce cognitive load and improve students' learning (Alibali et al, 2012;Alibali et al, 2013).
Augmented reality (AR) is an emerging technology that has become increasingly common to visualize abstract scientific learning (Wu, 2013). Augmented reality provides multiple affordances that can be beneficial for learning (Radu, 2014;Bujak et al, 2013). For example, AR can help reduce the cognitive load in learners while they are learning with multiple representations, by linking abstract representations to concrete physical representation through spatial and temporal contiguity (Radu, 2014;Bujak et al, 2013). This affordance is especially suitable for physics education, where abstract concepts accompany physical models. In one study of using AR for Newtonian force concepts, Enyedy et al (2012) found that children ages 6-8 years old were able to learn force concepts when AR was incorporated into role play. Cai et al (2017) revealed that AR-based motion-sensing tools could help junior high school students learn concepts about magnetic fields more intuitively, and that students were able to retain the concepts longer. Those findings suggest that AR can be an interesting medium for supporting collaborative learning. In the section below, we describe the larger experiment we ran and how we qualitatively analyze eight pairs from it.
A larger study (Radu & Schneider, 2019) compared 60 pairs of participants (N=120) who interacted with an augmented physical model of an audio producing speaker (Figure 1). The pairs were assigned to either do the activity with or without seeing the augmented reality (AR) visuals. The NonAR condition participants were given a physical model, a compass, and informational posters on the wall (Figure 1, left), while the AR condition participants were given the same tools, and also various AR visuals in addition to what the NonAR participants were given (Figure 1, right). There were multiple representational tools to support participants in both conditions, with the main difference being that AR condition received visual representation aligned with the physical system, whereas the NonAR condition received representations not overlaid on the physical system.
For this paper we focused on analyzing video recordings of eight pairs (n=16), which were chosen according to two variables: AR versus NonAR; and overall learning gains: high learning gains versus low learning gains. This is a 2x2 design with two pairs of students in each condition. We found that participants in AR learned more about spatial structures (ex: shapes of magnetic fields) but learned less about non-visual concepts (ex: relationship between physical movement and electricity). Furthermore, AR groups had higher levels of engagement and improved perception of self-efficacy (Figure 2):
Figure 2. Group differences in relative learning gains in percentage (left); and overall attitudes in 5-point scale (right). Red= AR group, Green = NonAR group; whiskers show standard errors (for more information, please refer to (Radu & Schneider, 2019)).
In this paper we build upon this project by using the same dataset and qualitatively analyzing the learning and teaching behaviors of participants, comparing between AR-vs-NonAR conditions, by comparing groups that had high learning vs low learning. We are interested in understanding specifically why participants learned differently between the two conditions. Our three research questions are as follow:
Research Question 1: How does AR impact collaboration? Research Question 2: How does AR impact the use of external aids and gestures? Research Question 3: How does AR impact teaching behaviors?
This paper analyzes video recording of participants learning by using the system in the AR and NonAR conditions. During the activity two participants had to work together to complete a worksheet on electromagnetism; the worksheet asked questions such as "Where is the strongest magnetic field located?" and "Draw the shape of the magnetic field when a cup is closer versus further". At fixed times during the experience, the facilitator would ask questions to participants, such as "How is the direction and the strength of the electric current influencing the cup?", which were meant as provocation to think about various aspects of the system, but did not require explicit answering. We performed qualitative video analysis to understand participant behavior. A coding scheme was constructed through iterative bottom-up coding. The final coding scheme consisted of three main categories: (1) communication type; (2) aid provided for communication; and (3) method of communication. Research study videos from the eight participant sessions were split into 30 seconds time frames and assigned one or more codes. For interrater reliability, one rater coded 100% of the videos, while the other rater coded 20% of each video. The eight analyzed sessions ranged between 27-33 minutes. In total, 489 30-sec. segments were coded, accounting for 4 hours of video. Inter-rater reliability reached a Cohen kappa of 0.8, which implies substantial/almost perfect agreement. The coding scheme categories that were derived from the videos are as follows:
Communication Types: Describes the purpose of the participants' communication: exploring the system, teaching each other, chatting about irrelevant topics, or non-interacting (Table 1).
Aid Provided for Communication: If participants were using an aid while communicating, this code describes the type of tool or representational aid used: poster, compass, and system. The last category is coded separately for AR vs. NonAR groups because the system was merged with holographic representations for the AR group; this resulted in two codes for our coding scheme: using NonAR system (only applicable for NonAR participants, Figure 1 left), and using AR system (only applicable for AR participants, Figure 1, right).
Methods of Communication. If participants were using gestures or drawings as a method of communication, this code describes that method: deictic gesture, iconic gesture, and self-drawing. Deictic gesture is the type of method that is the easiest to produce, it is when a participant is pointing or using gesture to make the other participant shift his/her attention to a certain location (Roth, 2001). In contrary, iconic gesture and self-drawing has a higher representation level. Iconic gesture is a symbolic gesture when one participant is using his/her hand to mimic a visual, while self-drawing is when one participant is drawing on a paper to support his/her explanation.
Exploring At least one of the two participants discuss about the activity with a clear intention to interaction with the other participant. Also includes non-verbal interaction when the participants test out the system together.
P1: "Alright, let's see what changes when you pull it down to 10x. Anything change?" P2: " I think it's quite-er" P1: "Yeah, I agree, it is definitely quite-er"
Teaching At least one of the two participants tries to make the other participant understands a physics concept through explanation or clarification.
P1: "Look at the green lines over there. There are more lines inside and the magnetic field becomes bigger. The polarity is also changing north and south."
Both participants talk on a topic unrelated to the physics concept based on the activity.
P1: "Do you know a lot of physics?" P2: "I forgot most of it"
There is no explicit intention to interact between the two participants, but at least one participant is engaging with the learning activity.
One participant played with the system, while the other sit still.
Results from the larger study (Radu & Schneider, 2019) indicate that there are differences in learning and attitudes across the AR vs NonAR conditions (Figure 2). When examining the eight pairs from this current study, we found similar results, namely that participants in the AR condition had higher learning gains in topics involving spatial structures (ex: identifying shapes of magnetic fields), while the NonAR condition participants had higher gains in topics related to physical movement (ex: relationship of magnetic field vs. movement). Additionally, similar to the overall results, participants in the AR condition had a higher tendency than NonAR participants to believe that physics is easy after completing the study. Figure 3 indicates that the eight groups we selected are representative of our sample.
Figure 3. Pre-and post-assessment: Difference in the attitude (curiosity and whether physics is easy) and learning scores on different areas (relationship: electricity-magnetic fields, electricity-movement, magnetic fields-movement;
shapes of magnetic fields) between the 8 dyad groups.
We analyzed how participants across the two conditions spent their time during the study. Figure 4 shows the distribution of participant activities over the 30 minutes of the study. Participants in the AR condition spent only 73% of their session time actively engaged, with the remaining 27% of their time conversing on non-task-relevant topics. In contrast the NonAR condition spent 90% of time actively engaged, and 10% of time conversing on nontask-relevant topics. This pattern is more prominent in groups with low learning gains -low AR groups conversed about non-task relevant topics 47% of the time while low NonAR groups conversed about non-task topics 19%; the high AR groups spent 8% of their time on non-task relevant topics, vs. 1% in high NonAR. This indicates that AR groups have an earlier tendency to believe that they are finished the activity than compared to NonAR groups, and this effect was stronger in groups with low learning gains. We illustrate the phenomenon of AR participants finishing the activity more quickly in Table 2. This example shows how differently AR and NonAR participants react to the same question asked by the facilitator. After hearing the question, the low AR group pause and stare at the AR system. Both participants observe the direction of the cup then come up with their assumption on the relationship between electricity-movement-magnetic fields. They appear to be satisfied with their answers and switch back to conversing on irrelevant topics. From this observation, this AR group seems to have a positive attitude towards the activity, but do not take effort to explore the questions more deeply. In contrast, it took longer for the NonAR group to discuss the same question. Participant 2 uses a lengthy explanation with various iconic hand gestures, while Participant 1 recaps. Subsequently, they use concepts from this discussion to extend their worksheet answer.
Facilitator: "How is the direction and the strength of the electric current influencing the cup?" [Facilitator leaves the room]
Facilitator: "How is the direction and the strength of the electric current influencing the cup?" [Facilitator leaves the room]
[P1 and P2 taking turns to press forward/backward buttons and look at the electromagnet with superimposed AR magnetic field] P1: "Oh! It's like the direction is either pushing it away or pulling it closer" P2: "Yeah" P1: "The strength when pushing it away is less" [P1 changing the magnitude of the amplifier and looking at the AR amplifier graph] P2: "When the current is weaker, the impact on the membrane of the cup and the magnetic field is smaller" P1: "Yeah" P2: "I don't know where to add on the paper" P1: "I think it's all there" [P1 pointing to their existing answers on the worksheet] [Both sitting silently, then chatting on irrelevant topics] P1: "Is the music linked to the current, so the stronger the current, the stronger the music is?" P2: "Yeah, kinda, So the music is like, little tiny current signals like saying push pull, push pull, modulating. And then it gets amplified from pushing like this, to pushing like that" [P2 using iconic hand gesture showing different strength]. "The more windings you have, the stronger it gets. So you are shaking this magnet at a very specific pace, which then vibrates and makes sound" [P1 looking at P2 while explaining and making iconic hand gestures] P1: "Do you mind answering the question, so the stronger the current, the greater amplification we hear from the cup?" [Adding answer to the question that they previously skipped, while continuing to talk about the activity]
We examined how participants in the two conditions used aids and gestures to facilitate learning, by calculating the percentage of time each aid or gesture was used while participants were involved in either exploring or teaching. In the AR condition, the main learning aid was the system, used for the majority of the time (92%). In contrast, in the NonAR condition participants focused on the system 38% of the time, and spent more time using other aids (compass 32% and poster 12%). Figure 5 illustrates the sequence of switching between different aids. Additionally, the AR condition mostly used deictic gesture (47%) with little drawing (3%) and some iconic gesture (9%) to assist with their learning. This pattern contrasts from the NonAR condition that used less deictic gestures (34%), and much more drawing (14%) and more iconic gesture (12%). This suggests that some "tunnel vision" from the AR participants: they totally neglected other resources that were at their disposal.
Figure 5. Time sequence for aids usage in each 30 seconds block. Null=not using any of the listed aids.
Figure 6 illustrates the sequence of gestures and drawings. These results indicate that the AR condition focused more on the AR-enhanced system and focused more on communication through pointing, while the NonAR participants used the system in concert with other external tools and communicated using iconic gestures and drawings. Again, this suggests more diverse behaviors from the NonAR participants -who used a variety of gestures and drawings to explore the concepts taught.
Figure 6. Time sequence for communication methods usage in each 30 seconds block. Null=not using any of the listed methods.
While running the study, we observed differences in how participants explained concepts to each other. To further explore this question, we analyzed teaching behaviors between AR and NonAR groups. We found that in the AR condition, participants used 3.72% of the activity time for teaching each other, comparing to 27.39% in the NonAR condition (Figure 7, left). Analyzing the use of aids in teaching, we found that AR participants only taught while using the system (100% of the teaching time). In the NonAR condition, participants taught using various aids: 33% of the time by using the system, 43% of the time using compass, 11% of the time using posters, and 12% using other methods such as drawings. Aside from the differences in systems/tools used as aids to teach, the participants in the two conditions also utilized different communication methods to teach (Figure 7, right): the AR condition participants mainly used deictic gestures (67% of the time), with little iconic gestures (8%) and no drawings (0%). In contrast, the NonAR condition participants used less deictic gestures (39%), and more iconic gesture (16%), and drawings (18%).
These results suggest that the AR condition was able to teach more concisely by just using the AR system and skip the process of acquiring representations from other aids or communication methods needed by the NonAR groups. As an illustration of this difference, Table 3 shows how an abstract concept (magnetic polarity) is taught more easily with AR. Without AR, participants read polarity by using a compass, whose use required learning and experimentation. Participant 1 from the NonAR group (Table 3, right) slowly taught Participant 2 on how to read polarity from a compass, whereas in the AR group (Table 3, left) Participant 1 was able to instruct the other participant to look at the polarity labels (North / South) provided in AR near the AR magnetic fields. In this scenario, the AR group is able to skip the process of understanding how to physically read the magnetic polarity, and directly talked about alternative current. In contrast, the NonAR group took a longer time to learn about current in relation to magnetic direction, and remained focused on understanding the slow motions of the magnet rather than fast oscillations resulting in music.
Table 3: Quotes from participants in the AR group and NonAR group teaching.
NonAR Group
[Both looking at the magnetic field AR] P1: "This button pushes the magnet in and out…" P2: "Um hm" P1: "Because the magnetic field changes, you see? It's inverted. It changes between north and south." [P1 referring to the AR magnetic field] P2: "Yea" P1: "So it causes the push and the pull of the magnet, right? P2: "I guess."
[P1 looking at the AR magnetic field while explaining] P1: "The magnetic field is inverted all the time. In order to have music, they change very quickly between forward and backward."
P2: "How do I use this?" [P2 trying to use a compass] P1: "This is north, [P1 moving a compass to the other end] this is south. Let me try one second. The moment you come outside of the membrane, it's north. When it is near the membrane, it is south. So north to south, that's how the magnet direction goes." [Using compass then iconic hand gesture] P2: "How about this part?" [P2 pointing to the coils] P1: "This is where they produce electricity current, it starts from the magnetic membrane, this is where the south and north pole comes in." P1: "When I push a forward current, it starts at south pole and ends at north pole. And the backward current is north to south."
In this qualitative analysis, we found that participants using augmented reality exhibited a 'tunnel vision' effect, where they spent most of their time utilizing the AR system without leveraging other aids available to them (ex: compass, wall poster), in contrast to the NonAR participants who frequently used external aids. This is a possible explanation for why the AR participants showed higher learning gains in concepts requiring visualization, such as understanding magnetic fields, whereas participants without AR showed higher learning gains in other physical concepts such as relationship between magnetic field and movement. It is possible that AR participants focused strongly on the visual experience and ignored physical aids, while the NonAR focused on physical aids, which in turn increased their awareness of physical effects in the learning experience. Therefore, AR experiences might be a "double-edged sword": the spatial-temporal contiguity affordance of AR (Radu, 2014) can help participants comprehend complex co-located visual representations, but also impede them from using physical tools and learning about non-visual aspects of the system. In other words, AR representations might be beneficial for learning visual concepts, but detrimental to acquiring kinesthetic knowledge -because users' attention is so strongly drawn to the "holograms" provided by the AR system, leading them to neglect other sources of information. Furthermore, our qualitative analysis suggests that AR representations may create a false impression of understanding of the concepts conveyed by the system. In our study, AR participants stopped focusing on the task much faster than NonAR participants; additionally, this effect was stronger for AR groups with low learning gains. This observation is supported by the findings from our larger sample, where we found that AR groups showed higher engagement and beliefs about self-efficacy than compared to NonAR groups (Figure 1), even if learning gains were sometimes worse than NonAR groups (Radu & Schneider, 2019). Furthermore, it may be that NonAR groups persisted with the activity for an extended period due to their need to create representations (such as by drawing, using iconic gestures, or using external aids). This suggests that when participants lack easily accessible information through tools such as AR, the extra effort caused them to engage longer with the content and potentially think more critically. In contrast, AR may give people a false sense of confidence.
Finally, our analysis shows that teaching moments were shorter in the AR groups. When participants had AR visualizations available, communication was more efficient as participants could point or simply refer to an AR visual representation. In contrast, NonAR groups (which lacked the visual representations), had to spent time describing invisible phenomena or generate their own representations by using iconic gestures or drawings. They frequently ran out of time while having to make use of various aids and methods to produce their representation. This aligns with results from Ainsworth (2008) which suggest that to use representational learning tools effectively, learners must be able to first understand the function of the representation as well as how the subject relates to the representation. However, the shorter teaching time with deictic gesture may not necessarily be superior to spending longer time teaching with iconic gesture. Alibali et al (2012) discuss how different gestures serve different purposes for learning and teaching-deictic gesture connects cognition to the physical environment, while iconic gesture represents mental images, and these may correspond to different types of learning.
In future work, we are planning to further investigate why participants in AR spent less time on specific tasks and how different conditions influenced collaborative interactions. To achieve that, we are considering to shrink our video observation time frames from 30 sec. to 15 sec. as we may observe more detailed behavior in shorter intervals. We also plan to revise our coding scheme (communication types) to be able to differentiate degrees in communication (e.g., verbal vs. non-verbal when exploring). Additionally, we are working on augmenting qualitative observations with other process metrics collected from the study (i.e., data from electrodermal wristbands and motion sensors).
We qualitatively compared video observations of students learning with and without AR visualizations. We found that AR participants learned more about visual concepts but less about non-visual content, stopped exploring the system quicker than NonAR participants, used less aids in exploration and teaching, and spent less time in teaching their collaborators. Those findings suggest that while there might be opportunities to design AR applications in education, there are drawbacks associated with the use of this technology (e.g., the "tunnel vision" effect described above). In summary, the qualitative findings presented in this paper shed new lights on the affordances of AR technology and provides a critical analysis of the use of augmented reality for co-located collaborative learning activities.