An official website of the United States government
Abstract While there is increased interest in using movement and embodiment to support learning due to the rise in theories of embodied cognition and learning, additional work needs to be done to explore how we can make sense of students collectively developing their understanding within a mixed-reality environment. In this paper, we explore embodied communication’s individual and collective functions as a way of seeing students’ learning through embodiment. We analyze data from a mixed-reality (MR) environment: Science through Technology Enhanced Play (STEP) (Danish et al., International Journal of Computer-Supported Collaborative Learning 15:49–87, 2020), using descriptive statistics and interaction analysis to explore the role of gesture and movement in student classroom activities and their pre-and post-interviews. The results reveal that students appear to develop gestures for representing challenging concepts within the classroom and then use these gestures to help clarify their understanding within the interview context. We further explore how students collectively develop these gestures in the classroom, with a focus on their communicative acts, then provide a list of individual and collective functions that are supported by student gestures and embodiment within the STEP MR environment, and discuss the functions of each act. Finally, we illustrate the value of attending to these gestures for educators and designers interested in supporting embodied learning.
more »
« less
Embodied mixed reality with passive haptics in STEM education: randomized control study with chemistry titration
Researchers, educators, and multimedia designers need to better understand how mixing physical tangible objects with virtual experiences affects learning and science identity. In this novel study, a 3D-printed tangible that is an accurate facsimile of the sort of expensive glassware that chemists use in real laboratories is tethered to a laptop with a digitized lesson. Interactive educational content is increasingly being placed online, it is important to understand the educational boundary conditions associated with passive haptics and 3D-printed manipulables. Cost-effective printed objects would be particularly welcome in rural and low Socio-Economic (SES) classrooms. A Mixed Reality (MR) experience was created that used a physical 3D-printed haptic burette to control a computer-based chemistry titration experiment. This randomized control trial study with 136 college students had two conditions: 1) low-embodied control (using keyboard arrows), and 2) high-embodied experimental (physically turning a valve/stopcock on the 3D-printed burette). Although both groups displayed similar significant gains on the declarative knowledge test, deeper analyses revealed nuanced Aptitude by Treatment Interactions (ATIs). These interactionsfavored the high-embodied experimental group that used the MR devicefor both titration-specific posttest knowledge questions and for science efficacy and science identity. Those students with higher prior science knowledge displayed higher titration knowledge scores after using the experimental 3D-printed haptic device. A multi-modal linguistic and gesture analysis revealed that during recall the experimental participants used the stopcock-turning gesture significantly more often, and their recalls created a significantly different Epistemic Network Analysis (ENA). ENA is a type of 2D projection of the recall data, stronger connections were seen in the high embodied group mainly centering on the key hand-turning gesture. Instructors and designers should consider the multi-modal and multi-dimensional nature of the user interface, and how the addition of another sensory-based learning signal (haptics) might differentially affect lower prior knowledge students. One hypothesis is that haptically manipulating novel devices during learning may create more cognitive load. For low prior knowledge students, it may be advantageous for them to begin learning content on a more ubiquitous interface (e.g., keyboard) before moving them to more novel, multi-modal MR devices/interfaces.
more »
« less
- Award ID(s):
- 1917912
- PAR ID:
- 10476637
- Publisher / Repository:
- Frontiers in Virtual Reality
- Date Published:
- Journal Name:
- Frontiers in Virtual Reality
- Volume:
- 4
- ISSN:
- 2673-4192
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Immersive Virtual Environments (IVEs) incorporating tangibles are becoming more accessible. The success of applications combining 3D printed tangibles and VR often depends on how accurately size is perceived. Research has shown that visuo-haptic perceptual information is important in the perception of size. However, it is unclear how these sensory-perceptual channels are affected by immersive virtual environments that incorporate tangible objects. Towards understanding the effects of different sensory information channels in the near field size perception of tangibles of graspable sizes in IVEs, we conducted a between-subjects study evaluating the accuracy of size perception across three experimental conditions (Vision-only, Haptics-only, Vision and Haptics). We found that overall, participants consistently over-estimated the size of the dials regardless of the type of perceptual information that was presented. Participants in the haptics only condition overestimated diameters to a larger degree as compared to other conditions. Participants were most accurate in the vision only condition and least accurate in the haptics only condition. Our results also revealed that increased efficiency in reporting size over time was most pronounced in the visuo- haptic condition.more » « less
-
Embodied, physical interaction can improve learning by making abstractions concrete, while online courses and interactive lesson plans have increased education access and versatility. Haptic technology could integrate these benefits, but requires both low-cost hardware (recently enabled by low-cost DIY devices) and accessible software that enables students to creatively explore haptic environments without writing code. To investigate haptic e-learning without user programming, we developed HandsOn, a conceptual model for exploratory, embodied STEM education software; and implemented it with the SpringSim interface and a task battery for high school students. In two studies, we confirm that low-cost devices can render haptics adequately for this purpose, find qualitative impact of SpringSim on student strategies and curiosity, and identify directions for tool improvement and extension.more » « less
-
The electronics-centered approach to physical computing presents challenges when designers build tangible interactive systems due to its inherent emphasis on circuitry and electronic components. To explore an alternative physical computing approach we have developed a computer vision (CV) based system that uses a webcam, computer, and printed fiducial markers to create functional tangible interfaces. Through a series of design studios, we probed how designers build tangible interfaces with this CV-driven approach. In this paper, we apply the annotated portfolio method to reflect on the fifteen outcomes from these studios. We observed that CV markers offer versatile materiality for tangible interactions, afford the use of democratic materials for interface construction, and engage designers in embodied debugging with their own vision as a proxy for CV. By sharing our insights, we inform other designers and educators who seek alternative ways to facilitate physical computing and tangible interaction design.more » « less
-
Haptic force feedback systems are unique in their ability to dynamically render physical representations. Although haptic devices have shown promise for supporting learning, prior work mainly describes results of haptic-supported learning without identifying underlying learning mechanisms. To this end, we designed a haptic-supported learning environment and analyzed four students who used it to make connections between two different mathematical representations of sine and cosine: the unit circle, and their graph on the Cartesian plane. We highlight moments where students made connections between the representations, and identify how the haptic feedback supported these moments of insight. We use this evidence in support of a proposed theoretical and design framework for educational haptics. This framework captures four types of haptic representations, and focuses on one -- the haptic bridge -- that effectively scaffolds sense-making with multiple representations.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/frvir.2023.1047833
