skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: TetRotation: Utilising Multimodal Analytics and Gestural Interaction to Nurture Mental Rotation Skills
Embodied Interaction (EI) offers unique opportunities to uncover novel ways to achieve experiential learning whilst keeping students stimulated and engaged. Spatial abilities have been repeatedly demonstrated as a success predictor for educations and professions in Science, Technology, Engineering and Mathematics. However, many researchers argue that training and assessment of this pertinent reasoning skill is vastly underrepresented in the school curriculum. This paper presents TetRotation, a PhD centred on how affordances coming from Multimodal Analytics can be coupled with EI to nurture Mental Rotation (MR) skills. The overarching objectives of the project are two fold. First, the TetRotation Interaction Design study will highlight best practices identified through the assessment of efficiency, level of engagement and learning gains achieved when using gesture based EI to solve MR tasks. Next, in the TetRotation Game study, these design practices will guide the implementation of an interactive serious game purposed to support the development of MR skills. This research relies on mixed method techniques, including data collections from users' actions, like motion sensing, EEG, gaze tracking, video-recordings, click streams, interviews and surveys.  more » « less
Award ID(s):
1919375
PAR ID:
10156503
Author(s) / Creator(s):
Date Published:
Journal Name:
Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’19).
Page Range / eLocation ID:
715 to 718
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Proceedings of the 30th International Conference on Computers in Education. Asia-Pacific Society for Computers in Education)
    Iyer, S (Ed.)
    In this work-in-progress poster, we will present how a team including game designers, learning scientists, and assessment scientists collaborated on an online adventure game, Aqualab, with the goal of creating a comprehensive long-format game that can be used across multiple classroom sessions to support development science inquiry practices as well as assess different learning pathways within the game. In this work-in-progress poster, we discuss how the team approached design and development of the game to ensure validity of the game, and how we are planning to further investigate validity evidence of the game as a whole. 
    more » « less
  2. ; ; ; ; (, Asia-Pacific Society for Computers in Education (APSCE))
    In this work-in-progress poster, we will present how a team including game designers, learning scientists,and assessment scientists collaborated on an online adventure game, Aqualab, with the goal of creating a comprehensive long-format game that can be used across multiple classroom sessions to support development science inquiry practices as well as assess different learning pathways within the game. In this work-in-progress poster, we discuss how the team approached design and development of the game to ensure validity of the game, and how we are planning to further investigate validity evidence of the gameas a whole. 
    more » « less
  3. ; ; ; ; ; (, 2018 IEEE Frontiers in Education Conference (FIE))
    This research work-in-progress paper investigated the application of emerging mixed reality (MR) technology in construction and engineering education. The construction industry is facing a severe shortage of skilled workforce. As the baby boomers are retiring, the younger generation, especially college students, are often criticized for their lack of professional experience and career-specific competency. To close the skills gap and accelerate the transition of college students to competent workforce, this paper proposed a new genre of learning and professional training using MR. The main promise of the MR technology resides in its ability to augment virtual contents on top of the physical reality to facilitate tacit knowledge learning, and simulate learning activities that traditionally can only be obtained from actual professional experience. An undergraduate wood framing lab was designed as a case study to explore how students might perform in this new learning and training environment. Specifically, the case study investigated if MR would facilitate student design comprehension and transfer such understanding into the knowledge and skills needed to build the wood structure. A randomly selected student control group was given traditional paper-based construction drawings to perform the same tasks with other student groups with various visualization technology assistance. Project performance and behavior of student groups were compared to determine if there was a significant difference between the control group and the experiment groups. A pair of pre- and post-survey on MR-intervened learning experience was also conducted to explore student perceptions towards this new genre of learning and training. The research design proposed in this work-in-progress study and its preliminary results could be a good reference and foundation to future research in this arena. 
    more » « less
  4. ; ; ; ; ; (, Artificial Intelligence in Education)
    A key untapped feature of game-based learning environments is their capacity to generate a rich stream of fine-grained learning interaction data. The learning behaviors captured in these data provide a wealth of information on student learning, which stealth assessment can utilize to unobtrusively draw inferences about student knowledge to provide tailored problem-solving support. In this paper, we present a long short-term memory network (LSTM)-based stealth assessment framework that takes as input an observed sequence of raw game-based learning environment interaction data along with external pre-learning measures to infer students’ post-competencies. The framework is evaluated using data collected from 191 middle school students interacting with a game-based learning environment for middle grade computational thinking. Results indicate that LSTM-based stealth assessors induced from student game-based learning interaction data outperform comparable models that required labor-intensive hand-engineering of input features. The findings suggest that the LSTM-based approach holds significant promise for evidence modeling in stealth assessment. 
    more » « less
  5. ; ; (, Frontiers in Virtual Reality)
    Efficient performance and acquisition of physical skills, from sports techniques to surgical procedures, require instruction and feedback. In the absence of a human expert, Mixed Reality Intelligent Task Support (MixITS) can offer a promising alternative. These systems integrate Artificial Intelligence (AI) and Mixed Reality (MR) to provide realtime feedback and instruction as users practice and learn skills using physical tools and objects. However, designing MixITS systems presents challenges beyond engineering complexities. The complex interactions between users, AI, MR interfaces, and the physical environment create unique design obstacles. To address these challenges, we present MixITS-Kit—an interaction design toolkit derived from our analysis of MixITS prototypes developed by eight student teams during a 10-week-long graduate course. Our toolkit comprises design considerations, design patterns, and an interaction canvas. Our evaluation suggests that the toolkit can serve as a valuable resource for novice practitioners designing MixITS systems and researchers developing new tools for human-AI interaction design. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email