skip to main content
Explore Scholarly Publications and Datasets in the NSF-PAR

Title: Eye-Track Modeling of Problem-Solving in Virtual Manufacturing Environments
Problem-solving focuses on defining and analyzing problems, then finding viable solutions through an iterative process that requires brainstorming and understanding of what is known and what is unknown in the problem space. With rapid changes of economic landscape in the United States, new types of jobs emerge when new industries are created. Employers report that problem-solving is the most important skill they are looking for in job applicants. However, there are major concerns about the lack of problem-solving skills in engineering students. This lack of problem-solving skills calls for an approach to measure and enhance these skills. In this research, we propose to understand and improve problem-solving skills in engineering education by integrating eye-tracking sensing with virtual reality (VR) manufacturing. First, we simulate a manufacturing system in a VR game environment that we call a VR learning factory. The VR learning factory is built in the Unity game engine with the HTC Vive VR system for navigation and motion tracking. The headset is custom-fitted with Tobii eye-tracking technology, allowing the system to identify the coordinates and objects that a user is looking at, at any given time during the simulation. In the environment, engineering students can see through the headset more » a virtual manufacturing environment composed of a series of workstations and are able to interact with workpieces in the virtual environment. For example, a student can pick up virtual plastic bricks and assemble them together using the wireless controller in hand. Second, engineering students are asked to design and assemble car toys that satisfy predefined customer requirements while minimizing the total cost of production. Third, data-driven models are developed to analyze eye-movement patterns of engineering students. For instance, problem-solving skills are measured by the extent to which the eye-movement patterns of engineering students are similar to the pattern of a subject matter expert (SME), an ideal person who sets the expert criterion for the car toy assembly process. Benchmark experiments are conducted with a comprehensive measure of performance metrics such as cycle time, the number of station switches, weight, price, and quality of car toys. Experimental results show that eye-tracking modeling is efficient and effective to measure problem-solving skills of engineering students. The proposed VR learning factory was integrated into undergraduate manufacturing courses to enhance student learning and problem-solving skills. « less
Authors:
; ; ;
Award ID(s):
1830741
Publication Date:
NSF-PAR ID:
10285346
Journal Name:
ASEE annual conference proceedings
ISSN:
1524-4857
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , Proceedings of the Winter Simulation Conference)
    Virtual reality (VR) technology allows for the creation of fully immersive environments that enable personalized manufacturing learning. This case study discusses the development of a virtual learning factory that integrates manual and automated manufacturing processes such as welding, fastening, 3D printing, painting, and automated assembly. Two versions of the virtual factory are developed: (1) a multiplayer VR environment for the design and assembly of car toys; which allows for the collaboration of multiple users in the same VR environment, and (2) a virtual plant that utilizes heavy machinery and automated assembly lines for car manufacturing. The virtual factory also includes an intelligent avatar that can interact with the users and guide them to the different sections of the plant. The virtual factory enhances the learning of advanced manufacturing concepts by combining virtual objects with hands-on activities and providing students with an engaging learning experience.
  2. ; ; ; ( , Proceedings of the Winter Simulation Conference)
    Virtual reality (VR) technology allows for the creation of fully immersive environments that enable personalized manufacturing learning. This case study discusses the development of a virtual learning factory that integrates manual and automated manufacturing processes such as welding, fastening, 3D printing, painting, and automated assembly. Two versions of the virtual factory are developed: (1) a multiplayer VR environment for the design and assembly of car toys; which allows for the collaboration of multiple users in the same VR environment, and (2) a virtual plant that utilizes heavy machinery and automated assembly lines for car manufacturing. The virtual factory also includes an intelligent avatar that can interact with the users and guide them to the different sections of the plant. The virtual factory enhances the learning of advanced manufacturing concepts by combining virtual objects with hands-on activities and providing students with an engaging learning experience.
  3. ; ; ; ; ; ( , Paleoanthropology)
    Like many natural sciences, a critical component of archaeology is field work. Despite its importance, field opportunities are available to few students for financial and logistical reasons. With little exposure to archaeological research, fewer students are entering archaeology, particularly minority students (Smith 2004; Wilson 2015). To counter these trends, we have leveraged the ongoing revolution in consumer electronics for the current, digitally-empowered generation by creating a game-based, virtual archaeology curriculum to 1) teach foundational principles of a discipline that is challenging to present in a traditional classroom by using sensory and cognitive immersion; and, 2) allow wider access to a field science that has previously been limited to only select students. Virtual reality (VR) is computer technology that creates a simulated three-dimensional world for a user to experience in a bodily way, thereby transforming data analysis into a sensory and cognitive experience. Using a widely-available, room-scale, VR platform, we have created a virtual archaeological excavation experience that conveys two overarching classroom objectives: 1) teach the physical methods of archaeological excavation by providing the setting and tools for a student to actively engage in field work; and, 2) teach archaeological concepts using a scientific approach to problem solving by couching themmore »within a role-playing game. The current prototype was developed with the HTC Vive VR platform, which includes a headset, hand controllers, and two base stations to track the position and orientation of the user’s head and hands within a 4x4 meter area. Environments were developed using Unreal Engine 4, an open source gaming engine, to maximize usability for different audiences, learning objectives, and skill levels. Given the inherent fun of games and widespread interest in archaeology and cultural heritage, the results of this research are adaptable and applicable to learners of all ages in formal and informal educational settings.« less
  4. ( , ASEE Annual Conference proceedings)
    Familiarity with manufacturing environments is an essential aspect for many engineering students. However, such environments in real world often contain expensive equipment making them difficult to recreate in an educational setting. For this reason, simulated physical environments where the process is approximated using scaled-down representations are usually used in education. However, such physical simulations alone may not capture all the details of a real environment. Virtual reality (VR) technology nowadays allows for the creation of fully immersive environments, bringing simulations to the next level. Using rapidly advancing gaming technology, this research paper explores the applicability of creating multiplayer serious games for manufacturing simulation. First, we create and validate a hands-on activity that engages groups of students in the design and assembly of toy cars. Then, a corresponding multiplayer VR game is developed, which allows for the collaboration of multiple VR users in the same virtual environment. With a VR headset and proper infrastructure, a user can participate in a simulation game from any location. This paper explores whether multiplayer VR simulations could be used as an alternative to physical simulations.
  5. ; ; ; ( , ASEE Annual Conference proceedings)
    Familiarity with manufacturing environments is an essential aspect for many engineering students. However, such environments in real world often contain expensive equipment making them difficult to recreate in an educational setting. For this reason, simulated physical environments where the process is approximated using scaled-down representations are usually used in education. However, such physical simulations alone may not capture all the details of a real environment. Virtual reality (VR) technology nowadays allows for the creation of fully immersive environments, bringing simulations to the next level. Using rapidly advancing gaming technology, this research paper explores the applicability of creating multiplayer serious games for manufacturing simulation. First, we create and validate a hands-on activity that engages groups of students in the design and assembly of toy cars. Then, a corresponding multiplayer VR game is developed, which allows for the collaboration of multiple VR users in the same virtual environment. With a VR headset and proper infrastructure, a user can participate in a simulation game from any location. This paper explores whether multiplayer VR simulations could be used as an alternative to physical simulations.
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email