The use of Unmanned Aerial Vehicles (UAV), also known as drones, in the construction industry has grown dramatically in recent years. However, along with several benefits, the integration of any novel technology such as UAVs can also bring in new challenges. As a result, more and more research and practice specialists have gradually focused on the challenges that drones may bring to the industry (Albeaino and Gheisari 2021;Khalid et al. 2021;Yahya et al. 2021). Among these challenges, the rise of new safety concerns triggered by such robotic technologies is the most prominent (Bademosi and Issa 2021;Gheisari and Esmaeili 2019). As such, construction workers, who already work in a relatively dangerous industry that accounts for 20% of fatal occupational injuries in the United States (Bureau of Labor Statistics 2019) , will face additional and unknown risks on their job site when it is shared with drones. Consequently, researchers have started investigating the potential risks and creating countermeasures for such challenges (Jeelani and Gheisari 2021). These potential hazards have been identified and categorized as physical risks, attentional costs, and psychological impacts, and several countermeasures have also been proposed to address these risks accordingly (Jeelani and Gheisari 2021).
Previous research regards safety training as an essential and urgently needed intervention to aid construction workers in coping with safety challenges, but there is a research gap on creating effective safety training content and delivery strategies for construction workers and professionals who work in a drone-populated work environment (Albeaino and Gheisari 2021;Khalid et al. 2021;Yahya et al. 2021). Currently, there is no similar training that educates construction workers about the safety challenges when working with drones on their jobsites. Therefore, this paper aims to create an effective training material to educate construction workers about the safety challenges that drones could pose on construction jobsites. The outline of the training program will be designed based on the previous studies that have discussed the specific hazards of drone application at construction sites.
The objectives of this study are to (1) develop a construction worker-drone safety training and (2) conduct assessments of the learning effects and user experience. Considering the known effectiveness of learning experiences through immersive training, the training material was created in a virtual reality (VR) environment and exported as a 360 video, providing an immersive device-agnostic experience that can be easily disseminated. The assessment in this study aims to provide an understanding of the effectiveness and user experience of the proposed training material. A knowledge test about the basic knowledge of drones and the relative safety challenges was performed by the participants before and after training to understand the users' learning effectiveness, and a validated user experience survey was also adopted followed by a discussion on users' qualitative feedback to provide useful insights for improving future training material.
Drones are defined as aerial vehicles that are controlled remotely and do not have a human pilot onboard (Gheisari et al. 2014). This type of robot has since been widely embraced by a variety of industries, including construction. According to DroneDeploy's 2018 commercial drone trends report, the use of drones in construction surged by more than 200 percent in a year, making the construction industry the quickest commercial drone adopter (DroneDeploy 2018). Drones have become increasingly significant in construction because they can execute several construction-related tasks more effectively, swiftly, safely, and at a reduced cost, especially in difficult-to-reach places (Patel et al. 2021). However, according to literature, drone adoption in the construction industry has been hampered by liability and legal concerns, technical concerns, weather condition, and safety concerns (Albeaino and Gheisari 2021;Gheisari and Esmaeili 2019;Yahya et al. 2021). These issues could not only be a deterrent to the construction sector adopting drones, but they could also have significant ramifications, especially given the safety concerns, when more drones enter the construction job site in the future. As a result, studies have begun to investigate these issues and potential solutions (Albeaino and Gheisari 2021;Khalid et al. 2021;Yahya et al. 2021). In a very recent study, Jeelani and Gheisari (Jeelani and Gheisari 2021) categorized the possible health and safety challenges of drones on construction sites into three groups:
(1) physical risks, (2) attentional costs, and (3) psychological impacts; and proposed several interventions-based research directions for ensuring safe integration of drones, including human-drone interaction, cyber-security, and privacy, as well as further research on regulatory, administrative, technological and training interventions. Further research on training intervention has been consistently emphasized in the literature to mitigate the hazards of using drones in construction (Albeaino and Gheisari 2021;Khalid et al. 2021;Yahya et al. 2021). However, limited research has been conducted to design an effective training material that focuses on the safety challenges of drone integration in construction for those who may work alongside drones. In this study, a 360 VR environment was created as a training platform to communicate the safety challenges of construction worker-drone interactions.
Safety training has been considered an important and effective measure to address safety risks in construction industry (Sacks et al. 2013). Lack of training has also been shown to have a strong correlation with construction fatalities (Eteifa and El-adaway 2018). Notably, traditional training methods have been proven to have a lower level of engagement and, therefore, could only provide minimal efficiency in conveying safety-related knowledge (Eiris et al. 2018;Jeelani et al. 2020;Sacks et al. 2013). Compared to traditional safety training methods, VR training, which could provide a sense of presence without exposing construction workers to real risk, has been shown to have better learning effectiveness, engagement, and motivation among learners (Eiris et al. 2021;Jeelani et al. 2020). VR is a technology that involves a computer-generated 3D environment to allow users to control items, move around and see the environment from various angles, immerse and navigate in real-time (Eiris et al. 2021;Wen and Gheisari 2020). Among the various forms of VR technology, 360 VR is thought to have great potential for widespread adoption because it can provide an immersive device-agnostic experience and be accessed on the most popular social media and video sharing platforms that support 360-degree video content (e.g., YouTube, Facebook) (Pham et al. 2018;Wen and Gheisari 2021). Therefore, 360 VR is considered to present new opportunities for improving construction safety education (Pham et al. 2018). In this study, 360 VR video was chosen as the medium for construction worker-drone safety training material to take advantage of its operating system independence and device-capability adaption.
In this study, 360 VR training content was developed in two phases: training content development and 360 VR development. The two phases will be described further in the coming sections.
This phase focused on developing the training content targeted for construction workers, who are most exposed to the risks posed by drones. According to the data, fall accidents are the leading cause of injuries and fatalities in the construction industry, accounting for 34.7% of fatalities in the past decade (CPWR 2021). Furthermore, according to literature, falls from roofs, ladders, and scaffolds are the major scenarios of fall-related accidents in construction, accounting for 71.9% of fall fatalities in the construction industry from 2011 to 2018 (Samantha Brown et al. 2020). Workers on heights are among the workers who are most exposed to drones because their work environment overlaps with the flightpath of drones. Thus, the content of this training was developed based on the potential incidents that could happen when working with or near drones in these "working on heights" scenarios. The scenarios were mainly located on roofs, ladders, and scaffolds, and the safety challenge categories were based on the newly published study by Jeelani and Gheisari (Jeelani and Gheisari 2021). The structure of the training content was based on the following learning objectives: (1) train users about the basic operation and uses of various types of drones used on construction sites; and (2) train users about the health and safety risks associated with drones (see Figure 1). In the first step, we developed a virtual-drone dominant construction site with virtual drones and virtual workers to simulate construction jobsites with multiple drones. We imported the construction-related assets into Unity, and virtual workers and drones were added to the virtual site and programmed to operate as they would in a real construction site. In the second step, the video content structure was defined in a more detailed script, including scene purpose, narration text, and scene descriptions. Following the script, the animation of virtual humans and drones was generated in the Unity game engine. The animations included a virtual instructor that would guide users through the process of learning about drones and construction worker-drone safety (see Figure 3). The VR content was then exported as 360 video to enable users to explore the construction environment better and enhance their sense of presence, which improves understanding and user engagement. The viewers' perspectives switch to attract their attention to specific areas within the 360 video. For instance, when presenting related topics, the users' viewing angle would shift to follow the flying drone, aiding users in understanding the content presented in the video.
The assessment of the developed training material is divided into two parts: learning outcomes assessment and user experience assessment. More than 50 subjects were recruited to evaluate the training video, and 43 effective data sets were collected and analyzed. The survey was primarily aimed at Architecture, Engineering, and Construction (AEC) students with varying professional experience and prior knowledge of construction safety or drones (see Table 1). Besides the quantitative results of learning outcomes and user experience, qualitative feedback from users were also collected. The result of the collected data will be discussed in the following subsections.
A pre-and post-training knowledge evaluation survey was used to evaluate the users' learning outcomes during the training session. Before beginning the training video, users were requested to complete a survey in which they answered questions about drones, their application in construction, and workerdrone safety. Following training, participants were asked to answer the same questions to determine whether the video effectively improved their knowledge level. The result of the learning outcomes assessment shows that the participants' knowledge level increased by 13%, while the P-value of the two sample T-test is 0.013 < 0.05, which indicates a statistically significant increase after going through the training (see Table 2). This result reveals that the training video provided some valuable information for participants.
The framework of the Tcha-Tokey et al. ( 2016) questionnaire was adopted in this study to evaluate user experience during the training session. This questionnaire adopts six criteria to measure user experience: engagement, immersion, emotion, experience consequence, judgment, and technology adoption. The definition of criteria is as follows (Tcha-Tokey et al. 2016):
• Engagement: Energy in action, the connection between a person and its activity, consisting of a behavioral, emotional, and cognitive form. • Immersion: The illusion that the virtual environment technology replaces the user's sensory stimuli with virtual sensory stimuli. • Emotion: The feelings (of joy, pleasure, satisfaction, frustration, disappointment, anxiety) of the user in a virtual environment. • Judgment: The overall judgment of the experience in the virtual environment.
• Experience Consequence: The symptoms (e.g., simulator sickness, stress, dizziness, headache) that user can experience in the virtual environment. • Technology Adoption: The actions and decisions taken by the user for future use or the intention to use the virtual environment.
The user experience results indicate that five criteria with at least acceptable reliability are better than average (2.5/5 or higher). Among these criteria, user engagement obtains the highest score (3.76/5), showing that the training video allows viewers to engage in the situation. In contrast, the experience consequence (measured negatively in the survey) is slightly better (2.34/5) than average, suggesting that video quality should be improved. Notably, the "Technology Adoption" result did not achieve an acceptable degree of reliability, indicating that the questionnaire should be modified to obtain more precise findings for this criterion. Overall, the user experience assessment results demonstrate that the proposed 360 VR training video has great potential to be used as an effective training tool. In addition, users' qualitative feedback was also collected to provide a more precise direction for improving the training video.
In the open-ended section of this survey, the users also provided feedback about the system and how it could be enhanced. The feedback can be divided into five categories. The categories and examples are shown as Table 4. First. some users brought up the low video quality. Limited by state-of-the-art display information, spatial resolution, angular resolution, and viewing angle have become critical trade-offs in 3D displays (Hua et al. 2021). The high-resolution 360 videos also consume more bandwidth (Afzal et al. 2017), which could also influence the user experience since users' internet connections could vary. As such, the technical trade-off should be considered when generating 360 videos in the future. However, some users also mentioned the possibility that the low quality of the video could have been caused by their own settings. So, a brief instruction should be added to ensure users can set the video setting correctly. Another similar feedback is about the poor quality of animations. To provide a better user experience, it was also suggested that the animation quality be improved, notably by animating the virtual characters with enhanced natural movements. Moreover, the instructor's voice was generated using a "text-to-speech" system. Related comments show that using a real human voice could help create a more natural training experience. Furthermore, several users reflected on 360 video's limited interactivity capability and the limited level of immersion of desktop-based VR. It should be noted that the desktop-based 360 VR video has a limited interactive capability with a low sense of realism, but it is device-agnostic and easy to access. To address the interactivity issue, a VR game with a more interactive user interface could be developed in the future. Moreover, future efforts could be made to evaluate if the limitation of immersion could significantly influence learning outcomes. Finally, some users reported cybersickness and exhaustion. A few suggestions were also provided about increasing the speaking pace to finish the training faster. To address this, one solution could be to break the training into several parts and arrange a short break between each part of the training.
The research aimed to develop a drone safety training video in a 360 VR environment and understand the learning effects and user experience of the proposed video, which is intended to fill a research gap concerning a lack of training interventions to mitigate the risks that construction workers may face when co-working with drones on construction job sites. To accomplish the research goals, a virtual construction job site with several virtual drones was created. The structure of the training material was designed to assist users in grasping the fundamentals of drone operation in construction and educate them about the safety challenges of working near drones. A small sample pilot evaluation with 43 effective data points was conducted to evaluate the learning effects and user experience of the developed training content. The training was delivered via YouTube and offered the web-based and deviceindependent benefits of 360 VR video. The results reveal that users' knowledge level can be enhanced by 13% after training, and five out of six criteria for evaluating user experience received acceptable reliability. The outcome demonstrates the high potential of the designed training content to serve as an effective training tool. Furthermore, several suggestions were derived from the qualitative comments from users, providing valuable feedback to improve the training video. Future research should collect a larger sample size and reexamine the questionnaire to analyze user replies using in-depth statistical analysis. In addition, only AEC-related students were recruited for this pilot study. To evaluate the learning impacts and user experience of different groups of users, a diverse range of backgrounds and professions connected to the AEC industry should be included. Furthermore, the qualitative responses from this study should be considered to improve the development of training videos in future studies.
Construction Worker-Drone Safety Training in a 360 Virtual Reality Environmen ...J.-Y. Cheng et al.