An official website of the United States government
This study aims at sensing and understanding the worker’s activity in a human-centered intelligent manufacturing system. We propose a novel multi-modal approach for worker activity recognition by leveraging information from different sensors and in different modalities. Specifically, a smart armband and a visual camera are applied to capture Inertial Measurement Unit (IMU) signals and videos, respectively. For the IMU signals, we design two novel feature transform mechanisms, in both frequency and spatial domains, to assemble the captured IMU signals as images, which allow using convolutional neural networks to learn the most discriminative features. Along with the above two modalities, we propose two other modalities for the video data, i.e., at the video frame and video clip levels. Each of the four modalities returns a probability distribution on activity prediction. Then, these probability distributions are fused to output the worker activity classification result. A worker activity dataset is established, which at present contains 6 common activities in assembly tasks, i.e., grab a tool/part, hammer a nail, use a power-screwdriver, rest arms, turn a screwdriver, and use a wrench. The developed multi-modal approach is evaluated on this dataset and achieves recognition accuracies as high as 97% and 100% in the leave-one-out and half-half experiments, respectively.
more »
« less
A self-aware and active-guiding training & assistant system for worker-centered intelligent manufacturing
Training and on-site assistance is critical to help workers master required skills, improve worker productivity, and guarantee the product quality. Traditional training methods lack worker-centered considerations that are particularly in need when workers are facing ever changing demands. In this study, we propose a worker-centered training & assistant system for intelligent manufacturing, which is featured with self-awareness and active-guidance. Multi-modal sensing techniques are applied to perceive each individual worker and a deep learning approach is developed to understand the worker’s behavior and intention. Moreover, an object detection algorithm is implemented to identify the parts/tools the worker is interacting with. Then the worker’s current state is inferred and used for quantifying and assessing the worker performance, from which the worker’s potential guidance demands are analyzed. Furthermore, onsite guidance with multi-modal augmented reality is provided actively and continuously during the operational process. Two case studies are used to demonstrate the feasibility and great potential of our proposed approach and system for applying to the manufacturing industry for frontline workers.
more »
« less
- Award ID(s):
- 1646162
- PAR ID:
- 10129791
- Date Published:
- Journal Name:
- Manufacturing Letters
- Volume:
- 21
- Page Range / eLocation ID:
- 45-49
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In manufacturing industries, equipment arrangement, and layout design are critical factors that directly influence productivity, workplace safety, and workers’ performance. Link analysis, as a human factors approach, has been widely used in industries for many years to improve layout design and machinery arrangement. This approach considers humans' physical and cognitive capabilities and movement limitations to find an optimal design. Virtual reality significantly impacts our society from product design to worker training. Hence, effective virtual training platforms require the same attention to layout design as manufacturing work settings which offer efficient testing of multiple layouts. This research focuses on developing a virtual 3D printing laboratory for workforce training and has used a link analysis and user perception study to improve the layout of the virtual workplace. The research demonstrates the importance of layout design in virtual training platforms and the potential benefits of utilizing link analysis in optimizing layout design.more » « less
-
Work-related musculoskeletal disorders (WMSDs) are a leading cause of injury for workers who are performing physically demanding and repetitive construction tasks. With recent advances in robotics, wearable robots are introduced into the construction industry to mitigate the risk of WMSDs by correcting the workers’ postures and reducing the load exerted on their body joints. While wearable robots promise to reduce the muscular and physical demands on workers to perform tasks, there is a lack of understanding of the impact of wearable robots on worker ergonomics. This lack of understanding may lead to new ergonomic injuries for worker swearing exoskeletons. To bridge this gap, this study aims to assess the workers’ ergonomic risk when using a wearable robot (back-support exoskeleton) in one of the most common construction tasks, material handling. In this research, a vision-based pose estimation algorithm was developed to estimate the pose of the worker while wearing a back-support exoskeleton. As per the estimated pose, joint angles between connected body parts were calculated. Then, the worker’s ergonomic risk was assessed from the calculated angles based on the Rapid Entire Body Assessment (REBA) method. Results showed that using the back-support exoskeleton reduced workers’ ergonomic risk by 31.7% by correcting awkward postures of the trunk and knee during material handling tasks, compared to not using the back-support exoskeleton. The results are expected to facilitate the implementation of wearable robots in the construction industry.more » « less
-
Smart City is a key component in Internet of Things (IoTs), so it has attracted much attention. The emergence of Mobile Crowd Sensing (MCS) systems enables many smart city applications. In an MCS system, sensing tasks are allocated to a number of mobile users. As a result, the sensing related context of each mobile user plays a significant role on service quality. However, some important sensing context is ignored in the literature. This motivates us to propose a Context-aware Multi-Armed Bandit (C-MAB) incentive mechanism to facilitate quality-based worker selection in an MCS system. We evaluate a worker’s service quality by its context (i.e., extrinsic ability and intrinsic ability) and cost. Based on our proposed C-MAB incentive mechanism and quality evaluation design, we develop a Modified Thompson Sampling Worker Selection (MTS-WS) algorithm to select workers in a reinforcement learning manner. MTS-WS is able to choose effective workers because it can maintain accurate worker quality information by updating evaluation parameters according to the status of task accomplishment. We theoretically prove that our C-MAB incentive mechanism is selection efficient, computationally efficient, individually rational, and truthful. Finally, we evaluate our MTS-WS algorithm on simulated and real-world datasets in comparison with some other classic algorithms. Our evaluation results demonstrate that MTS-WS achieves the highest cumulative utility of the requester and social welfare.more » « less
-
With the fast development of Industry 4.0, the ways in which manufacturing workers handle machines, materials, and products also change drastically. Such changes post several demanding challenges to the training of future workforce. First, personalized manufacturing will lead to small batch and fast changing tasks. The training procedure must demonstrate agility. Second, new interfaces to interact with human or robots will change the training procedure. Last but not least, in addition to handling the physical objects, a worker also needs to be trained to digest and respond to rich data generated at the manufacturing site. To respond to these challenges, in this paper we describe the design of an AI-assisted training platform for manufacturing workforce. The platform will collect rich data from both machines and workers. It will capture and analyze both macro and micro movement of trainees with the help of AI algorithms. At the same time, training for interaction with robot/cobot will also be covered. Mixed reality will be used to create in-situ experiences for the trainee.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
