skip to main content


This content will become publicly available on June 1, 2024

Title: Forecasting the Range of Possible Human Hand Movement in Consumer Electronics Disassembly Using Machine Learning
Robotic technology can benefit disassembly operations by reducing human operators’ workload and assisting them with handling hazardous materials. Safety consideration and predicting human movement is a priority in human-robot close collaboration. The point-by-point forecasting of human hand motion which forecasts one point at each time does not provide enough information on human movement due to errors between the actual movement and predicted value. This study provides a range of possible hand movements to enhance safety. It applies three machine learning techniques including Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Bayesian Neural Network (BNN) combined with Bagging and Monte Carlo Dropout (MCD), namely LSTM-Bagging, GRU-Bagging, and BNN-MCD to predict the possible movement range. The study uses an Inertial Measurement Units (IMU) dataset collected from the disassembly of desktop computers to show the application of the proposed method. The findings reveal that BNN-MCD outperforms other models in forecasting the range of possible hand movement.  more » « less
Award ID(s):
2026276
NSF-PAR ID:
10465138
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the ASME 2023 18th International Manufacturing Science and Engineering Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Madden, John D. ; Anderson, Iain A. ; Shea, Herbert R. (Ed.)
    Ras Labs makes Synthetic Muscle™, which is a class of electroactive polymer (EAP) based materials and actuators that sense pressure (gentle touch to high impact), controllably contract and expand at low voltage (1.5 V to 50 V, including use of batteries), and attenuate force. We are in the robotics era, but robots do have their challenges. Currently, robotic sensing is mainly visual, which is useful up until the point of contact. To understand how an object is being gripped, tactile feedback is needed. For handling fragile objects, if the grip is too tight, breakage occurs, and if the grip is too loose, the object will slip out of the grasp, also leading to breakage. Rigid robotic grippers using a visual feedback loop can struggle to determine the exact point and quality of contact. Robotic grippers can also get a stuttering effect in the visual feedback loop. By using soft Synthetic Muscle™ based EAP pads as the sensors, immediate feedback was generated at the first point of contact. Because these pads provided a soft, compliant interface, the first point of contact did not apply excessive force, allowing the force applied to the object to be controlled. The EAP sensor could also detect a change in pressure location on its surface, making it possible to detect and prevent slippage by then adjusting the grip strength. In other words, directional glide provided feedback for the presence of possible slippage to then be able to control a slightly tighter grip, without stutter, due to both the feedback and the soft gentleness of the fingertip-like EAP pads themselves. The soft nature of the EAP fingertip pad also naturally held the gripped object, improving the gripping quality over rigid grippers without an increase in applied force. Analogous to finger-like tactile touch, the EAPs with appropriate coatings and electronics were positioned as pressure sensors in the fingertip or end effector regions of robotic grippers. This development of using Synthetic Muscle™ based EAPs as soft sensors provided for sensors that feel like the pads of human fingertips. Basic pressure position and magnitude tests have been successful, with pressure sensitivity down to 0.05 N. Most automation and robots are very strong, very fast, and usually need to be partitioned away from humans for safety reasons. For many repetitive tasks that humans do with delicate or fragile objects, it would be beneficial to use robotics; whether it is for agriculture, medical surgery, therapeutic or personal care, or in extreme environments where humans cannot enter, including with contagions that have no cure. Synthetic Muscle™ was also retrofitted as actuator systems into off-the-shelf robotic grippers and is being considered in novel biomimetic gripper designs, operating at low voltages (less than 50 V). This offers biomimetic movement by contracting like human muscles, but also exceeds natural biological capabilities by expanding under reversed electric polarity. Human grasp is gentle yet firm, with tactile touch feedback. In conjunction with shape-morphing abilities, these EAPs also are being explored to intrinsically sense pressure due to the correlation between mechanical force applied to the EAP and its electronic signature. The robotic field is experiencing phenomenal growth in this fourth phase of the industrial revolution, the robotics era. The combination of Ras Labs’ EAP shape-morphing and sensing features promises the potential for robotic grippers with human hand-like control and tactile sensing. This work is expected to advance both robotics and prosthetics, particularly for collaborative robotics to allow humans and robots to intuitively work safely and effectively together. 
    more » « less
  2. Power grid operators rely on solar irradiance forecasts to manage uncertainty and variability associated with solar power. Meteorological factors such as cloud cover, wind direction, and wind speed affect irradiance and are associated with a high degree of variability and uncertainty. Statistical models fail to accurately capture the dependence between these factors and irradiance. In this paper, we introduce the idea of applying multivariate Gated Recurrent Units (GRU) to forecast Direct Normal Irradiance (DNI) hourly. The proposed GRU-based forecasting method is evaluated against traditional Long Short-Term Memory (LSTM) using historical irradiance data (i.e., weather variables that include cloud cover, wind direction, and wind speed) to forecast irradiance forecasting over intra-hour and inter-hour intervals. Our evaluation on one of the sites from Measurement and Instrumentation Data Center indicate that both GRU and LSTM improved DNI forecasting performance when evaluated under different conditions. Moreover, including wind direction and wind speed can have substantial improvement in the accuracy of DNI forecasts. Besides, the forecasting model can accurately forecast irradiance values over multiple forecasting horizons. 
    more » « less
  3. Variation in solar irradiance causes power generation fluctuations in solar power plants. Power grid operators need accurate irradiance forecasts to manage this variability. Many factors affect irradiance, including the time of year, weather and time of day. Cloud cover is one of the most important variables that affects solar power generation, but is also characterized by a high degree of variability and uncertainty. Deep learning methods have the ability to learn long-term dependencies within sequential data. We investigate the application of Gated Recurrent Units (GRU) to forecast solar irradiance and present the results of applying multivariate GRU to forecast hourly solar irradiance in Phoenix, Arizona. We compare and evaluate the performance of GRU against Long Short-Term Memory (LSTM) using strictly historical solar irradiance data as well as the addition of exogenous weather variables and cloud cover data. Based on our results, we found that the addition of exogenous weather variables and cloud cover data in both GRU and LSTM significantly improved forecasting accuracy, performing better than univariate and statistical models. 
    more » « less
  4. We propose Chirality Nets, a family of deep nets that is equivariant to the “chirality transform,” i.e., the transformation to create a chiral pair. Using parameter sharing, odd and even symmetry, we develop variants of standard building blocks of deep nets that satisfy the equivariance property, including fully connected layers, convolutional layers, batch-normalization, and LSTM/GRU cells. The proposed layers lead to a more data efficient representation and a reduction in computation due to their symmetry. To assess efficacy, we study three pose regression tasks: 3D pose estimation from video, 2D pose forecasting, and skeleton based activity recognition. Our approach achieves/matches state-of-the-art results, with more significant gains on small datasets and limited-data settings. 
    more » « less
  5. This paper presents a comprehensive disassembly sequence planning (DSP) algorithm in the human–robot collaboration (HRC) setting with consideration of several important factors including limited resources and human workers’ safety. The proposed DSP algorithm is capable of planning and distributing disassembly tasks among the human operator, the robot, and HRC, aiming to minimize the total disassembly time without violating resources and safety constraints. Regarding the resource constraints, we consider one human operator and one robot, and a limited quantity of disassembly tools. Regarding the safety constraints, we consider avoiding potential human injuries from to-be-disassembled components and possible collisions between the human operator and the robot due to the short distance between disassembly tasks. In addition, the transitions for tool changing, the moving between disassembly modules, and the precedence constraint of components to be disassembled are also considered and formulated as constraints in the problem formulation. Both numerical and experimental studies on the disassembly of a used hard disk drive (HDD) have been conducted to validate the proposed algorithm. 
    more » « less