Abstract Disassembly is an essential process for the recovery of end-of-life (EOL) electronics in remanufacturing sites. Nevertheless, the process remains labor-intensive due to EOL electronics’ high degree of uncertainty and complexity. The robotic technology can assist in improving disassembly efficiency; however, the characteristics of EOL electronics pose difficulties for robot operation, such as removing small components. For such tasks, detecting small objects is critical for robotic disassembly systems. Screws are widely used as fasteners in ordinary electronic products while having small sizes and varying shapes in a scene. To enable robotic systems to disassemble screws, the location information and the required tools need to be predicted. This paper proposes a computer vision framework for detecting screws and recommending related tools for disassembly. First, a YOLOv4 algorithm is used to detect screw targets in EOL electronic devices and a screw image extraction mechanism is executed based on the position coordinates predicted by YOLOv4. Second, after obtaining the screw images, the EfficientNetv2 algorithm is applied for screw shape classification. In addition to proposing a framework for automatic small-object detection, we explore how to modify the object detection algorithm to improve its performance and discuss the sensitivity of tool recommendations to the detection predictions. A case study of three different types of screws in EOL electronics is used to evaluate the performance of the proposed framework.
more »
« less
This content will become publicly available on June 12, 2024
Single Shot 3D Shape Measurement of Non-Volatile Data Storage Devices
The value of electronic waste at present is estimated to increase rapidly year after year, and with rapid advances in electronics, shows no signs of slowing down. Storage devices such as SATA Hard Disks and Solid State Devices are electronic devices with high value recyclable raw materials which often goes unrecovered. Most of the e-waste currently generated, including HDDs, is either managed by the informal recycling sector, or is improperly landfilled with the municipal solid waste, primarily due to insufficient recovery infrastructure and labor shortage in the recycling industry. This emphasizes the importance of developing modern advanced recycling technologies such as robotic disassembly. Performing smooth robotic disassembly operations of precision electronics necessitates fast and accurate geometric 3D profiling to provide a quick and precise location of key components. Fringe Projection Profilometry (FPP), as a variation of the well-known structured light technology, provides both the high speed and high accuracy needed to accomplish this. However, Using FPP for disassembly of high-precision electronics such as hard disks can be especially challenging, given that the hard disk platter is almost completely reflective. Furthermore, the metallic nature of its various components make it difficult to render an accurate 3D reconstruction. To address this challenge, We have developed a single-shot approach to predict the 3D point cloud of these devices using a combination of computer graphics, fringe projection, and deep learning. We calibrate a physical FPP-based 3D shape measurement system and set up its digital twin using computer graphics. We capture HDD and SSD CAD models at various orientations to generate virtual training datasets consisting of fringe images and their point cloud reconstructions. This is used to train the U-NET which is then found efficient to predict the depth of the parts to a high accuracy with only a single shot fringe image. This proposed technology has the potential to serve as a valuable fast 3D vision tool for robotic re-manufacturing and is a stepping stone for building a completely automated assembly system.
more »
« less
- Award ID(s):
- 2132773
- NSF-PAR ID:
- 10484224
- Publisher / Repository:
- American Society of Mechanical Engineers
- Date Published:
- Format(s):
- Medium: X
- Location:
- New Brunswick, New Jersey, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Disassembly is an essential process for the recovery of end-of-life (EOL) electronics in remanufacturing sites. Nevertheless, the process remains labor-intensive due to EOL electronics' high degree of uncertainty and complexity. The robotic technology can assist in improving disassembly efficiency, however, the characteristics of EOL electronics pose difficulties for robot operation, such as removing small components. For such tasks, detecting small objects is critical for robotic disassembly systems. Screws are widely used as fasteners in ordinary electronic products while having small sizes and varying shapes in a scene. To achieve robotic disassembly of screws, the location information and the required tools need to be predicted. This paper proposes a framework to automatically detect screws and recommend related tools for disassembly. First, the YOLOv4 algorithm is used to detect screw targets in EOL electronic devices, and then a screw image extraction mechanism is executed based on the position coordinates predicted by YOLOv4. Second, after obtaining the screw images, the EfficientNetv2 algorithm is applied for screw shape classification. In addition to proposing a framework for automatic small-object detection, we explore how to modify the object detection algorithm to improve its performance and discuss the sensitivity of tool recommendations to the detection predictions. A case study of three different types of screws is used to evaluate the performance of the proposed framework.more » « less
-
This paper proposes a phase-to-depth deep learning model to repair shadow-induced errors for fringe projection profilometry (FPP). The model comprises two hourglass branches that extract information from texture images and phase maps and fuses the information from the two branches by concatenation and weights. The input of the proposed model contains texture images, masks, and unwrapped phase maps, and the ground truth is the depth map from CAD models. A loss function was chosen to consider image details and structural similarity. The training data contain 1200 samples in the verified virtual FPP system. After training, we conduct experiments on the virtual and real-world scanning data, and the results support the model’s effectiveness. The mean absolute error and the root mean squared error are 1.0279 mm and 1.1898 mm on the validation dataset. In addition, we analyze the influence of ambient light intensity on the model’s performance. Low ambient light limits the model’s performance as the model cannot extract valid information from the completely dark shadow regions in texture images. The contribution of each branch network is also investigated. Features from the texture-dominant branch are leveraged as guidance to remedy shadow-induced errors. Information from the phase-dominant branch network makes accurate predictions for the whole object. Our model provides a good reference for repairing shadow-induced errors in the FPP system.more » « less
-
A rapid rise in the recycling and remanufacturing of end-of-use electronic waste (e-waste) has been observed due to multiple factors including our increased dependence on electronic products and the lack of resources to meet the demand. E-waste disassembly, which is the operation of extracting valuable components for recycling purposes, has received ever increasing attention as it can serve both the economy and the environment. Traditionally, e-waste disassembly is labor intensive with significant occupational hazards. To reduce labor costs and enhance working efficiency, collaborative robots (cobots) might be a viable option and the feasibility of deploying cobots in high-risk or low value-added e-waste disassembly operations is of tremendous significance to be investigated. Therefore, the major objective of this study was to evaluate the effects of working with a cobot during e-waste disassembly processes on human workload and ergonomics through a human subject experiment. Statistical results revealed that using a cobot to assist participants with the desktop disassembly task reduced the sum of the NASA-TLX scores significantly compared to disassembling by themselves (p = 0.001). With regard to ergonomics, a significant reduction was observed in participants’ mean L5/S1 flexion angle as well as mean shoulder flexion angle on both sides when working with the cobot (p < 0.001). However, participants took a significantly longer time to accomplish the disassembly task when working with the cobot (p < 0.001), indicating a trade-off of deploying cobot in the e-waste disassembly process. Results from this study could advance the knowledge of how human workers would behave and react during human-robot collaborative e-waste disassembly tasks and shed light on the design of better HRC for this specific context.more » « less
-
more » « less
Due to high volumes of production from the consumer electronics industry, it is highly desirable to develop green electronics comprised of biodegradable components derived from cheap resources or even agro‐industrial wastes. In this work, a facile and benign production route is proposed to transform the agro‐industrial waste sugarcane bagasse into value‐added super‐clear nanopaper for flexible and transparent electronics. High‐value cellulose nanocrystals (CNCs) are isolated from abundant and inexpensive sugarcane bagasse by a one‐step 2,2,6,6‐tetramethylpiperidine‐1‐oxyl treatment, and the obtained CNCs are then used to fabricate a super‐clear nanopaper substrate for next‐generation flexible electronics. The super‐clear nanopaper exhibits superior optical properties (91% transmittance and 1.4% transmission haze at 600 nm), excellent nanometer‐scale surface roughness (3.08 nm), and good oxygen barrier properties (1.2 cm3µm (m2day kPa)−1at 50% relative humidity). Moreover, proof‐of‐concept field‐effect transistors with an on/off ratio of >103are demonstrated on this super‐clear nanopaper. The efficient and scalable production of value‐added CNCs from bagasse waste coupled with advanced applications of CNC‐based super‐clear nanopaper in electronic devices presents a solution for the conversion of agricultural wastes to value‐added electronic applications, which is beneficial for green electronics and recycling and ecoindustries.
