An official website of the United States government
Abstract Achieving multicapability in a single soft gripper for handling ultrasoft, ultrathin, and ultraheavy objects is challenging due to the tradeoff between compliance, strength, and precision. Here, combining experiments, theory, and simulation, we report utilizing angle-programmed tendril-like grasping trajectories for an ultragentle yet ultrastrong and ultraprecise gripper. The single gripper can delicately grasp fragile liquids with minimal contact pressure (0.05 kPa), lift objects 16,000 times its own weight, and precisely grasp ultrathin, flexible objects like 4-μm-thick sheets and 2-μm-diameter microfibers on flat surfaces, all with a high success rate. Its scalable and material-independent design allows for biodegradable noninvasive grippers made from natural leaves. Explicitly controlled trajectories facilitate its integration with robotic arms and prostheses for challenging tasks, including picking grapes, opening zippers, folding clothes, and turning pages. This work showcases soft grippers excelling in extreme scenarios with potential applications in agriculture, food processing, prosthesis, biomedicine, minimally invasive surgeries, and deep-sea exploration.
more »
« less
This content will become publicly available on November 22, 2025
Leveraging Dexterous Picking Skills for Complex Multi-Object Scenes
This work focuses on the problem of robotic picking in challenging multi-object scenarios. These scenarios include difficult-to-pick objects (e.g., too small, too flat objects) and challenging conditions (e.g., objects obstructed by other objects and/or the environment). To solve these challenges, we leverage four dexterous picking skills inspired by human manipulation techniques and propose methods based on deep neural networks that predict when and how to apply the skills based on the shape of the objects, their relative locations to each other, and the environmental factors. We utilize a compliant, under-actuated hand to reliably apply the identified skills in an open-loop manner. The capabilities of the proposed system are evaluated through a series of real-world experiments, comprising 45 trials with 150+ grasps, to assess its reliability and robustness, particularly in cluttered settings. The videos of all experiments are provided at https://dexterouspicking.wpi.edu/. This research helps bridge the gap between human and robotic grasping, showcasing promising results in various practical scenarios.
more »
« less
- Award ID(s):
- 2338703
- PAR ID:
- 10613923
- Publisher / Repository:
- IEEE
- Date Published:
- ISSN:
- 2164-0580
- ISBN:
- 979-8-3503-7357-8
- Page Range / eLocation ID:
- 250 to 257
- Subject(s) / Keyword(s):
- grasping dexterous manipulation underactuated hands deep learning
- Format(s):
- Medium: X
- Location:
- Nancy, France
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Picking an item in the presence of other objects can be challenging as it involves occlusions and partial views. Given object models, one approach is to perform object pose estimation and use the most likely candidate pose per object to pick the target without collisions. This approach, however, ignores the uncertainty of the perception process both regarding the target’s and the surrounding objects’ poses. This work proposes first a perception process for 6D pose estimation, which returns a discrete distribution of object poses in a scene. Then, an open-loop planning pipeline is proposed to return safe and effective solutions for moving a robotic arm to pick, which (a) minimizes the probability of collision with the obstructing objects; and (b) maximizes the probability of reaching the target item. The planning framework models the challenge as a stochastic variant of the Minimum Constraint Removal (MCR) problem. The effectiveness of the methodology is verified given both simulated and real data in different scenarios. The experiments demonstrate the importance of considering the uncertainty of the perception process in terms of safe execution. The results also show that the methodology is more effective than conservative MCR approaches, which avoid all possible object poses regardless of the reported uncertainty.more » « less
-
Madden, John D.; Anderson, Iain A.; Shea, Herbert R. (Ed.)Ras Labs makes Synthetic Muscle™, which is a class of electroactive polymer (EAP) based materials and actuators that sense pressure (gentle touch to high impact), controllably contract and expand at low voltage (1.5 V to 50 V, including use of batteries), and attenuate force. We are in the robotics era, but robots do have their challenges. Currently, robotic sensing is mainly visual, which is useful up until the point of contact. To understand how an object is being gripped, tactile feedback is needed. For handling fragile objects, if the grip is too tight, breakage occurs, and if the grip is too loose, the object will slip out of the grasp, also leading to breakage. Rigid robotic grippers using a visual feedback loop can struggle to determine the exact point and quality of contact. Robotic grippers can also get a stuttering effect in the visual feedback loop. By using soft Synthetic Muscle™ based EAP pads as the sensors, immediate feedback was generated at the first point of contact. Because these pads provided a soft, compliant interface, the first point of contact did not apply excessive force, allowing the force applied to the object to be controlled. The EAP sensor could also detect a change in pressure location on its surface, making it possible to detect and prevent slippage by then adjusting the grip strength. In other words, directional glide provided feedback for the presence of possible slippage to then be able to control a slightly tighter grip, without stutter, due to both the feedback and the soft gentleness of the fingertip-like EAP pads themselves. The soft nature of the EAP fingertip pad also naturally held the gripped object, improving the gripping quality over rigid grippers without an increase in applied force. Analogous to finger-like tactile touch, the EAPs with appropriate coatings and electronics were positioned as pressure sensors in the fingertip or end effector regions of robotic grippers. This development of using Synthetic Muscle™ based EAPs as soft sensors provided for sensors that feel like the pads of human fingertips. Basic pressure position and magnitude tests have been successful, with pressure sensitivity down to 0.05 N. Most automation and robots are very strong, very fast, and usually need to be partitioned away from humans for safety reasons. For many repetitive tasks that humans do with delicate or fragile objects, it would be beneficial to use robotics; whether it is for agriculture, medical surgery, therapeutic or personal care, or in extreme environments where humans cannot enter, including with contagions that have no cure. Synthetic Muscle™ was also retrofitted as actuator systems into off-the-shelf robotic grippers and is being considered in novel biomimetic gripper designs, operating at low voltages (less than 50 V). This offers biomimetic movement by contracting like human muscles, but also exceeds natural biological capabilities by expanding under reversed electric polarity. Human grasp is gentle yet firm, with tactile touch feedback. In conjunction with shape-morphing abilities, these EAPs also are being explored to intrinsically sense pressure due to the correlation between mechanical force applied to the EAP and its electronic signature. The robotic field is experiencing phenomenal growth in this fourth phase of the industrial revolution, the robotics era. The combination of Ras Labs’ EAP shape-morphing and sensing features promises the potential for robotic grippers with human hand-like control and tactile sensing. This work is expected to advance both robotics and prosthetics, particularly for collaborative robotics to allow humans and robots to intuitively work safely and effectively together.more » « less
-
In this paper, we introduce the Warehouse Augmented Reality Program (WARP), its functionality, practicality, and potential use cases in education. We build this application on the backbone of WebXR. Using this application programming interface (API), we create an interactive web tool that displays a life-sized warehouse in augmented reality (AR) in front of users that can be viewed on a smartphone or a tablet. AR is a technology that displays virtual objects in the real world on a digital device’s screen, allowing users to interact with virtual objects and locations while moving about a real-world environment. This tool can enhance warehousing education by making it immersive and more interactive. In addition, the tool can make warehousing operations more efficient and warehouse design less costly. We highlight how our tool can be applicable and beneficial to education and industry. We demonstrate how this tool can be integrated into a problem-based learning (PBL) assignment about warehouse layout design and order picking. The PBL activity involves comparing two different warehouse layouts (fishbone and traditional) by completing a set of order picking tasks in AR warehouse environments. The task is to perform single item picking over thirty orders and comparing the average order picking time per layout. Then, we use the results of these human subject experiments for validating the realism of the warehouse layouts generated by the tool by comparing the empirical completion times with the analytical results from the literature. We also administer a system usability scale (SUS) survey and collect feedback from industry experts.more » « less
-
null (Ed.)Although general purpose robotic manipulators are becoming more capable at manipulating various objects, their ability to manipulate millimeter-scale objects are usually limited. On the other hand, ultrasonic levitation devices have been shown to levitate a large range of small objects, from polystyrene balls to living organisms. By controlling the acoustic force fields, ultrasonic levitation devices can compensate for robot manipulator positioning uncertainty and control the grasping force exerted on the target object. The material agnostic nature of acoustic levitation devices and their ability to dexterously manipulate millimeter-scale objects make them appealing as a grasping mode for general purpose robots. In this work, we present an ultrasonic, contact-less manipulation device that can be attached to or picked up by any general purpose robotic arm, enabling millimeter-scale manipulation with little to no modification to the robot itself. This device is capable of performing the very first phase-controlled picking action on acoustically reflective surfaces. With the manipulator placed around the target object, the manipulator can grasp objects smaller in size than the robot's positioning uncertainty, trap the object to resist air currents during robot movement, and dexterously hold a small and fragile object, like a flower bud. Due to the contact-less nature of the ultrasound-based gripper, a camera positioned to look into the cylinder can inspect the object without occlusion, facilitating accurate visual feature extraction.more » « less
