skip to main content

Explore Research Products in the NSF-PAR

Title: Grasp Taxonomy for Robot Assistants Inferred from Finger Pressure and Flexion
Grasp is an integral part of manipulation actions in activities of daily living and programming by demonstration is a powerful paradigm for teaching the assistive robots how to perform a grasp. Since finger configuration and finger force are the fundamental features that need to be controlled during a grasp, using these variables is a natural choice for learning by demonstration. An important question then becomes whether the existing grasp taxonomies are appropriate when one considers these modalities. The goal of our paper is to answer this question by investigating grasp patterns that can be inferred from a static analysis of the grasp data, as the object is securely grasped. Human grasp data is measured using a newly developed data glove. The data includes pressure sensor measurements from eighteen areas of the hand, and measurements from bend sensors placed at finger joints. The pressure sensor measurements are calibrated and mapped into force by employing a novel data-driven approach. Unsupervised learning is used to identify patterns for different grasp types. Multiple clustering algorithms are used to partition the data. When the results are taken in aggregate, 25 human grasp types are reduced to 9 different clusters.  more » « less
Award ID(s):
1762924 1705058
NSF-PAR ID:
10096225
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2019 International Symposium on Medical Robotics (ISMR)
Page Range / eLocation ID:
1 to 7
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ( , Proc. SPIE 11587, Electroactive Polymer Actuators and Devices (EAPAD) XXIII,)
    Madden, John D. ; Anderson, Iain A. ; Shea, Herbert R. (Ed.)
    Ras Labs makes Synthetic Muscle™, which is a class of electroactive polymer (EAP) based materials and actuators that sense pressure (gentle touch to high impact), controllably contract and expand at low voltage (1.5 V to 50 V, including use of batteries), and attenuate force. We are in the robotics era, but robots do have their challenges. Currently, robotic sensing is mainly visual, which is useful up until the point of contact. To understand how an object is being gripped, tactile feedback is needed. For handling fragile objects, if the grip is too tight, breakage occurs, and if the grip is too loose, the object will slip out of the grasp, also leading to breakage. Rigid robotic grippers using a visual feedback loop can struggle to determine the exact point and quality of contact. Robotic grippers can also get a stuttering effect in the visual feedback loop. By using soft Synthetic Muscle™ based EAP pads as the sensors, immediate feedback was generated at the first point of contact. Because these pads provided a soft, compliant interface, the first point of contact did not apply excessive force, allowing the force applied to the object to be controlled. The EAP sensor could also detect a change in pressure location on its surface, making it possible to detect and prevent slippage by then adjusting the grip strength. In other words, directional glide provided feedback for the presence of possible slippage to then be able to control a slightly tighter grip, without stutter, due to both the feedback and the soft gentleness of the fingertip-like EAP pads themselves. The soft nature of the EAP fingertip pad also naturally held the gripped object, improving the gripping quality over rigid grippers without an increase in applied force. Analogous to finger-like tactile touch, the EAPs with appropriate coatings and electronics were positioned as pressure sensors in the fingertip or end effector regions of robotic grippers. This development of using Synthetic Muscle™ based EAPs as soft sensors provided for sensors that feel like the pads of human fingertips. Basic pressure position and magnitude tests have been successful, with pressure sensitivity down to 0.05 N. Most automation and robots are very strong, very fast, and usually need to be partitioned away from humans for safety reasons. For many repetitive tasks that humans do with delicate or fragile objects, it would be beneficial to use robotics; whether it is for agriculture, medical surgery, therapeutic or personal care, or in extreme environments where humans cannot enter, including with contagions that have no cure. Synthetic Muscle™ was also retrofitted as actuator systems into off-the-shelf robotic grippers and is being considered in novel biomimetic gripper designs, operating at low voltages (less than 50 V). This offers biomimetic movement by contracting like human muscles, but also exceeds natural biological capabilities by expanding under reversed electric polarity. Human grasp is gentle yet firm, with tactile touch feedback. In conjunction with shape-morphing abilities, these EAPs also are being explored to intrinsically sense pressure due to the correlation between mechanical force applied to the EAP and its electronic signature. The robotic field is experiencing phenomenal growth in this fourth phase of the industrial revolution, the robotics era. The combination of Ras Labs’ EAP shape-morphing and sensing features promises the potential for robotic grippers with human hand-like control and tactile sensing. This work is expected to advance both robotics and prosthetics, particularly for collaborative robotics to allow humans and robots to intuitively work safely and effectively together. 
    more » « less
  2. ; ; ( , Journal of Mechanisms and Robotics)
    Abstract Soft robotic grippers can gently grasp and maneuver objects. However, they are difficult to model and control due to their highly deformable fingers and complex integration with robotic systems. This paper investigates the design requirements as well as the grasping capabilities and performance of a soft gripper system based on fluidic prestressed composite (FPC) fingers. An analytical model is constructed as follows: each finger is modeled using the chained composite model (CCM); strain energy and work done by pressure and loads are computed using polynomials with unknown coefficients; net energy is minimized using the Rayleigh–Ritz method to calculate the deflected equilibrium shapes of the finger as a function of pressure and loads; and coordinate transformation and gripper geometries are combined to analyze the grasping performance. The effects of prestrain, integration angle, and finger overlap on the grasping performance are examined through a parametric study. We also analyze gripping performance for cuboidal and spherical objects and show how the grasping force can be controlled by varying fluidic pressure. The quasi-static responses of fabricated actuators are measured under pressures and loads. It is shown that the actuators’ modeled responses agree with the experimental results. This work provides a framework for the theoretical analysis of soft robotic grippers and the methods presented can be extended to model grippers with different types of actuation. 
    more » « less
  3. ; ; ; ; ; ( , IEEE International Conference on Robotics and Automation)
    This paper presents an online data collection method that captures human intuition about what grasp types are preferred for different fundamental object shapes and sizes. Survey questions are based on an adopted taxonomy that combines grasp pre-shape, approach, wrist orientation, object shape, orientation and size which covers a large swathe of common grasps. For example, the survey identifies at what object height or width dimension (normalized by robot hand size) the human prefers to use a two finger precision grasp versus a three-finger power grasp. This information is represented as a confidence-interval based polytope in the object shape space. The result is a database that can be used to quickly find potential pre-grasps that are likely to work, given an estimate of the object shape and size. 
    more » « less
  4. ( , Databrary)
    The PoseASL dataset consists of color and depth videos collected from ASL signers at the Linguistic and Assistive Technologies Laboratory under the direction of Matt Huenerfauth, as part of a collaborative research project with researchers at the Rochester Institute of Technology, Boston University, and the University of Pennsylvania. Access: After becoming an authorized user of Databrary, please contact Matt Huenerfauth if you have difficulty accessing this volume. We have collected a new dataset consisting of color and depth videos of fluent American Sign Language signers performing sequences ASL signs and sentences. Given interest among sign-recognition and other computer-vision researchers in red-green-blue-depth (RBGD) video, we release this dataset for use by the research community. In addition to the video files, we share depth data files from a Kinect v2 sensor, as well as additional motion-tracking files produced through post-processing of this data. Organization of the Dataset: The dataset is organized into sub-folders, with codenames such as "P01" or "P16" etc. These codenames refer to specific human signers who were recorded in this dataset. Please note that there was no participant P11 nor P14; those numbers were accidentally skipped during the process of making appointments to collect video stimuli. Task: During the recording session, the participant was met by a member of our research team who was a native ASL signer. No other individuals were present during the data collection session. After signing the informed consent and video release document, participants responded to a demographic questionnaire. Next, the data-collection session consisted of English word stimuli and cartoon videos. The recording session began with showing participants stimuli consisting of slides that displayed English word and photos of items, and participants were asked to produce the sign for each (PDF included in materials subfolder). Next, participants viewed three videos of short animated cartoons, which they were asked to recount in ASL: - Canary Row, Warner Brothers Merrie Melodies 1950 (the 7-minute video divided into seven parts) - Mr. Koumal Flies Like a Bird, Studio Animovaneho Filmu 1969 - Mr. Koumal Battles his Conscience, Studio Animovaneho Filmu 1971 The word list and cartoons were selected as they are identical to the stimuli used in the collection of the Nicaraguan Sign Language video corpora - see: Senghas, A. (1995). Children’s Contribution to the Birth of Nicaraguan Sign Language. Doctoral dissertation, Department of Brain and Cognitive Sciences, MIT. Demographics: All 14 of our participants were fluent ASL signers. As screening, we asked our participants: Did you use ASL at home growing up, or did you attend a school as a very young child where you used ASL? All the participants responded affirmatively to this question. A total of 14 DHH participants were recruited on the Rochester Institute of Technology campus. Participants included 7 men and 7 women, aged 21 to 35 (median = 23.5). All of our participants reported that they began using ASL when they were 5 years old or younger, with 8 reporting ASL use since birth, and 3 others reporting ASL use since age 18 months. Filetypes: *.avi, *_dep.bin: The PoseASL dataset has been captured by using a Kinect 2.0 RGBD camera. The output of this camera system includes multiple channels which include RGB, depth, skeleton joints (25 joints for every video frame), and HD face (1,347 points). The video resolution produced in 1920 x 1080 pixels for the RGB channel and 512 x 424 pixels for the depth channels respectively. Due to limitations in the acceptable filetypes for sharing on Databrary, it was not permitted to share binary *_dep.bin files directly produced by the Kinect v2 camera system on the Databrary platform. If your research requires the original binary *_dep.bin files, then please contact Matt Huenerfauth. *_face.txt, *_HDface.txt, *_skl.txt: To make it easier for future researchers to make use of this dataset, we have also performed some post-processing of the Kinect data. To extract the skeleton coordinates of the RGB videos, we used the Openpose system, which is capable of detecting body, hand, facial, and foot keypoints of multiple people on single images in real time. The output of Openpose includes estimation of 70 keypoints for the face including eyes, eyebrows, nose, mouth and face contour. The software also estimates 21 keypoints for each of the hands (Simon et al, 2017), including 3 keypoints for each finger, as shown in Figure 2. Additionally, there are 25 keypoints estimated for the body pose (and feet) (Cao et al, 2017; Wei et al, 2016). Reporting Bugs or Errors: Please contact Matt Huenerfauth to report any bugs or errors that you identify in the corpus. We appreciate your help in improving the quality of the corpus over time by identifying any errors. Acknowledgement: This material is based upon work supported by the National Science Foundation under award 1749376: "Collaborative Research: Multimethod Investigation of Articulatory and Perceptual Constraints on Natural Language Evolution." 
    more » « less
  5. ; ; ; ; ; ; ; ; ( , Frontiers in Robotics and AI)
    null (Ed.)
    Current designs of powered prosthetic limbs are limited by the nearly exclusive use of DC motor technology. Soft actuators promise new design freedom to create prosthetic limbs which more closely mimic intact neuromuscular systems and improve the capabilities of prosthetic users. This work evaluates the performance of a hydraulically amplified self-healing electrostatic (HASEL) soft actuator for use in a prosthetic hand. We compare a linearly-contracting HASEL actuator, termed a Peano-HASEL, to an existing actuator (DC motor) when driving a prosthetic finger like those utilized in multi-functional prosthetic hands. A kinematic model of the prosthetic finger is developed and validated, and is used to customize a prosthetic finger that is tuned to complement the force-strain characteristics of the Peano-HASEL actuators. An analytical model is used to inform the design of an improved Peano-HASEL actuator with the goal of increasing the fingertip pinch force of the prosthetic finger. When compared to a weight-matched DC motor actuator, the Peano-HASEL and custom finger is 10.6 times faster, has 11.1 times higher bandwidth, and consumes 8.7 times less electrical energy to grasp. It reaches 91% of the maximum range of motion of the original finger. However, the DC motor actuator produces 10 times the fingertip force at a relevant grip position. In this body of work, we present ways to further increase the force output of the Peano-HASEL driven prosthetic finger system, and discuss the significance of the unique properties of Peano-HASELs when applied to the field of upper-limb prosthetic design. This approach toward clinically-relevant actuator performance paired with a substantially different form-factor compared to DC motors presents new opportunities to advance the field of prosthetic limb design. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email