More Like this

1. A kinematic and EMG dataset of online adjustment of reach-to-grasp movements to visual perturbations

   https://doi.org/doi.org/10.1038/s41597-021-01107-2  

   Furmanek, Mariusz P. ; Mangalam, Madhur ; Yarossi, Mathew ; Lockwood, Kyle ; Tunik, Eugene ( January 2022 , Scientific data)

   Control of reach-to-grasp movements for deft and robust interactions with objects requires rapid sensorimotor updating that enables online adjustments to changing external goals (e.g., perturbations or instability of objects we interact with). Rarely do we appreciate the remarkable coordination in reach-to-grasp, until control becomes impaired by neurological injuries such as stroke, neurodegenerative diseases, or even aging. Modeling online control of human reach-to-grasp movements is a challenging problem but fundamental to several domains, including behavioral and computational neuroscience, neurorehabilitation, neural prostheses, and robotics. Currently, there are no publicly available datasets that include online adjustment of reach-to-grasp movements to object perturbations. This work aims to advance modeling efforts of reach-to-grasp movements by making publicly available a large kinematic and EMG dataset of online adjustment of reach-to-grasp movements to instantaneous perturbations of object size and distance performed in immersive haptic-free virtual environment (hf-VE). The presented dataset is composed of a large number of perturbation types (10 for both object size and distance) applied at three different latencies after the start of the movement. 

   more » « less
2. Effects of Sensory Feedback and Collider Size on Reach-to-Grasp Coordination in Haptic-Free Virtual Reality

   https://doi.org/10.3389/frvir.2021.648529  

   Furmanek, Mariusz P. ; Mangalam, Madhur ; Lockwood, Kyle ; Smith, Andrea ; Yarossi, Mathew ; Tunik, Eugene ( August 2021 , Frontiers in Virtual Reality)

   null (Ed.)

   Technological advancements and increased access have prompted the adoption of head- mounted display based virtual reality (VR) for neuroscientific research, manual skill training, and neurological rehabilitation. Applications that focus on manual interaction within the virtual environment (VE), especially haptic-free VR, critically depend on virtual hand-object collision detection. Knowledge about how multisensory integration related to hand-object collisions affects perception-action dynamics and reach-to-grasp coordination is needed to enhance the immersiveness of interactive VR. Here, we explored whether and to what extent sensory substitution for haptic feedback of hand-object collision (visual, audio, or audiovisual) and collider size (size of spherical pointers representing the fingertips) influences reach-to-grasp kinematics. In Study 1, visual, auditory, or combined feedback were compared as sensory substitutes to indicate the successful grasp of a virtual object during reach-to-grasp actions. In Study 2, participants reached to grasp virtual objects using spherical colliders of different diameters to test if virtual collider size impacts reach-to-grasp. Our data indicate that collider size but not sensory feedback modality significantly affected the kinematics of grasping. Larger colliders led to a smaller size-normalized peak aperture. We discuss this finding in the context of a possible influence of spherical collider size on the perception of the virtual object’s size and hence effects on motor planning of reach-to-grasp. Critically, reach-to-grasp spatiotemporal coordination patterns were robust to manipulations of sensory feedback modality and spherical collider size, suggesting that the nervous system adjusted the reach (transport) component commensurately to the changes in the grasp (aperture) component. These results have important implications for research, commercial, industrial, and clinical applications of VR. 

   more » « less
3. The Platyrrhine Primate Cebus imitator Uses Gaze to Manipulate and Withdraw Food to the Mouth

   https://doi.org/10.26451/abc.11.01.01.2024  

   Whishaw, Ian ; Mah, Megan ; Casorso, Julia ; Chacon, Evin ; Chalk-Wilayto, Janine ; Laird, Myra ; Melin, Amanda ( February 2024 , Animal Behavior and Cognition)

   Orienting a food item held in the hand to withdraw and optimally place it in the mouth for eating (withdraw-to-eat) is mediated by vision in catarrhine anthropoids and by nonvisual strategies in strepsirrhine primates. The present study asks whether vision contributes to the withdraw-to-eat movements in a platyrrhine anthropoid Cebus imitator, a member of a monophyletic primate suborder whose stem group diverged from catarrhines about 40 million years ago. Cebus imitator’s gaze and hand use for foraging for fruit is examined in its fine branch niche, the terminal branches of trees. Video of reach, grasp and withdraw-to-eat movements with associated gaze were examined frame-by-frame to assess food manipulation and its sensory control. Cebus imitator uses vision and touch to reach for and grasp food items with precision and whole hand grasps. They use vision to orient food items held in-hand into a precision grip and their withdraw-to-eat is assisted with a vertically oriented hand. The conjoint use of vision, a precision grasp, and hand posture and a central representation of object control likely originated in stem anthropoids and was derived from the staged evolution of the visual manipulation of food and other objects.

    

   more » « less
4. The Grasp Reset Mechanism: An Automated Apparatus for Conducting Grasping Trials

   DuFrene, Kyle ; Nave, Keegan ; Campbell, Joshua ; Balasubramanian, Ravi ; Grimm, Cindy ( May 2024 , IEEE)

   Hasegawa, Yasuhisa (Ed.)

   Advancing robotic grasping and manipulation requires the ability to test algorithms and/or train learning models on large numbers of grasps. Towards the goal of more advanced grasping, we present the Grasp Reset Mechanism (GRM), a fully automated apparatus for conducting large-scale grasping trials. The GRM automates the process of resetting a grasping environment, repeatably placing an object in a fixed location and controllable 1-D orientation. It also collects data and swaps between multiple objects enabling robust dataset collection with no human intervention. We also present a standardized state machine interface for control, which allows for integration of most manipulators with minimal effort. In addition to the physical design and corresponding software, we include a dataset of 1,020 grasps. The grasps were created with a Kinova Gen3 robot arm and Robotiq 2F-85 Adaptive Gripper to enable training of learning models and to demonstrate the capabilities of the GRM. The dataset includes ranges of grasps conducted across four objects and a variety of orientations. Manipulator states, object pose, video, and grasp success data are provided for every trial. 

   more » « less
5. Dex-Net 3.0: Computing Robust Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning

   Mahler, J. ; Matl, M. ; Liu, X. ; Li, A. ; Gealy, D. ; Goldberg, K. ( January 2018 , IEEE International Conference on Robotics and Automation)

   Vacuum-based end effectors are widely used in in- dustry and are often preferred over parallel-jaw and multifinger grippers due to their ability to lift objects with a single point of contact. Suction grasp planners often target planar surfaces on point clouds near the estimated centroid of an object. In this paper, we propose a compliant suction contact model that computes the quality of the seal between the suction cup and local target surface and a measure of the ability of the suction grasp to resist an external gravity wrench. To characterize grasps, we estimate robustness to perturbations in end-effector and object pose, material properties, and external wrenches. We analyze grasps across 1,500 3D object models to generate Dex- Net 3.0, a dataset of 2.8 million point clouds, suction grasps, and grasp robustness labels. We use Dex-Net 3.0 to train a Grasp Quality Convolutional Neural Network (GQ-CNN) to classify robust suction targets in point clouds containing a single object. We evaluate the resulting system in 350 physical trials on an ABB YuMi fitted with a pneumatic suction gripper. When eval- uated on novel objects that we categorize as Basic (prismatic or cylindrical), Typical (more complex geometry), and Adversarial (with few available suction-grasp points) Dex-Net 3.0 achieves success rates of 98%, 82%, and 58% respectively, improving to 81% in the latter case when the training set includes only adversarial objects. Code, datasets, and supplemental material can be found at http://berkeleyautomation.github.io/dex-net. 

   more » « less