In this work, we investigate the influence of different visualizations on a manipulation task in virtual reality (VR). Without the haptic feedback of the real world, grasping in VR might result in intersections with virtual objects. As people are highly sensitive when it comes to perceiving collisions, it might look more appealing to avoid intersections and visualize non-colliding hand motions. However, correcting the position of the hand or fingers results in a visual-proprioceptive discrepancy and must be used with caution. Furthermore, the lack of haptic feedback in the virtual world might result in slower actions as a user might not know exactly when a grasp has occurred. This reduced performance could be remediated with adequate visual feedback.
In this study, we analyze the performance, level of ownership, and user preference of eight different visual feedback techniques for virtual grasping. Three techniques show the tracked hand (with or without grasping feedback), even if it intersects with the grasped object. Another three techniques display a hand without intersections with the object, called outer hand, simulating the look of a real world interaction. One visualization is a compromise between the two groups, showing both a primary outer hand and a secondary tracked hand. Finally, in the last visualization the hand disappears during the grasping activity.
In an experiment, users perform a pick-and-place task for each feedback technique. We use high fidelity marker-based hand tracking to control the virtual hands in real time. We found that the tracked hand visualizations result in better performance, however, the outer hand visualizations were preferred. We also find indications that ownership is higher with the outer hand visualizations.
more »
« less
The complexities of grasping in the wild
The recent ubiquity of high-framerate (120 fps and higher) handheld cameras creates the opportunity to study human grasping at a greater level of detail than normal speed cameras allow. We first collected 91 slow-motion interactions with objects in a convenience store setting. We then annotated the actions through the lenses of various existing manipulation taxonomies. We found manipulation, particularly the process of forming a grasp, is complicated and proceeds quickly. Our dataset shows that there are many ways that people deal with clutter in order to form a strong grasp of an object. It also reveals several errors and how people recover from them. Though annotating motions in detail is time-consuming, the annotation systems we used nevertheless leave out important aspects of understanding manipulation actions, such as how the environment is functioning as a “finger” of sorts, how different parts of the hand can be involved in different grasping tasks, and high-level intent.
more »
« less
- Award ID(s):
- 1637853
- NSF-PAR ID:
- 10059450
- Date Published:
- Journal Name:
- IEEE-RAS International Conference on Humanoid Robotics (Humanoids)
- Page Range / eLocation ID:
- 233 to 240
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Robotic grasping can enable mobile vehicles to physically interact with the environment for delivery, repositioning, or landing. However, the requirements for grippers on mobile vehicles differ substantially from those used for conventional manipulation. Specifically, grippers for dynamic mobile robots should be capable of rapid activation, high force density, low power consumption, and minimal computation. In this work, we present a biologically-inspired robotic gripper designed specifically for mobile platforms. This design exploits a bistable shell to achieve “reflexive” activation based on contact with the environment. The mechanism can close its grasp within 0. 12s without any sensing or control. Electrical input power is not required for grasping or holding load. The reflexive gripper utilizes a novel pneumatic design to open its grasp with low power, and the gripper can carry slung loads up to 28 times its weight. This new mechanism, including the kinematics, static behavior, control structure, and fabrication, is described in detail. A proof of concept prototype is designed, built, and tested. Experimental results are used to characterize performance and demonstrate the potential of these methods.more » « less
-
null (Ed.)Abstract Successful object manipulation, such as preventing object roll, relies on the modulation of forces and centers of pressure (point of application of digits on each grasp surface) prior to lift onset to generate a compensatory torque. Whether or not generalization of learned manipulation can occur after adding or removing effectors is not known. We examined this by recruiting participants to perform lifts in unimanual and bimanual grasps and analyzed results before and after transfer. Our results show partial generalization of learned manipulation occurred when switching from a (1) unimanual to bimanual grasp regardless of object center of mass, and (2) bimanual to unimanual grasp when the center of mass was on the thumb side. Partial generalization was driven by the modulation of effectors’ center of pressure, in the appropriate direction but of insufficient magnitude, while load forces did not contribute to torque generation after transfer. In addition, we show that the combination of effector forces and centers of pressure in the generation of compensatory torque differ between unimanual and bimanual grasping. These findings highlight that (1) high-level representations of learned manipulation enable only partial learning transfer when adding or removing effectors, and (2) such partial generalization is mainly driven by modulation of effectors’ center of pressure.more » « less
-
null (Ed.)Technological advancements and increased access have prompted the adoption of head- mounted display based virtual reality (VR) for neuroscientific research, manual skill training, and neurological rehabilitation. Applications that focus on manual interaction within the virtual environment (VE), especially haptic-free VR, critically depend on virtual hand-object collision detection. Knowledge about how multisensory integration related to hand-object collisions affects perception-action dynamics and reach-to-grasp coordination is needed to enhance the immersiveness of interactive VR. Here, we explored whether and to what extent sensory substitution for haptic feedback of hand-object collision (visual, audio, or audiovisual) and collider size (size of spherical pointers representing the fingertips) influences reach-to-grasp kinematics. In Study 1, visual, auditory, or combined feedback were compared as sensory substitutes to indicate the successful grasp of a virtual object during reach-to-grasp actions. In Study 2, participants reached to grasp virtual objects using spherical colliders of different diameters to test if virtual collider size impacts reach-to-grasp. Our data indicate that collider size but not sensory feedback modality significantly affected the kinematics of grasping. Larger colliders led to a smaller size-normalized peak aperture. We discuss this finding in the context of a possible influence of spherical collider size on the perception of the virtual object’s size and hence effects on motor planning of reach-to-grasp. Critically, reach-to-grasp spatiotemporal coordination patterns were robust to manipulations of sensory feedback modality and spherical collider size, suggesting that the nervous system adjusted the reach (transport) component commensurately to the changes in the grasp (aperture) component. These results have important implications for research, commercial, industrial, and clinical applications of VR.more » « less
-
Task-relevant grasping is critical for industrial assembly, where downstream manipulation tasks constrain the set of valid grasps. Learning how to perform this task, however, is challenging, since task-relevant grasp labels are hard to define and annotate. There is also yet no consensus on proper representations for modeling or off-the-shelf tools for performing task-relevant grasps. This work proposes a framework to learn task-relevant grasping for industrial objects without the need of time-consuming real-world data collection or manual annotation. To achieve this, the entire framework is trained solely in simulation, including supervised training with synthetic label generation and self-supervised, hand-object interaction. In the context of this framework, this paper proposes a novel, object-centric canonical representation at the category level, which allows establishing dense correspondence across object instances and transferring task-relevant grasps to novel instances. Extensive experiments on task-relevant grasping of densely-cluttered industrial objects are conducted in both simulation and real-world setups, demonstrating the effectiveness of the proposed framework.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/HUMANOIDS.2017.8246880
