Abstract Humans are adept at a wide variety of motor skills, including the handling of complex objects and using tools. Advances to understand the control of voluntary goal-directed movements have focused on simple behaviors such as reaching, uncoupled to any additional object dynamics. Under these simplified conditions, basic elements of motor control, such as the roles of body mechanics, objective functions, and sensory feedback, have been characterized. However, these elements have mostly been examined in isolation, and the interactions between these elements have received less attention. This study examined a task with internal dynamics, inspired by the daily skill of transporting a cup of coffee, with additional expected or unexpected perturbations to probe the structure of the controller. Using optimal feedback control (OFC) as the basis, it proved necessary to endow the model of the body with mechanical impedance to generate the kinematic features observed in the human experimental data. The addition of mechanical impedance revealed that simulated movements were no longer sensitively dependent on the objective function, a highly debated cornerstone of optimal control. Further, feedforward replay of the control inputs was similarly successful in coping with perturbations as when feedback, or sensory information, was included. These findings suggest that when the control model incorporates a representation of the mechanical properties of the limb, that is, embodies its dynamics, the specific objective function and sensory feedback become less critical, and complex interactions with dynamic objects can be successfully managed.
more »
« less
This content will become publicly available on May 29, 2024
Multi-modal Interactive Perception in Human Control of Complex Objects
Tactile sensing has been increasingly utilized in robot
control of unknown objects to infer physical properties and
optimize manipulation. However, there is limited understanding
about the contribution of different sensory modalities during
interactive perception in complex interaction both in robots
and in humans. This study investigated the effect of visual and
haptic information on humans’ exploratory interactions with
a ‘cup of coffee’, an object with nonlinear internal dynamics.
Subjects were instructed to rhythmically transport a virtual
cup with a rolling ball inside between two targets at a specified
frequency, using a robotic interface. The cup and targets were
displayed on a screen, and force feedback from the cup-andball
dynamics was provided via the robotic manipulandum.
Subjects were encouraged to explore and prepare the dynamics
by “shaking” the cup-and-ball system to find the best initial
conditions prior to the task. Two groups of subjects received the
full haptic feedback about the cup-and-ball movement during
the task; however, for one group the ball movement was visually
occluded. Visual information about the ball movement had two
distinctive effects on the performance: it reduced preparation
time needed to understand the dynamics and, importantly, it
led to simpler, more linear input-output interactions between
hand and object. The results highlight how visual and haptic information
regarding nonlinear internal dynamics have distinct
roles for the interactive perception of complex objects.
more »
« less
- Award ID(s):
- 1825942
- NSF-PAR ID:
- 10465773
- Date Published:
- Journal Name:
- IEEE International Conference on Robotics and Automation (ICRA 2023),
- Page Range / eLocation ID:
- 12500 to 12506
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Technological advancements and increased access have prompted the adoption of head- mounted display based virtual reality (VR) for neuroscientific research, manual skill training, and neurological rehabilitation. Applications that focus on manual interaction within the virtual environment (VE), especially haptic-free VR, critically depend on virtual hand-object collision detection. Knowledge about how multisensory integration related to hand-object collisions affects perception-action dynamics and reach-to-grasp coordination is needed to enhance the immersiveness of interactive VR. Here, we explored whether and to what extent sensory substitution for haptic feedback of hand-object collision (visual, audio, or audiovisual) and collider size (size of spherical pointers representing the fingertips) influences reach-to-grasp kinematics. In Study 1, visual, auditory, or combined feedback were compared as sensory substitutes to indicate the successful grasp of a virtual object during reach-to-grasp actions. In Study 2, participants reached to grasp virtual objects using spherical colliders of different diameters to test if virtual collider size impacts reach-to-grasp. Our data indicate that collider size but not sensory feedback modality significantly affected the kinematics of grasping. Larger colliders led to a smaller size-normalized peak aperture. We discuss this finding in the context of a possible influence of spherical collider size on the perception of the virtual object’s size and hence effects on motor planning of reach-to-grasp. Critically, reach-to-grasp spatiotemporal coordination patterns were robust to manipulations of sensory feedback modality and spherical collider size, suggesting that the nervous system adjusted the reach (transport) component commensurately to the changes in the grasp (aperture) component. These results have important implications for research, commercial, industrial, and clinical applications of VR.more » « less
-
This study examines human control of physical interaction with objects that exhibit complex (nonlinear, chaotic, underactuated) dynamics. We hypothesized that humans exploited stability properties of the human-object interaction. Using a simplified 2D model for carrying a “cup of coffee”, we developed a virtual implementation to identify human control strategies. Transporting a cup of coffee was modeled as a cart with a suspended pendulum, where humans moved the cart on a horizontal line via a robotic manipulandum. The specific task was to transport the cart-pendulum system to a target, as fast as possible, while accommodating assistive and resistive perturbations. To assess trajectory stability, we applied contraction analysis. We showed that when the perturbation was assistive, humans absorbed the perturbation by controlling cart trajectories into a contraction region prior to the perturbation. When the perturbation was resistive, subjects passed through a contraction region following the perturbation. Entering a contraction region stabilizes performance and makes the dynamics more predictable. This human control strategy could inspire more robust control strategies for physical interaction in robots.more » « less
-
This study examines human control of physical interaction with objects that exhibit complex (nonlinear, chaotic, underactuated) dynamics. We hypothesized that humans exploited stability properties of the human-object interaction. Using a simplified 2D model for carrying a “cup of coffee”, we developed a virtual implementation to identify human control strategies. Transporting a cup of coffee was modeled as a cart with a suspended pendulum, where humans moved the cart on a horizontal line via a robotic manipulandum. The specific task was to transport the cart-pendulum system to a target, as fast as possible, while accommodating assistive and resistive perturbations. To assess trajectory stability, we applied contraction analysis. We showed that when the perturbation was assistive, humans absorbed the perturbation by controlling cart trajectories into a contraction region prior to the perturbation. When the perturbation was resistive, subjects passed through a contraction region following the perturbation. Entering a contraction region stabilizes performance and makes the dynamics more predictable. This human control strategy could inspire more robust control strategies for physical interaction in robots.more » « less
-
Many goal-directed actions that require rapid visuomotor planning and perceptual decision-making are affected in older adults, causing difficulties in execution of many functional activities of daily living. Visuomotor planning and perceptual identification are mediated by the dorsal and ventral visual streams, respectively, but it is unclear how age-induced changes in sensory processing in these streams contribute to declines in visuomotor decision-making performance. Previously, we showed that in young adults, task demands influenced movement strategies during visuomotor decision-making, reflecting differential integration of sensory information between the two streams. Here, we asked the question if older adults would exhibit deficits in interactions between the two streams during demanding motor tasks. Older adults ( n = 15) and young controls ( n = 26) performed reaching or interception movements toward virtual objects. In some blocks of trials, participants also had to select an appropriate movement goal based on the shape of the object. Our results showed that older adults corrected fewer initial decision errors during both reaching and interception movements. During the interception decision task, older adults made more decision- and execution-related errors than young adults, which were related to early initiation of their movements. Together, these results suggest that older adults have a reduced ability to integrate new perceptual information to guide online action, which may reflect impaired ventral-dorsal stream interactions. NEW & NOTEWORTHY Older adults show declines in vision, decision-making, and motor control, which can lead to functional limitations. We used a rapid visuomotor decision task to examine how these deficits may interact to affect task performance. Compared with healthy young adults, older adults made more errors in both decision-making and motor execution, especially when the task required intercepting moving targets. This suggests that age-related declines in integrating perceptual and motor information may contribute to functional deficits.more » « less
