skip to main content

Title: Towards Modeling Human Motor Learning Dynamics in High-Dimensional Spaces
Designing effective rehabilitation strategies for upper extremities, particularly hands and fingers, warrants the need for a computational model of human motor learning. The presence of large degrees of freedom (DoFs) available in these systems makes it difficult to balance the trade-off between learning the full dexterity and accomplishing manipulation goals. The motor learning literature argues that humans use motor synergies to reduce the dimension of control space. Using the low-dimensional space spanned by these synergies, we develop a computational model based on the internal model theory of motor control. We analyze the proposed model in terms of its convergence properties and fit it to the data collected from human experiments. We compare the performance of the fitted model to the experimental data and show that it captures human motor learning behavior well.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
American Control Conference
Page Range / eLocation ID:
683 to 688
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Noises are ubiquitous in sensorimotor interactions and contaminate the information provided to the central nervous system (CNS) for motor learning. An interesting question is how the CNS manages motor learning with imprecise information. Integrating ideas from reinforcement learning and adaptive optimal control, this paper develops a novel computational mechanism to explain the robustness of human motor learning to the imprecise information, caused by control-dependent noise that exists inherently in the sensorimotor systems. Starting from an initial admissible control policy, in each learning trial the mechanism collects and uses the noisy sensory data (caused by the control-dependent noise) to form an imprecise evaluation of the performance of the current policy and then constructs an updated policy based on the imprecise evaluation. As the number of learning trials increases, the generated policies mathematically provably converge to a (potentially small) neighborhood of the optimal policy under mild conditions, despite the imprecise information in the learning process. The mechanism directly synthesizes the policies from the sensory data, without identifying an internal forward model. Our preliminary computational results on two classic arm reaching tasks are in line with experimental observations reported in the literature. The model-free control principle proposed in the paper sheds more lights into the inherent robustness of human sensorimotor systems to the imprecise information, especially control-dependent noise, in the CNS. 
    more » « less
  2. Abstract Background

    Muscle synergies, computationally identified intermuscular coordination patterns, have been utilized to characterize neuromuscular control and learning in humans. However, it is unclear whether it is possible to alter the existing muscle synergies or develop new ones in an intended way through a relatively short-term motor exercise in adulthood. This study aimed to test the feasibility of expanding the repertoire of intermuscular coordination patterns through an isometric, electromyographic (EMG) signal-guided exercise in the upper extremity (UE) of neurologically intact individuals.


    10 participants were trained for six weeks to induce independent control of activating a pair of elbow flexor muscles that tended to be naturally co-activated in force generation. An untrained isometric force generation task was performed to assess the effect of the training on the intermuscular coordination of the trained UE. We applied a non-negative matrix factorization on the EMG signals recorded from 12 major UE muscles during the assessment to identify the muscle synergies. In addition, the performance of training tasks and the characteristics of individual muscles’ activity in both time and frequency domains were quantified as the training outcomes.


    Typically, in two weeks of the training, participants could use newly developed muscle synergies when requested to perform new, untrained motor tasks by activating their UE muscles in the trained way. Meanwhile, their habitually expressed muscle synergies, the synergistic muscle activation groups that were used before the training, were conserved throughout the entire training period. The number of muscle synergies activated for the task performance remained the same. As the new muscle synergies were developed, the neuromotor control of the trained muscles reflected in the metrics, such as the ratio between the targeted muscles, number of matched targets, and task completion time, was improved.


    These findings suggest that our protocol can increase the repertoire of readily available muscle synergies and improve motor control by developing the activation of new muscle coordination patterns in healthy adults within a relatively short period. Furthermore, the study shows the potential of the isometric EMG-guided protocol as a neurorehabilitation tool for aiming motor deficits induced by abnormal intermuscular coordination after neurological disorders.

    Trial registration

    This study was registered at the Clinical Research Information Service (CRiS) of the Korea National Institute of Health (KCT0005803) on 1/22/2021.

    more » « less
  3. null (Ed.)
    Abstract Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This “Learn to Move” competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research 
    more » « less
  4. Latash, Mark L. (Ed.)
    This chapter reviews major principles of neural control of movement proposed by N. A. Bernstein based on his biomechanical studies of human movements and published in his 1947 book ‘On Construction of Movements’. These principles include the hierarchical organization of the motor control system; synergistic sensorimotor control; the principle of sensory corrections, and the principles of repetition without repetition and fixating and subsequent releasing kinematic degrees of freedom during motor skill acquisition. These principles simplify control of the musculoskeletal system with redundant degrees of freedom and unpredictable effects of reactive and muscle forces arising in multi-segment kinematic chains. We also discuss the relevant contemporary research that has been inspired by and further developed Bernstein’s ideas. We demonstrate, in particular, examples of complex muscle and kinematic synergies organized by different levels of the motor control system, consequences of loss of proprioceptive sensory corrections on movement coordination, and emergence of economical and stable kinematic and muscle invariant movement characteristics in the process of skill acquisition by trials and errors. We conclude this chapter with motor control related parables told by N. A. Bernstein to one of the authors (VMZ). 
    more » « less
  5. Isometric force generation and kinematic reaching in the upper extremity has been found to be represented by a limited number of muscle synergies, even across task-specific variations. However, the extent of the generalizability of muscle synergies between these two motor tasks within the arm workspace remains unknown. In this study, we recorded electromyographic (EMG) signals from 13 different arm, shoulder, and back muscles of ten healthy individuals while they performed isometric and kinematic center-out target matches to one of 12 equidistant directional targets in the horizontal plane and at each of four starting arm positions. Non-negative matrix factorization was applied to the EMG data to identify the muscle synergies. Five and six muscle synergies were found to represent the isometric force generation and point-to-point reaches. We also found that the number and composition of muscle synergies were conserved across the arm workspace per motor task. Similar tuning directions of muscle synergy activation profiles were observed at different starting arm locations. Between the isometric and kinematic motor tasks, we found that two to four out of five muscle synergies were common in the composition and activation profiles across the starting arm locations. The greater number of muscle synergies that were involved in achieving a target match in the reaching task compared to the isometric task may explain the complexity of neuromotor control in arm reaching movements. Overall, our results may provide further insight into the neuromotor compartmentalization of shared muscle synergies between two different arm motor tasks and can be utilized to assess motor disabilities in individuals with upper limb motor impairments.

    more » « less