An official website of the United States government
Background: Motor imitation difficulties are pervasive in children with Autism Spectrum Disorder (ASD). Previous research demonstrated the validity and reliability of an algorithm called Computerized Assessment of Motor Imitation (CAMI) using 3D depth cameras. However, incorporating CAMI into serious games and making it accessible in clinic and home settings requires a more scalable approach that uses “off-the-shelf” 2D cameras. Method: In a brief (one-minute) task, children (23 ASD, 17 typically developing [TD]) imitated a model’s dance movements while simultaneously being recorded using Kinect Xbox motion tracking technology (Kinect 3D) and a single 2D camera. Pose-estimation software (OpenPose 2D) was used on the 2D camera video to fit a skeleton to the imitating child. Motor imitation scores computed from the fully automated OpenPose 2D CAMI method were compared to scores computed from the Kinect 3D CAMI and Human Observation Coding (HOC) methods. Results: Motor imitation scores obtained from the OpenPose 2D CAMI method were significantly correlated with scores obtained from the Kinect 3D CAMI method (r40 = 0.82, p < 0.001) and the HOC method (r40 = 0.80, p < 0.001). Both 2D and 3D CAMI methods showed better discriminative ability than the HOC, with the Kinect 3D CAMI method outperforming the OpenPose 2D CAMI method (area under ROC curve (AUC): AUCHOC = 0.799, AUC2D-CAMI = 0.876, AUC3D-CAMI= 0.94). Finally, all motor imitation scores were significantly associated with the social communication impairment (all p ≤ 0.003). Conclusions: This pilot-study demonstrated that motor imitation can be automatically quantified using a single 2D camera.
more »
« less
Motion Segmentation via Generalized Curvatures
New depth sensors, like the Microsoft Kinect, produce streams of human pose data. These discrete pose streams can be viewed as noisy samples of an underlying continuous ideal curve that describes a trajectory through high-dimensional pose space. This paper introduces a technique for generalized curvature analysis (GCA) that determines features along the trajectory which can be used to characterize change and segment motion. Tools are developed for approximating generalized curvatures at mean points along a curve in terms of the singular values of local mean-centered data balls. The features of the GCA algorithm are illustrated on both synthetic and real examples, including data collected from a Kinect II sensor. We also applied GCA to the Carnegie Mellon University Motion Capture (MoCaP) database. Given that GCA scales linearly with the length of the time series we are able to analyze large data sets without down sampling. It is demonstrated that the generalized curvature approximations can be used to segment pose streams into motions and transitions between motions. The GCA algorithm can identify 94.2% of the transitions between motions without knowing the set of possible motions in advance, even though the subjects do not stop or pause between motions.
more »
« less
- Award ID(s):
- 1633830
- PAR ID:
- 10099066
- Date Published:
- Journal Name:
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- ISSN:
- 0162-8828
- Page Range / eLocation ID:
- 1 to 1
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)In this paper, we propose a novel online algorithm for motion similarity measurements during human-robot interaction (HRI). Specifically, we formulate a Segment-based Online Dynamic Time Warping (SODTW) algorithm that can be used for understanding of repeated and cyclic human motions, in the context of rehabilitation or social interaction. The algorithm can estimate both the human-robot motion similarity and the time delay to initiate motion and combine these values as a metric to adaptively select appropriate robot imitation repertoires. We validated the algorithm offline by post-processing experimental data collected from a cohort of 55 subjects during imitation episodes with our social robot Zeno. Furthermore, we implemented the algorithm online on Zeno and collected further experimental results with 13 human subjects. These results show that the algorithm can reveal important features of human movement including the quality of motion and human reaction time to robot stimuli. Moreover, the robot can adapt to appropriate human motion speeds based on similarity measurements calculated using this algorithm, enabling future adaptive rehabilitation interventions for conditions such as Autism Spectrum Disorders (ASD).more » « less
-
To represent motions from a mechanical point of view, this paper explores motion embedding using the motion taxonomy. With this taxonomy, manipulations can be described and represented as binary strings called motion codes. Motion codes capture mechanical properties, such as contact type and trajectory, that should be used to define suitable distance metrics between motions or loss functions for deep learning and reinforcement learning. Motion codes can also be used to consolidate aliases or cluster motion types that share similar properties. Using existing data sets as a reference, we discuss how motion codes can be created and assigned to actions that are commonly seen in activities of daily living based on intuition as well as real data. Motion codes are compared to vectors from pre-trained Word2Vec models, and we show that motion codes maintain distances that closely match the reality of manipulation.more » « less
-
To represent motions from a mechanical point of view, this paper explores motion embedding using the motion taxonomy. With this taxonomy, manipulations can be described and represented as binary strings called motion codes. Motion codes capture mechanical properties, such as contact type and trajectory, that should be used to define suitable distance metrics between motions or loss functions for deep learning and rein- forcement learning. Motion codes can also be used to consolidate aliases or cluster motion types that share similar properties. Using existing data sets as a reference, we discuss how motion codes can be created and assigned to actions that are commonly seen in activities of daily living based on intuition as well as real data. Motion codes are compared to vectors from pre-trained Word2Vec models, and we show that motion codes maintain distances that closely match the reality of manipulation.more » « less
-
To represent motions from a mechanical point of view, this paper explores motion embedding using the motion taxonomy. With this taxonomy, manipulations can be described and represented as binary strings called motion codes. Motion codes capture mechanical properties, such as contact type and trajectory, that should be used to define suitable distance metrics between motions or loss functions for deep learning and rein- forcement learning. Motion codes can also be used to consolidate aliases or cluster motion types that share similar properties. Using existing data sets as a reference, we discuss how motion codes can be created and assigned to actions that are commonly seen in activities of daily living based on intuition as well as real data. Motion codes are compared to vectors from pre-trained Word2Vec models, and we show that motion codes maintain distances that closely match the reality of manipulation.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/TPAMI.2018.2869741
