Search for: All records

Research at the intersection of robots and dance promises to create vehicles for expression that enable new creative pursuits and allow robots to function better, especially in human-facing scenarios. Moving this research beyond fringe spectacle and establishing it as a serious, systematic field—a proper subdiscipline of both robotics and dance—will require answering a key question: How does dance advance the fundamentals of robotics, and vice versa? Focusing on the former, this article offers glimpses of this new field with examples of meaningful contributions to control, robotics, and autonomous systems, such as novel actuator designs, improved sensing systems, salient motion profiles for robots, reproducible experiment designs, and new theories of motion derived from the study of dance. It also poses two grand challenges for the emerging field of choreobotics: developing a robust symbolic system for representing bodily action and establishing rich, repeatable testing environments for human–robot interaction. 

In this opinion piece, the authors, from the fields of artificial intelligence (AI) and psychology, reflect on how the foundational discoveries of Nobel laureates Hopfield and Hinton have influenced their research. They also discuss emerging directions in AI and the challenges that lie ahead for neural networks and machine learning. 

The rapid expansion of social media platforms has provided unprecedented access to massive amounts of multimodal user-generated content. Comprehending user emotions can provide valuable insights for improving communication and understanding of human behaviors. Despite significant advancements in Affective Computing, the diverse factors influencing user emotions in social networks remain relatively understudied. Moreover, there is a notable lack of deep learning-based methods for predicting user emotions in social networks, which could be addressed by leveraging the extensive multimodal data available. This work presents a novel formulation of personalized emotion prediction in social networks based on heterogeneous graph learning. Building upon this formulation, we design HMG-Emo, a Heterogeneous Multimodal Graph Learning Framework that utilizes deep learning-based features for user emotion recognition. Additionally, we include a dynamic context fusion module in HMG-Emo that is capable of adaptively integrating the different modalities in social media data. Through extensive experiments, we demonstrate the effectiveness of HMG-Emo and verify the superiority of adopting a graph neural network-based approach, which outperforms existing baselines that use rich hand-crafted features. To the best of our knowledge, HMG-Emo is the first multimodal and deep-learning-based approach to predict personalized emotions within online social networks. Our work highlights the significance of exploiting advanced deep learning techniques for less-explored problems in Affective Computing. 

As social robots and other intelligent machines enter the home, artificial emotional intelligence (AEI) is taking center stage to address users’ desire for deeper, more meaningful human-machine interaction. To accomplish such efficacious interaction, the next-generation AEI need comprehensive human emotion models for training. Unlike theory of emotion, which has been the historical focus in psychology, emotion models are a descriptive tools. In practice, the strongest models need robust coverage, which means defining the smallest core set of emotions from which all others can be derived. To achieve the desired coverage, we turn to word embeddings from natural language processing. Using unsupervised clustering techniques, our experiments show that with as few as 15 discrete emotion categories, we can provide maximum coverage across six major languages–Arabic, Chinese, English, French, Spanish, and Russian. In support of our findings, we also examine annotations from two large-scale emotion recognition datasets to assess the validity of existing emotion models compared to human perception at scale. Because robust, comprehensive emotion models are foundational for developing real-world affective computing applications, this work has broad implications in social robotics, human-machine interaction, mental healthcare, and computational psychology. 

How do we make a machine that indicates changes to its internal state, e.g., status, goals, attitude, or even emotion, through changes in movement profiles? This workshop will pose a possible direction toward such ends that leverages movement notation as a source for clearly defining abstract concepts of similarity and symbolic representation of the parts and patterns of movement - in order to identify, record and interpret patterns of human movement on both the micro and macro levels. First, we will move together. This will activate an innate ability to imitate each other and, in doing so, illuminate the principal components of Laban/Bartenieff Movement Studies – a field comprised of Laban Movement Analysis and Bartenieff Fundamentals – and the Body, Effort, Shape, Space, and Time (BESST) System of movement analysis. This system of work, deriving from dance and physical therapy practices, which is a textbook; thus, a key value proposition of the workshop is in its embodied, situated nature that can be supplemented by textbooks, including a newly released book from MIT Press authored by the workshop organizers. Next, we will try to write down what we’re doing. A set of symbols for describing elements of the BESST System, which seem to be particularly perceptually meaningful to human observers, will be presented so that movement ideas can be notated and, thus, translated between bodies. We will explore both Labanotation and a related “motif”-style notation. This workshop is supported by NSF grant numbers 2234195 and 2234197. 

The British landscape painter John Constable is considered foundational for the Realist movement in 19th-century European painting. Constable’s painted skies, in particular, were seen as remarkably accurate by his contemporaries, an impression shared by many viewers today. Yet, assessing the accuracy of realist paintings like Constable’s is subjective or intuitive, even for professional art historians, making it difficult to say with certainty what set Constable’s skies apart from those of his contemporaries. Our goal is to contribute to a more objective understanding of Constable’s realism. We propose a new machine-learning-based paradigm for studying pictorial realism in an explainable way. Our framework assesses realism by measuring the similarity between clouds painted by artists noted for their skies, like Constable, and photographs of clouds. The experimental results of cloud classification show that Constable approximates more consistently than his contemporaries the formal features of actual clouds in his paintings. The study, as a novel interdisciplinary approach that combines computer vision and machine learning, meteorology, and art history, is a springboard for broader and deeper analyses of pictorial realism. 

Data science, an emerging multidisciplinary field, resides at the intersec- tion of computational sciences, statistical modeling, and domain-specific sciences. The current norm for data science education predominantly focuses on graduate programs, which presume a pre-existing knowledge base in one or more relevant sciences. However, this framework often overlooks those who don’t plan to pursue graduate studies, thereby limiting their exposure to this rapidly expanding field. Penn State addressed this gap by establishing one of the first undergraduate degree programs in Data Sciences, a collaboration between the College of Information Sci- ences and Technology, the Department of Computer Science and Engineering, and the Department of Statistics. One key component of this program is a project-focused, writing-intensive course designed for upper-class undergraduates. This course guides students through the entire data science project pipeline, from problem identifica- tion to solution presentation. It allows students to apply foundational data science principles to real-world problems, advancing their understanding through practi- cal application. This chapter details the objectives, rationale, and course design, alongside reflections from our teaching experience. The insights provided could be helpful to instructors developing similar data science programs or courses at an undergraduate level, broadening the influence of this important field 

Bodily expressed emotion understanding (BEEU) aims to automatically recognize human emotional expressions from body movements. Psychological research has demonstrated that people often move using specific motor elements to convey emotions. This work takes three steps to integrate human motor elements to study BEEU. First, we introduce BoME (body motor elements), a highly precise dataset for human motor elements. Second, we apply baseline models to estimate these elements on BoME, showing that deep learning methods are capable of learning effective representations of human movement. Finally, we propose a dual-source solution to enhance the BEEU model with the BoME dataset, which trains with both motor element and emotion labels and simultaneously produces predictions for both. Through experiments on the BoLD in-the-wild emotion understanding benchmark, we showcase the significant benefit of our approach. These results may inspire further research utilizing human motor elements for emotion understanding and mental health analysis.