skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Social-Emotional-Sensory Design Map for Affective Computing Informed by Neurodivergent Experiences
One of the grand challenges of artificial intelligence and affective computing is for technology to become emotionally-aware and thus, more human-like. Modeling human emotions is particularly complicated when we consider the lived experiences of people who are on the autism spectrum. To understand the emotional experiences of autistic adults and their attitudes towards common representations of emotions, we deployed a context study as the first phase of a Grounded Design research project. Based on community observations and interviews, this work contributes empirical evidence of how the emotional experiences of autistic adults are entangled with social interactions as well as the processing of sensory inputs. We learned that (1) the emotional experiences of autistic adults are embodied and co-constructed within the context of physical environments, social relationships, and technology use, and (2) conventional approaches to visually representing emotion in affective education and computing systems fail to accurately represent the experiences and perceptions of autistic adults. We contribute a social-emotional-sensory design map to guide designers in creating more diverse and nuanced affective computing interfaces that are enriched by accounting for neurodivergent users.  more » « less
Award ID(s):
1845023
PAR ID:
10321744
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the ACM on Human-Computer Interaction
Volume:
5
Issue:
CSCW1
ISSN:
2573-0142
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, ACM Digital Library)
    For autistic individuals, navigating social and emotional interactions can be complex, often involving disproportionately high cognitive labor in contrast to neurotypical conversation partners. Through a novel approach to speculative co-design, autistic adults explored affective imaginaries — imagined futuristic technology interventions — to probe a provocative question: What if technology could translate emotions like it can translate spoken language? The resulting speculative prototype for an image-enabled emotion translator chat application included: (1) a visual system for representing personalized emotion taxonomies, and (2) a Wizard of Oz implementation of these taxonomies in a low-fidelity chat application. Although wary of technology that purports to understand emotions, autistic participants saw value in being able to deploy visual emotion taxonomies during chats with neurotypical conversation partners. This work shows that affective technology should enable users to: (1) curate encodings of emotions used in system artifacts, (2) enhance interactive emotional understanding, and (3) have agency over how and when to use emotion features. 
    more » « less
  2. ; (, Proceedings of the ACM on Human-Computer Interaction)
    During interpersonal interactions, conversational moves can help people establishcommon ground ---a shared frame of reference to support communication. Neurodiverse conversation dyads that include autistic and non-autistic members can experience challenges in creating and maintaining such a shared frame of reference due to differing communication and cognitive styles. We conducted a design study to understand conversational patterns among neurodiverse dyads and then used those patterns to co-design concepts for supporting the creation and maintenance of common ground by those conversation pairs. Our study involved two activities with participants: (1) a paired interview with autistic adults and a trusted conversation partner that used a novel swimlane visual elicitation activity, and (2) a remote design study during which the autistic participants designed a game intended to visualize and support neurodiverse conversation dynamics. We found that communication technology can scaffold neurodiverse dyads in locating common ground by supporting crucial individual and joint decision-making; clarification of language and emotions; and embodied sense-making of identity, relationships, and shared information. This project generated insights related to two distinct aspects of designing assistive Information and Communication Technology (ICT) to support autistic individuals: (1) the ability for visual elicitation activities to help autistic individuals recognize interaction patterns, gain a deeper understanding of other's perspectives, and imagine more desirable alternatives, and (2) the importance of recognizing and supporting multi-dimensional aspects of communication practices (i.e., social, emotional, sensory) in establishing and maintaining shared points of reference for neurodiverse conversation dyads. 
    more » « less
  3. ; ; (, Computational Theory of Mind for Human-Machine Teams. AAAI-FSS 2021. Lecture Notes in Computer Science)
    Gurney, Nikolos; Sukthankar, Gita (Ed.)
    Computational emotion, is naturally predicated on an operating theory of emotion. This paper seeks to explore the prevalence of three different approaches in the literature, namely basic emotion, dimensional emotion, and constructed emotion. Basic emotion maintains that there exists a discrete set of primitive emotions evolved as responses to certain stimuli; dimensional emotion sees different emotions as systematically related by two or more dimensions (typically valence and arousal); and constructed emotion describes emotional experience as a function of the brain’s general predictive faculties applied to learned social concepts of different emotions. In order to see how these approaches are represented in affective computing literature, we conduct a systematic survey spanning the IEEE, ACM, ScienceDirect, and Engineering Village databases. Out of 204 selected papers, 151 apply basic emotion theory, 48 apply dimensional emotion, and 5 apply constructed emotion. We find promising representation of the constructed emotion theory in the affective computing literature and conclude that it provides a theoretical basis worth pursuing for affective engagement human computer interaction (HCI) applications. 
    more » « less
  4. ; (, Current Directions in Psychological Science)
    Models of episodic emotional memory typically concern why emotional events are more likely to be remembered than neutral events, focusing on interactions between the amygdala and other medial temporal lobe regions. But memories of emotional events can be distinguished by their affective tone and framing. We propose that the dorsomedial prefrontal cortex (dmPFC), a region that is increasingly recognized to crosscut socio-affective and cognitive domains, plays a key role in this aspect of emotional memory. After briefly reviewing the role of the dmPFC in the control of behaviors ranging from actions to emotions to social cognition, we delve into the accumulating evidence that its functions also subserve the abstraction of meaning from events and the control of memories, particularly emotional memories. Its role begins during the encoding of emotional experiences, continues through their stabilization, and endures during the retrieval of memory content. At each phase, the dmPFC participates in the integration of affective and cognitive components of memories, setting up networks and framings that either emphasize or de-emphasize emotional content. Incorporating the dmPFC into models of episodic emotional memory should provide leverage in understanding the affective tone with which experiences are brought to memory. 
    more » « less
  5. ; ; ; ; ; (, IEEE)
    na (Ed.)
    In the field of affective computing, emotional annotations are highly important for both the recognition and synthesis of human emotions. Researchers must ensure that these emotional labels are adequate for modeling general human perception. An unavoidable part of obtaining such labels is that human annotators are exposed to known and unknown stimuli before and during the annotation process that can affect their perception. Emotional stimuli cause an affective priming effect, which is a pre-conscious phenomenon in which previous emotional stimuli affect the emotional perception of a current target stimulus. In this paper, we use sequences of emotional annotations during a perceptual evaluation to study the effect of affective priming on emotional ratings of speech. We observe that previous emotional sentences with extreme emotional content push annotations of current samples to the same extreme. We create a sentence-level bias metric to study the effect of affective priming on speech emotion recognition (SER) modeling. The metric is used to identify subsets in the database with more affective priming bias intentionally creating biased datasets. We train and test SER models using the full and biased datasets. Our results show that although the biased datasets have low inter-evaluator agreements, SER models for arousal and dominance trained with those datasets perform the best. For valence, the models trained with the less-biased datasets perform the best. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email