An official website of the United States government
This study examines an aspect of the role of emotion in multimedia learning, i.e., whether participants can recognize the instructor’s positive or negative emotion based on hearing short clips involving only the instructor’s voice just as well as also seeing an embodied onscreen agent. Participants viewed 16 short video clips from a statistics lecture in which an animated instructor, conveying a happy, content, frustrated, or bored emotion, stands next to a slide as she lectures (agent present) or uses only her voice (agent absent). For each clip, participants rated the instructor on five-point scales for how happy, content, frustrated, and bored the instructor seemed. First, for happy, content, and bored instructors, participants were just as accurate in rating emotional tone based on voice only as with voice plus onscreen agent. This supports the voice hypothesis, which posits that voice is a powerful source of social-emotional information. Second, participants rated happy and content instructors higher on happy and content scales and rated frustrated and bored instructors higher on frustrated and bored scales. This supports the positivity hypothesis, which posits that people are particularly sensitive to the positive or negative tone of multimedia instructional messages.
more »
« less
Recognizing and Relating to the Race/Ethnicity and Gender of Animated Pedagogical Agents
This study examined how well people can recognize and relate to animated pedagogical agents of varying ethnicities/races and genders. For both Study 1 (realistic-style agents) and Study 2 (cartoon-style agents), participants viewed brief video clips of virtual agents of varying racial/ethnic categories and gender types and then identified their race/ethnicity and gender and rated how human-like and likable the agent appeared. Participants were highly accurate in identifying Black and White agents but were less accurate for Asian, Indian, and Hispanic agents. Participants were accurate in recognizing gender differences. Participants rated all types of agents as moderately human-like, except for White agents. Likability ratings were lowest for White and male agents. The same pattern of results was obtained across two independent studies with different participants and different onscreen agents, which indicates that the results are not solely due to one specific set of agents. Consistent with the Media Equation Hypothesis and the Alliance Hypothesis, this work shows that people are sensitive to the race/ethnicity and gender of onscreen agents and relate to them differently. These findings have implications for how to design animated pedagogical agents for improved multimedia learning environments in the future and serve as a crucial first step in highlighting the possibility and feasibility of incorporating diverse onscreen virtual agents into educational computer software.
more »
« less
- Award ID(s):
- 2201019
- PAR ID:
- 10473967
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- Journal of Educational Computing Research
- Volume:
- 62
- Issue:
- 3
- ISSN:
- 0735-6331
- Format(s):
- Medium: X Size: p. 675-701
- Size(s):
- p. 675-701
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Do students learn from video lessons presented by pedagogical agents of different racial and gender types equivalently to those delivered by a real human instructor? How do the race and gender of these agents impact students’ learning experiences and outcomes? In this between-subject design study, college students were randomly assigned to view a six 9-minute video lesson on chemical bonds, presented by pedagogical agents varying in gender (male, female) and race (Asian, Black, White), or to view the original lesson with a real human instructor. In comparing learning with a human instructor versus with a pedagogical agent of various races and genres, ANOVAs revealed no significant differences in learning outcomes (retention and transfer scores) or learner emotions, but students reported a stronger social connection with the human instructor over pedagogical agents. Students reported stronger positive emotions and social connections with female agents over male agents. Additionally, there was limited evidence of a race-matching effect, with White students showing greater positive emotion while learning with pedagogical agents of the same race. These findings highlight the limitations of pedagogical agents compared to human instructors in video lessons, while partially reflecting gender stereotypes and intergroup bias in instructor evaluations.more » « less
-
The paper reports ongoing research toward the design of multimodal affective pedagogical agents that are effective for different types of learners and applications. In particular, the work reported in the paper investigated the extent to which the type of character design (realistic versus stylized) affects students’ perception of an animated agent’s facial emotions, and whether the effects are moderated by learner characteristics (e.g. gender). Eighty-two participants viewed 10 animation clips featuring a stylized character exhibiting 5 different emotions, e.g. happiness, sadness, fear, surprise and anger (2 clips per emotion), and 10 clips featuring a realistic character portraying the same emotional states. The participants were asked to name the emotions and rate their sincerity, intensity, and typicality. The results indicated that for recognition, participants were slightly more likely to recognize the emotions displayed by the stylized agent, although the difference was not statistically significant. The stylized agent was on average rated significantly higher for facial emotion intensity, whereas the differences in ratings for typicality and sincerity across all emotions were not statistically significant. A significant difference in ratings was shown in regard to sadness (within typicality), happiness (within sincerity), fear, anger, sadness and happiness (within intensity) with the stylized agent rated higher. Gender was not a significant correlate across all emotions or for individual emotions.more » « less
-
The overall goal of our research is to develop a system of intelligent multimodal affective pedagogical agents that are effective for different types of learners (Adamo et al., 2021). While most of the research on pedagogical agents tends to focus on the cognitive aspects of online learning and instruction, this project explores the less-studied role of affective (or emotional) factors. We aim to design believable animated agents that can convey realistic, natural emotions through speech, facial expressions, and body gestures and that can react to the students’ detected emotional states with emotional intelligence. Within the context of this goal, the specific objective of the work reported in the paper was to examine the extent to which the agents’ facial micro-expressions affect students’ perception of the agents’ emotions and their naturalness. Micro-expressions are very brief facial expressions that occur when a person either deliberately or unconsciously conceals an emotion being felt (Ekman &Friesen, 1969). Our assumption is that if the animated agents display facial micro expressions in addition to macro expressions, they will convey higher expressive richness and naturalness to the viewer, as “the agents can possess two emotional streams, one based on interaction with the viewer and the other based on their own internal state, or situation” (Queiroz et al. 2014, p.2).The work reported in the paper involved two studies with human subjects. The objectives of the first study were to examine whether people can recognize micro-expressions (in isolation) in animated agents, and whether there are differences in recognition based on the agent’s visual style (e.g., stylized versus realistic). The objectives of the second study were to investigate whether people can recognize the animated agents’ micro-expressions when integrated with macro-expressions, the extent to which the presence of micro + macro-expressions affect the perceived expressivity and naturalness of the animated agents, the extent to which exaggerating the micro expressions, e.g. increasing the amplitude of the animated facial displacements affects emotion recognition and perceived agent naturalness and emotional expressivity, and whether there are differences based on the agent’s design characteristics. In the first study, 15 participants watched eight micro-expression animations representing four different emotions (happy, sad, fear, surprised). Four animations featured a stylized agent and four a realistic agent. For each animation, subjects were asked to identify the agent’s emotion conveyed by the micro-expression. In the second study, 234 participants watched three sets of eight animation clips (24 clips in total, 12 clips per agent). Four animations for each agent featured the character performing macro-expressions only, four animations for each agent featured the character performing macro- + micro-expressions without exaggeration, and four animations for each agent featured the agent performing macro + micro-expressions with exaggeration. Participants were asked to recognize the true emotion of the agent and rate the emotional expressivity ad naturalness of the agent in each clip using a 5-point Likert scale. We have collected all the data and completed the statistical analysis. Findings and discussion, implications for research and practice, and suggestions for future work will be reported in the full paper. ReferencesAdamo N., Benes, B., Mayer, R., Lei, X., Meyer, Z., &Lawson, A. (2021). Multimodal Affective Pedagogical Agents for Different Types of Learners. In: Russo D., Ahram T., Karwowski W., Di Bucchianico G., Taiar R. (eds) Intelligent Human Systems Integration 2021. IHSI 2021. Advances in Intelligent Systems and Computing, 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-68017-6_33Ekman, P., &Friesen, W. V. (1969, February). Nonverbal leakage and clues to deception. Psychiatry, 32(1), 88–106. https://doi.org/10.1080/00332747.1969.11023575 Queiroz, R. B., Musse, S. R., &Badler, N. I. (2014). Investigating Macroexpressions and Microexpressions in Computer Graphics Animated Faces. Presence, 23(2), 191-208. http://dx.doi.org/10.1162/more » « less
-
Building on research demonstrating significant differences in how Black and White Americans view law enforcement, this study assesses how those differential views shape potential jurors’ decision-making in the context of a federal drug conspiracy case in which the primary evidence against the defendant is provided by an FBI agent and an informant cooperating with the agent. A sample of 649 Black and White jury-eligible U.S. citizens were exposed to the case, in which a Black defendant is being tried, and where the informant-witness race (Black or White) was varied. Participants determined verdict, evaluated evidence, and completed additional measures. Results indicated that Black participants were significantly less likely to convict than White participants, especially in the White informant condition; rated the law enforcement witness as less credible, and viewed police more negatively across three composite measures. Exploratory analysis of how juror race and gender interacted indicates Black women largely drove racial differences in verdicts. Perceptions of police legitimacy mediated the relationship between juror race and verdict choice. We conclude that it is critical that citizens are not prevented from being seated on juries due to skepticism about police, given the risk of disproportionate exclusion of Black potential jurors. The legal processes relevant to juror excusals need to be reconsidered to ensure that views of police, rooted in actual experience or knowledge about the problems with fair and just policing, are not used to disproportionately exclude persons of color, or to seat juries overrepresented by people who blindly trust police.more » « less
