skip to main content


Title: MGpi: A Computational Model of Multiagent Group Perception and Interaction
Toward enabling next-generation robots capable of socially intelligent interaction with humans, we present a computational model of interactions in a social environment of multiple agents and multiple groups. The Multiagent Group Perception and Interaction (MGpi) network is a deep neural network that predicts the appropriate social action to execute in a group conversation (e.g., speak, listen, respond, leave), taking into account neighbors' observable features (e.g., location of people, gaze orientation, distraction, etc.). A central component of MGpi is the Kinesic-Proxemic-Message (KPM) gate, that performs social signal gating to extract important information from a group conversation. In particular, KPM gate filters incoming social cues from nearby agents by observing their body gestures (kinesics) and spatial behavior (proxemics). The MGpi network and its KPM gate are learned via imitation learning, using demonstrations from our designed social interaction simulator. Further, we demonstrate the efficacy of the KPM gate as a social attention mechanism, achieving state-of-the-art performance on the task of group identification without using explicit group annotations, layout assumptions, or manually chosen parameters.  more » « less
Award ID(s):
1637927
NSF-PAR ID:
10308753
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Despite increasing awareness and research about online strategic information operations, there remain gaps in our understanding, including how information operations leverage the wider information ecosystem and take shape on and across multiple social media platforms. In this paper we use mixed methods, including digital trace ethnography, to look beyond a single social media platform to the broader information ecosystem. We aim to understand how multiple social media platforms are used, in parallel and complementary ways, to achieve the strategic goals of online information operations. We focus on a specific case study: the contested online conversation surrounding Syria Civil Defense (the White Helmets), a group of first responders that assists civilians affected by the civil war within the country. Our findings reveal a network of social media platforms from which content is produced, stored, and integrated into the Twitter conversation. We highlight specific activities that sustain the strategic narratives and attempt to influence the media agenda. And we note that underpinning these efforts is the work of resilience-building: the use of alternative (non-mainstream) platforms to counter perceived threats of 'censorship' by large, established social media platforms. We end by discussing the implications on social media platform policy. 
    more » « less
  2. Objectives We examine the community epistemologies in youth’s iterative refinements of STEM-rich inventions across settings and time. Iteration in STEM-rich engineering/invention work refers to re-thinking ideas/designs within prototyping processes (Cunningham & Kelly, 2017). The objective of this paper is to examine the political dimensions of iteration through a) how iteration involves pre- and post-design “lives” of inventions especially towards new social futures, and b) the intentional incorporation of cultural epistemologies towards advancing new forms of legitimate inventor knowledge/practice (Yosso, 2005). Framing We draw from critical justice and consequential learning studies. Critical justice focuses on recognizing diversity and addressing structural inequalities perpetuated through systemic racism and classism. It seeks re-shifted relations of power and position within multiple scales-of-activity in learning, intersected with historicized injustices in learning environments. Consequential learning examines what matters to people, and how associated values and practices, when coordinated through social activity, allows for imagining new social futures (Gutierrez, 2012). Viewing the iterative process of inventing through a justice-oriented consequential lens calls into question traditional modes of knowing, and challenges/expands who and what areas of expertise are recognized and valued. Methods Our study takes place in two community makerspaces in mid-sized cities. Both center community engagement and support youth in designing/inventing to address problems they and their communities care about. Both also support minoritized youth in inventing through engagement with a wide range of community/STEM stakeholders. In researcher-educator roles, we collaborated with both makerspaces to establish programs supporting youth in sustained engagement in STEM and making/inventing in culturally-sustaining ways. In our two-year, longitudinal critical ethnography, data were generated in weekly community making sessions between 2016-2018. Data include artifacts, youth conversation groups, and videos capturing youth interaction with STEM and community experts at various stages in their design process. Analysis involved multiple stages and levels of coding based on open-coding and constant comparison procedures. Findings We ground our paper in four in-depth longitudinal cases of youth’s iterative design work: Nila’s light-up #stopracism sign; Su’zanne’s massaging slipper, Sharon’s geodesic play dome, and Jazmyn’s portable fan. Across cases, we illustrate three findings. First, youth located broader injustices within local making/inventing discourses with support from community and STEM allies, suggesting youth drew from multiple epistemologies, some grounded in community cultural wealth, others in STEM. For example, Su’Zanne drew from a familial culture of care and resistance in recognizing injustices nested in homelessness while iterating a way to make her slipper “more massaging.” The geodesic dome youth-makers drew from collective solidarity/resistance in making a structure for younger peers due to unjust lack of play infrastructure. Second, iterative engagement involving community wealth afforded further design and inventing experiences and expanded ownership over inventions across many stakeholders. For example, youth turned Nila’s #stopracism sign on during group discussions when they felt that racism needed to be foregrounded. Third, the afterlife of youth invention processes impacted the emergent inventor-maker culture through influencing the iterative process. Significance Iterations expand hybridization of cultural knowledge/practice and STEM-rich inventing, re-shaping whose cultural knowledge matters, and fostering justice-oriented collective outcomes. 
    more » « less
  3. Internships are widely viewed as “door openers” to opportunity, yet students without ample financial, social, and institutional supports are often excluded from these experiences. This exclusion is especially problematic for Latinx students attending Hispanic-Serving Institutions, for whom an internship could be transformative. In this article, we elaborate upon Núñez’s (2014) multilevel model of intersectionality to highlight how agents’ perceptions, social categories, embodied practices, and broader contextual forces interrelate to shape Latinx students’ access to internship opportunities at a university in the border region of Texas. Using inductive thematic, correlational, and social-network analysis techniques to analyze survey (n = 192) and focus group (n = 12) data, we find that gender, academic major, socioeconomic status, and race intersect with organizational (e.g., insufficient information) and contextual (e.g., labor markets) factors to shape students’ access and perceptions. We provide recommendations for disrupting systemic inequalities in internship access and culturally appropriate programming and ideas for future research. 
    more » « less
  4. The culture within engineering colleges and departments has been historically quiet when considering social justice issues. Often the faculty in those departments are less concerned with social issues and are primarily focused on their disciplines and the concrete ways that they can make impacts academically and professionally in their respective arena’s. However, with the social climate of the United States shifting ever more towards a politically charged climate, and current events, particularly the protests against police brutality in recent years, faculty and students are constantly inundated with news of injustices happening in our society. The murder of George Floyd on May 25th 2020 sent shockwaves across the United States and the world. The video captured of his death shared across the globe brought everyone’s attention to the glaringly ugly problem of police brutality, paired with the COVID-19 pandemic, and US election year, the conditions were just right for a social activist movement to grow to a size that no one could ignore. Emmanuel Acho spoke out, motivated by injustices seen in the George Floyd murder, initially with podcasts and then by writing his book “Uncomfortable Converstations with a Black Man” [1]. In his book he touched on various social justice issues such as: racial terminology (i.e., Black or African American), implicit biases, white privilege, cultural appropriation, stereotypes (e.g., the “angry black man”), racial slurs (particularly the n-word), systemic racism, the myth of reverse racism, the criminal justice system, the struggles faced by black families, interracial families, allyship, and anti-racism. Students and faculty at Anonymous University felt compelled to set aside the time to meet and discuss this book in depth through the video conferencing client Zoom. In these meetings diverse facilitators were tasked with bringing the topics discussed by Acho in his book into conversation and pushing attendees of these meetings to consider those topics critically and personally. In an effort to avoid tasking attendees with reading homework to be able to participate in these discussions, the discussed chapter of the audiobook version of Acho’s book was played at the beginning of each meeting. Each audiobook chapter lasted between fifteen and twenty minutes, after which forty to forty-five minutes were left in the hour-long meetings to discuss the content of the chapter in question. Efforts by students and faculty were made to examine how some of the teachings of the book could be implemented into their lives and at Anonymous University. For broader topics, they would relate the content back to their personal lives (e.g., raising their children to be anti-racist and their experiences with racism in American and international cultures). Each meeting was recorded for posterity in the event that those conversations would be used in a paper such as this. Each meeting had at least one facilitator whose main role was to provide discussion prompts based on the chapter and ensure that the meeting environment was safe and inclusive. Naturally, some chapters address topics that are highly personal to some participants, so it was vital that all participants felt comfortable and supported to share their thoughts and experiences. The facilitator would intervene if the conversation veered in an aggressive direction. For example, if a participant starts an argument with another participant in a non-constructive manner, e.g., arguing over the definition of ethnicity, then the facilitator will interrupt, clear the air to bring the group back to a common ground, and then continue the discussion. Otherwise, participants were allowed to steer the direction of the conversation as new avenues of discussion popped up. These meetings were recorded with the goal of returning to these conversations and analyzing the conversations between attendees. Grounded theory will be used to first assess the most prominent themes of discussion between attendees for each meeting [2]. Attendees will be contacted to expressly ask their permission to have their words and thoughts used in this work, and upon agreement that data will begin to be processed. Select attendees will be asked to participate in focus group discussions, which will also be recorded via Zoom. These discussions will focus around the themes pulled from general discussion and will aim to dive deeper into the impact that this experience has had on them as either students or faculty members. A set of questions will be developed as prompts, but conversation is expected to evolve organically as these focus groups interact. These sessions will be scheduled for an hour, and a set of four focus groups with four participants are expected to participate for a total of sixteen total focus group participants. We hope to uncover how this experience changed the lives of the participants and present a model of how conversations such as this can promote diversity, equity, inclusion, and access activities amongst faculty and students outside of formal programs and strategic plans that are implemented at university, college, or departmental levels. 
    more » « less
  5. Customers are rapidly turning to social media for customer support. While brand agents on these platforms are motivated and well-intentioned to help and engage with customers, their efforts are often ignored if their initial response to the customer does not match a specific tone, style, or topic the customer is aiming to receive. The length of a conversation can reflect the effort and quality of the initial response made by a brand toward collaborating and helping consumers, even when the overall sentiment of the conversation might not be very positive. Thus, through this study, we aim to bridge this critical gap in the existing literature by analyzing language’s content and stylistic aspects such as expressed empathy, psycho-linguistic features, dialogue tags, and metrics for quantifying personalization of the utterances that can influence the engagement of an interaction. This paper demonstrates that we can predict engagement using initial customer and brand posts. 
    more » « less