skip to main content

Title: MGpi: A Computational Model of Multiagent Group Perception and Interaction
Toward enabling next-generation robots capable of socially intelligent interaction with humans, we present a computational model of interactions in a social environment of multiple agents and multiple groups. The Multiagent Group Perception and Interaction (MGpi) network is a deep neural network that predicts the appropriate social action to execute in a group conversation (e.g., speak, listen, respond, leave), taking into account neighbors' observable features (e.g., location of people, gaze orientation, distraction, etc.). A central component of MGpi is the Kinesic-Proxemic-Message (KPM) gate, that performs social signal gating to extract important information from a group conversation. In particular, KPM gate filters incoming social cues from nearby agents by observing their body gestures (kinesics) and spatial behavior (proxemics). The MGpi network and its KPM gate are learned via imitation learning, using demonstrations from our designed social interaction simulator. Further, we demonstrate the efficacy of the KPM gate as a social attention mechanism, achieving state-of-the-art performance on the task of group identification without using explicit group annotations, layout assumptions, or manually chosen parameters.
Authors:
; ;
Award ID(s):
1637927
Publication Date:
NSF-PAR ID:
10308753
Journal Name:
AAMAS '20: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems
Sponsoring Org:
National Science Foundation
More Like this
  1. Despite increasing awareness and research about online strategic information operations, there remain gaps in our understanding, including how information operations leverage the wider information ecosystem and take shape on and across multiple social media platforms. In this paper we use mixed methods, including digital trace ethnography, to look beyond a single social media platform to the broader information ecosystem. We aim to understand how multiple social media platforms are used, in parallel and complementary ways, to achieve the strategic goals of online information operations. We focus on a specific case study: the contested online conversation surrounding Syria Civil Defense (the White Helmets), a group of first responders that assists civilians affected by the civil war within the country. Our findings reveal a network of social media platforms from which content is produced, stored, and integrated into the Twitter conversation. We highlight specific activities that sustain the strategic narratives and attempt to influence the media agenda. And we note that underpinning these efforts is the work of resilience-building: the use of alternative (non-mainstream) platforms to counter perceived threats of 'censorship' by large, established social media platforms. We end by discussing the implications on social media platform policy.
  2. Objectives We examine the community epistemologies in youth’s iterative refinements of STEM-rich inventions across settings and time. Iteration in STEM-rich engineering/invention work refers to re-thinking ideas/designs within prototyping processes (Cunningham & Kelly, 2017). The objective of this paper is to examine the political dimensions of iteration through a) how iteration involves pre- and post-design “lives” of inventions especially towards new social futures, and b) the intentional incorporation of cultural epistemologies towards advancing new forms of legitimate inventor knowledge/practice (Yosso, 2005). Framing We draw from critical justice and consequential learning studies. Critical justice focuses on recognizing diversity and addressing structural inequalities perpetuated through systemic racism and classism. It seeks re-shifted relations of power and position within multiple scales-of-activity in learning, intersected with historicized injustices in learning environments. Consequential learning examines what matters to people, and how associated values and practices, when coordinated through social activity, allows for imagining new social futures (Gutierrez, 2012). Viewing the iterative process of inventing through a justice-oriented consequential lens calls into question traditional modes of knowing, and challenges/expands who and what areas of expertise are recognized and valued. Methods Our study takes place in two community makerspaces in mid-sized cities. Both center community engagement and supportmore »youth in designing/inventing to address problems they and their communities care about. Both also support minoritized youth in inventing through engagement with a wide range of community/STEM stakeholders. In researcher-educator roles, we collaborated with both makerspaces to establish programs supporting youth in sustained engagement in STEM and making/inventing in culturally-sustaining ways. In our two-year, longitudinal critical ethnography, data were generated in weekly community making sessions between 2016-2018. Data include artifacts, youth conversation groups, and videos capturing youth interaction with STEM and community experts at various stages in their design process. Analysis involved multiple stages and levels of coding based on open-coding and constant comparison procedures. Findings We ground our paper in four in-depth longitudinal cases of youth’s iterative design work: Nila’s light-up #stopracism sign; Su’zanne’s massaging slipper, Sharon’s geodesic play dome, and Jazmyn’s portable fan. Across cases, we illustrate three findings. First, youth located broader injustices within local making/inventing discourses with support from community and STEM allies, suggesting youth drew from multiple epistemologies, some grounded in community cultural wealth, others in STEM. For example, Su’Zanne drew from a familial culture of care and resistance in recognizing injustices nested in homelessness while iterating a way to make her slipper “more massaging.” The geodesic dome youth-makers drew from collective solidarity/resistance in making a structure for younger peers due to unjust lack of play infrastructure. Second, iterative engagement involving community wealth afforded further design and inventing experiences and expanded ownership over inventions across many stakeholders. For example, youth turned Nila’s #stopracism sign on during group discussions when they felt that racism needed to be foregrounded. Third, the afterlife of youth invention processes impacted the emergent inventor-maker culture through influencing the iterative process. Significance Iterations expand hybridization of cultural knowledge/practice and STEM-rich inventing, re-shaping whose cultural knowledge matters, and fostering justice-oriented collective outcomes.« less
  3. Web-based interactions enable agents to coordinate and generate collective action. Coordination can facilitate the spread of contagion to large groups within networked populations. In game theoretic contexts, coordination requires that agents share common knowledge about each other. Common knowledge emerges within a group when each member knows the states and the thresholds (preferences) of the other members, and critically, each member knows that everyone else has this information. Hence, these models of common knowledge and coordination on communication networks are fundamentally di fferent from influence-based unilateral contagion models, such as those devised by Granovetter and Centola. Moreover, these models utilize different mechanisms for driving contagion. We evaluate three mechanisms of a common knowledge model that can represent web-based communication among groups of people on Facebook, using nine social (media) networks. We provide theoretical results indicating the intractability in identifying all node-maximal bicliques in a network, which is the characterizing network structure that produces common knowledge. Bicliques are required for model execution. We also show that one of the mechanisms (named PD2) dominates another mechanism (named ND2). Using simulations, we compute the spread of contagion on these networks in the Facebook model and demonstrate that di fferent mechanisms can produce widelymore »varying behaviors in terms of the extent of the spread and the speed of contagion transmission. We also quantify, through the fraction of nodes acquiring contagion, di erences in the eff ects of the ND2 and PD2 mechanisms, which depend on network structure and other simulation inputs.« less
  4. Web-based interactions enable agents to coordinate and generate collective action. Coordination can facilitate the spread of contagion to large groups within networked populations. In game theoretic contexts, coordination requires that agents share common knowledge about each other. Common knowledge emerges within a group when each member knows the states and the thresholds (preferences) of the other members, and critically, each member knows that everyone else has this information. Hence, these models of common knowledge and coordination on communication networks are fundamentally different from influence-based unilateral contagion models, such as those devised by Granovetter and Centola. Moreover, these models utilize different mechanisms for driving contagion. We evaluate three mechanisms of a common knowledge model that can represent web-based communication among groups of people on Facebook, using nine social (media) networks. We provide theoretical results indicating the intractability in identifying all node-maximal bicliques in a network, which is the characterizing network structure that produces common knowledge. Bicliques are required for model execution. We also show that one of the mechanisms (named PD2) dominates another mechanism (named ND2). Using simulations, we compute the spread of contagion on these networks in the Facebook model and demonstrate that different mechanisms can produce widely varying behaviorsmore »in terms of the extent of the spread and the speed of contagion transmission. We also quantify, through the fraction of nodes acquiring contagion, differences in the effects of the ND2 and PD2 mechanisms, which depend on network structure and other simulation inputs.« less
  5. Recent years have witnessed the emerging of conversational systems, including both physical devices and mobile-based applications, such as Amazon Echo, Google Now, Microsoft Cortana, Apple Siri, and many others. Both the research community and industry believe that conversational systems will have a major impact on human-computer interaction, and specifically, the IR community has begun to focus on Conversational Search. Conversational search based on user-system dialog exhibits major differences from conventional search in that 1) the user and system can interact for multiple semantically coherent rounds on a task through natural language dialog, and 2) it becomes possible for the system to understand user needs or to help users clarify their needs by asking appropriate questions from the users directly. In this paper, we propose and evaluate a unified conversational search framework. Specifically, we define the major components for conversational search, assemble them into a unified framework, and test an implementation of the framework using a conversational product search scenario in Amazon. To accomplish this, we propose the Multi-Memory Network (MMN) architecture, which is end-to-end trainable based on large-scale collections of user reviews in e-commerce. The system is capable of asking aspect-based questions in the right order so as to understandmore »user needs, while (personalized) search is conducted during the conversation and results are provided when the system feels confident. Experiments on real-world user purchasing data verified the advantages of conversational search against conventional search algorithms in terms of standard evaluation measures such as NDCG.« less