skip to main content

Title: Conversations Gone Alright: Quantifying and Predicting Prosocial Outcomes in Online Conversations
Online conversations can go in many directions: some turn out poorly due to antisocial behavior, while others turn out positively to the benefit of all. Research on improving online spaces has focused primarily on detecting and reducing antisocial behavior. Yet we know little about positive outcomes in online conversations and how to increase them—is a prosocial outcome simply the lack of antisocial behavior or something more? Here, we examine how conversational features lead to prosocial outcomes within online discussions. We introduce a series of new theory-inspired metrics to define prosocial outcomes such as mentoring and esteem enhancement. Using a corpus of 26M Reddit conversations, we show that these outcomes can be forecasted from the initial comment of an online conversation, with the best model providing a relative 24% improvement over human forecasting performance at ranking conversations for predicted outcome. Our results indicate that platforms can use these early cues in their algorithmic ranking of early conversations to prioritize better outcomes.
Authors:
; ; ; ;
Award ID(s):
1850221 2007251
Publication Date:
NSF-PAR ID:
10226970
Journal Name:
The Web Conference
Sponsoring Org:
National Science Foundation
More Like this
  1. One of the main challenges online social systems face is the prevalence of antisocial behavior, such as harassment and personal attacks. In this work, we introduce the task of predicting from the very start of a conversation whether it will get out of hand. As opposed to detecting undesirable behavior after the fact, this task aims to enable early, actionable prediction at a time when the conversation might still be salvaged. To this end, we develop a framework for capturing pragmatic devices---such as politeness strategies and rhetorical prompts---used to start a conversation, and analyze their relation to its future trajectory. Applying this framework in a controlled setting, we demonstrate the feasibility of detecting early warning signs of antisocial behavior in online discussions.
  2. One of the main challenges online social systems face is the prevalence of antisocial behavior, such as harassment and personal attacks. In this work, we introduce the task of predicting from the very start of a conversation whether it will get out of hand. As opposed to detecting undesirable behavior after the fact, this task aims to enable early, actionable prediction at a time when the conversation might still be salvaged. To this end, we develop a framework for capturing pragmatic devices---such as politeness strategies and rhetorical prompts---used to start a conversation, and analyze their relation to its future trajectory. Applying this framework in a controlled setting, we demonstrate the feasibility of detecting early warning signs of antisocial behavior in online discussions.
  3. To address the widespread problem of uncivil behavior, many online discussion platforms employ human moderators to take action against objectionable content, such as removing it or placing sanctions on its authors. Thisreactive paradigm of taking action against already-posted antisocial content is currently the most common form of moderation, and has accordingly underpinned many recent efforts at introducing automation into the moderation process. Comparatively less work has been done to understand other moderation paradigms---such as proactively discouraging the emergence of antisocial behavior rather than reacting to it---and the role algorithmic support can play in these paradigms. In this work, we investigate such a proactive framework for moderation in a case study of a collaborative setting: Wikipedia Talk Pages. We employ a mixed methods approach, combining qualitative and design components for a holistic analysis. Through interviews with moderators, we find that despite a lack of technical and social support, moderators already engage in a number of proactive moderation behaviors, such as preemptively intervening in conversations to keep them on track. Further, we explore how automation could assist with this existing proactive moderation workflow by building a prototype tool, presenting it to moderators, and examining how the assistance it provides might fit intomore »their workflow. The resulting feedback uncovers both strengths and drawbacks of the prototype tool and suggests concrete steps towards further developing such assisting technology so it can most effectively support moderators in their existing proactive moderation workflow.« less
  4. Abstract Background

    Change strategies may leverage interpersonal relationships and conversations to spread teaching innovations among science faculty. Knowledge sharing refers to the process by which individuals transfer information and thereby spread innovative ideas within an organization. We use knowledge sharing as a lens for identifying factors that encourage productive teaching-related conversations between individuals, characterizing the context and content of these discussions, and understanding how peer interactions may shape instructional practices. In this study, we interview 19 science faculty using innovative teaching practices about the teaching-focused conversations they have with different discussion partners.

    Results

    This qualitative study describes characteristics of the relationship between discussion partners, what they discuss with respect to teaching, the amount of help-seeking that occurs, and the perceived impacts of these conversations on their teaching. We highlight the role of office location and course overlap in bringing faculty together and characterize the range of topics they discuss, such as course delivery and teaching strategies. We note the tendency of faculty to seek out partners with relevant expertise and describe how faculty perceive their discussion partners to influence their instructional practices and personal affect. Finally, we elaborate on how these themes vary depending on the relationship between discussion partners.

    Conclusions

    The knowledgemore »sharing framework provides a useful lens for investigating how various factors affect faculty conversations around teaching. Building on this framework, our results lead us to propose two hypotheses for how to promote sharing teaching knowledge among faculty, thereby identifying productive directions for further systematic inquiry. In particular, we propose that productive teaching conversations might be cultivated by fostering collaborative teaching partnerships and developing departmental structures to facilitate sharing of teaching expertise. We further suggest that social network theories and other examinations of faculty behavior can be useful approaches for researching the mechanisms that drive teaching reform.

    « less
  5. Understanding and characterizing how people interact in information-seeking conversations will be a crucial component in developing effective conversational search systems. In this paper, we introduce a new dataset designed for this purpose and use it to analyze information-seeking conversations by user intent distribution, co-occurrence, and flow patterns. The MSDialog dataset is a labeled conversation dataset of question answering (QA) interactions between information seekers and providers from an online forum on Microsoft products. The dataset contains more than 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with user intents on the utterance level. Annotations were done using crowdsourcing. With MSDialog, we find some highly recurring patterns in user intent during an information-seeking process. They could be useful for designing conversational search systems. We will make our dataset freely available to encourage exploration of information-seeking conversation models.