skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Crowdsourcing to analyze belief systems underlying social issues
People’s beliefs and attitudes about social and scientific issues, such as capital punishment and climate change, appear to form complex but generally coherent networks. Understanding the nature of these networks is a prerequisite for designing interventions for changing beliefs on the basis of rational arguments and evidence. It is therefore important to develop methods to represent and analyze the form and nature of belief networks, which may not be explicitly verbalizable. Adopting an emerging approach that utilizes crowdsourcing to develop educational interventions, we mined discussions from the Reddit forum Change My View to determine which beliefs and types of information underlie people’s attitudes about capital punishment. By combining computational analyses based on a topic model with more qualitative assessments of the extracted topics, we found that moral arguments are more prevalent than statistical or data-based arguments. The present study serves as a test case for the open sourced software crowdpy, a Python toolkit for running naturalistic studies on the web, which will enable other researchers to use crowdsourcing in their research. This approach sets the stage for research exploring potential interventions to change people’s beliefs.  more » « less
Award ID(s):
1827374
PAR ID:
10157636
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the Annual Conference of the Cognitive Science Society
ISSN:
1069-7977
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Punishment can serve as a form of communication: People use punishment to express information to its recipients and interpret punishment between third parties as having communicative content. Prior work on the expressive function of punishment has primarily investigated the capacity of punishment in general to communicate a single type of message – e.g., that the punished behavior violated an important norm. The present work expands this framework by testing whether different types of punishment communicate different messages. We distinguish between person-oriented punishments, which seek to harm the recipient, and action-oriented punishments, which seek to undo a harmful action. We show that people interpret action-oriented punishments, compared to person-oriented punishments, to indicate that the recipient will change for the better (Study 1). The communicative theory can explain this finding if people understand action-oriented punishment to send a message that is more effective than person-oriented punishment at causing such a change. Supporting this explanation, inferences about future behavior track the recipients' beliefs about the punishment they received, rather than the punisher's intentions or the actual punishment imposed (Study 2). Indeed, when actual recipients of a person-oriented punishment believed they received an action-oriented punishment and vice versa, predictions of future behavior tracked the recipients' beliefs rather than reality, and judgments about what the recipients learned from the punishments mediated this effect (Study 3). Together, these studies demonstrate that laypeople think different types of punishment send different messages to recipients and that these messages are differentially effective at bringing about behavioral changes. 
    more » « less
  2. null (Ed.)
    Strategic response options to the COVID-19 pandemic have been greatly influenced by predictive epidemiological models. Effects of non-pharmaceutical interventions (NPIs; such as mask wearing) unfortunately are based on an abundance of very large uncertainties around the extent to which the population adopts risk reducing behaviors. The effects of NPIs appear to have large heterogeneity across regions, subgroups, and individual mindsets and capabilities. We hypothesize that these uncertainties can be improved with higher-fidelity computational modeling of the social-psychological reactions of individuals, groups, and populations. We build up the ACT-R theory and Instance-Based Learning Theory to formulate psychologically valid agents and develop a framework that integrates multi-level cognitive and social simulation with information networks analysis, and epidemiological predictions. We present initial results from analyses of beliefs and sentiments about COVID-19 NPIs induced from online social media that can provide inputs to seed and validate cognitive agents. We present illustrations of cognitive model hypotheses about the dynamics of behavior change in response to intentions, attitudes, messaging, and source credibility. We present an example of social networks propagating attitude change in response to NPIs. 
    more » « less
  3. Narrative visualization is a popular style of data-driven storytelling. Authors use this medium to engage viewers with complex and sometimes controversial issues. A challenge for authors is to not only deliver new information, but to also overcome people’s biases and misconceptions. We study how people adjust their attitudes toward (or away from) a message experienced through a narrative visualization. In a mixed-methods analysis, we investigate whether eliciting participants’ prior beliefs, and visualizing those beliefs alongside actual data, can increase narrative persuasiveness. We find that incorporating priors does not significantly affect attitudinal change. However, participants who externalized their beliefs expressed greater surprise at the data. Their comments also indicated a greater likelihood of acquiring new information, despite the minimal change in attitude. Our results also extend prior findings, showing that visualizations are more persuasive than equivalent textual data representations for exposing contentious issues. We discuss the implications and outline future research directions. 
    more » « less
  4. People form perceptions and interpretations of AI through external sources prior to their interaction with new technology. For example, shared anecdotes and media stories influence prior beliefs that may or may not accurately represent the true nature of AI systems. We hypothesize people's prior perceptions and beliefs will affect human-AI interactions and usage behaviors when using new applications. This paper presents a user experiment to explore the interplay between user's pre-existing beliefs about AI technology, individual differences, and previously established sources of cognitive bias from first impressions with an interactive AI application. We employed questionnaire measures as features to categorize users into profiles based on their prior beliefs and attitudes about technology. In addition, participants were assigned to one of two controlled conditions designed to evoke either positive or negative first impressions during an AI-assisted judgment task using an interactive application. The experiment and results provide empirical evidence that profiling users by surveying them on their prior beliefs and differences can be a beneficial approach for bias (and/or unanticipated usage) mitigation instead of seeking one-size-fits-all solutions. 
    more » « less
  5. Background Social networks such as Twitter offer the clinical research community a novel opportunity for engaging potential study participants based on user activity data. However, the availability of public social media data has led to new ethical challenges about respecting user privacy and the appropriateness of monitoring social media for clinical trial recruitment. Researchers have voiced the need for involving users’ perspectives in the development of ethical norms and regulations. Objective This study examined the attitudes and level of concern among Twitter users and nonusers about using Twitter for monitoring social media users and their conversations to recruit potential clinical trial participants. Methods We used two online methods for recruiting study participants: the open survey was (1) advertised on Twitter between May 23 and June 8, 2017, and (2) deployed on TurkPrime, a crowdsourcing data acquisition platform, between May 23 and June 8, 2017. Eligible participants were adults, 18 years of age or older, who lived in the United States. People with and without Twitter accounts were included in the study. Results While nearly half the respondents—on Twitter (94/603, 15.6%) and on TurkPrime (509/603, 84.4%)—indicated agreement that social media monitoring constitutes a form of eavesdropping that invades their privacy, over one-third disagreed and nearly 1 in 5 had no opinion. A chi-square test revealed a positive relationship between respondents’ general privacy concern and their average concern about Internet research (P<.005). We found associations between respondents’ Twitter literacy and their concerns about the ability for researchers to monitor their Twitter activity for clinical trial recruitment (P=.001) and whether they consider Twitter monitoring for clinical trial recruitment as eavesdropping (P<.001) and an invasion of privacy (P=.003). As Twitter literacy increased, so did people’s concerns about researchers monitoring Twitter activity. Our data support the previously suggested use of the nonexceptionalist methodology for assessing social media in research, insofar as social media-based recruitment does not need to be considered exceptional and, for most, it is considered preferable to traditional in-person interventions at physical clinics. The expressed attitudes were highly contextual, depending on factors such as the type of disease or health topic (eg, HIV/AIDS vs obesity vs smoking), the entity or person monitoring users on Twitter, and the monitored information. Conclusions The data and findings from this study contribute to the critical dialogue with the public about the use of social media in clinical research. The findings suggest that most users do not think that monitoring Twitter for clinical trial recruitment constitutes inappropriate surveillance or a violation of privacy. However, researchers should remain mindful that some participants might find social media monitoring problematic when connected with certain conditions or health topics. Further research should isolate factors that influence the level of concern among social media users across platforms and populations and inform the development of more clear and consistent guidelines. 
    more » « less