More Like this

1. Generative AI and Discovery of Preferences for Single-Use Plastics Regulations

   https://doi.org/10.1609/aaaiss.v2i1.27651  

   Hollauer, Catharina; Garcelán, Jorge; Ragam, Nikhita; Vaish, Tia; Isaac_Asensio, Omar (January 2024, Proceedings of the AAAI Symposium Series)

   Given the heightened global awareness and attention to the negative externalities of plastics use, many state and local governments are considering legislation that will limit single-use plastics for consumers and retailers under extended producer responsibility laws. Considering the growing momentum of these single-use plastics regulations globally, there is a need for reliable and cost-effective measures of the public response to this rulemaking for inference and prediction. Automated computational approaches such as generative AI could enable real-time discovery of consumer preferences for regulations but have yet to see broad adoption in this domain due to concerns about evaluation costs and reliability across large-scale social data. In this study, we leveraged the zero and few-shot learning capabilities of GPT-4 to classify public sentiment towards regulations with increasing complexity in expert prompting. With a zero-shot approach, we achieved a 92% F1 score (s.d. 1%) and 91% accuracy (s.d. 1%), which resulted in three orders of magnitude lower research evaluation cost at 0.138 pennies per observation. We then use this model to analyze 5,132 tweets related to the policy process of the California SB-54 bill, which mandates user fees and limits plastic packaging. The policy study reveals a 12.4% increase in opposing public sentiment immediately after the bill was enacted with no significant changes earlier in the policy process. These findings shed light on the dynamics of public engagement with lower cost models for research evaluation. We find that public opposition to single-use plastics regulations becomes evident in social data only when a bill is effectively enacted. 

   more » « less
2. Can Large Language Models Transform Computational Social Science?

   https://doi.org/10.1162/coli_a_00502  

   Ziems, Caleb; Held, William; Shaikh, Omar; Chen, Jiaao; Zhang, Zhehao; Yang, Diyi (January 2024, Computational Linguistics)

   Abstract Large language models (LLMs) are capable of successfully performing many language processing tasks zero-shot (without training data). If zero-shot LLMs can also reliably classify and explain social phenomena like persuasiveness and political ideology, then LLMs could augment the computational social science (CSS) pipeline in important ways. This work provides a road map for using LLMs as CSS tools. Towards this end, we contribute a set of prompting best practices and an extensive evaluation pipeline to measure the zero-shot performance of 13 language models on 25 representative English CSS benchmarks. On taxonomic labeling tasks (classification), LLMs fail to outperform the best fine-tuned models but still achieve fair levels of agreement with humans. On free-form coding tasks (generation), LLMs produce explanations that often exceed the quality of crowdworkers’ gold references. We conclude that the performance of today’s LLMs can augment the CSS research pipeline in two ways: (1) serving as zero-shot data annotators on human annotation teams, and (2) bootstrapping challenging creative generation tasks (e.g., explaining the underlying attributes of a text). In summary, LLMs are posed to meaningfully participate in social science analysis in partnership with humans. 

   more » « less
3. Public Reaction to Scientific Research via Twitter Sentiment Prediction

   https://doi.org/10.2478/jdis-2022-0003  

   Shahzad, Murtuza; Alhoori, Hamed (December 2021, Journal of Data and Information Science)

   Abstract Purpose Social media users share their ideas, thoughts, and emotions with other users. However, it is not clear how online users would respond to new research outcomes. This study aims to predict the nature of the emotions expressed by Twitter users toward scientific publications. Additionally, we investigate what features of the research articles help in such prediction. Identifying the sentiments of research articles on social media will help scientists gauge a new societal impact of their research articles. Design/methodology/approach Several tools are used for sentiment analysis, so we applied five sentiment analysis tools to check which are suitable for capturing a tweet's sentiment value and decided to use NLTK VADER and TextBlob. We segregated the sentiment value into negative, positive, and neutral. We measure the mean and median of tweets’ sentiment value for research articles with more than one tweet. We next built machine learning models to predict the sentiments of tweets related to scientific publications and investigated the essential features that controlled the prediction models. Findings We found that the most important feature in all the models was the sentiment of the research article title followed by the author count. We observed that the tree-based models performed better than other classification models, with Random Forest achieving 89% accuracy for binary classification and 73% accuracy for three-label classification. Research limitations In this research, we used state-of-the-art sentiment analysis libraries. However, these libraries might vary at times in their sentiment prediction behavior. Tweet sentiment may be influenced by a multitude of circumstances and is not always immediately tied to the paper's details. In the future, we intend to broaden the scope of our research by employing word2vec models. Practical implications Many studies have focused on understanding the impact of science on scientists or how science communicators can improve their outcomes. Research in this area has relied on fewer and more limited measures, such as citations and user studies with small datasets. There is currently a critical need to find novel methods to quantify and evaluate the broader impact of research. This study will help scientists better comprehend the emotional impact of their work. Additionally, the value of understanding the public's interest and reactions helps science communicators identify effective ways to engage with the public and build positive connections between scientific communities and the public. Originality/value This study will extend work on public engagement with science, sociology of science, and computational social science. It will enable researchers to identify areas in which there is a gap between public and expert understanding and provide strategies by which this gap can be bridged. 

   more » « less
4. Topics and Sentiments of Public Concerns Regarding COVID-19 Vaccines: Social Media Trend Analysis

   https://doi.org/10.2196/30765  

   Monselise, Michal; Chang, Chia-Hsuan; Ferreira, Gustavo; Yang, Rita; Yang, Christopher C (January 2021, Journal of Medical Internet Research)

   Background As a number of vaccines for COVID-19 are given emergency use authorization by local health agencies and are being administered in multiple countries, it is crucial to gain public trust in these vaccines to ensure herd immunity through vaccination. One way to gauge public sentiment regarding vaccines for the goal of increasing vaccination rates is by analyzing social media such as Twitter. Objective The goal of this research was to understand public sentiment toward COVID-19 vaccines by analyzing discussions about the vaccines on social media for a period of 60 days when the vaccines were started in the United States. Using the combination of topic detection and sentiment analysis, we identified different types of concerns regarding vaccines that were expressed by different groups of the public on social media. Methods To better understand public sentiment, we collected tweets for exactly 60 days starting from December 16, 2020 that contained hashtags or keywords related to COVID-19 vaccines. We detected and analyzed different topics of discussion of these tweets as well as their emotional content. Vaccine topics were identified by nonnegative matrix factorization, and emotional content was identified using the Valence Aware Dictionary and sEntiment Reasoner sentiment analysis library as well as by using sentence bidirectional encoder representations from transformer embeddings and comparing the embedding to different emotions using cosine similarity. Results After removing all duplicates and retweets, 7,948,886 tweets were collected during the 60-day time period. Topic modeling resulted in 50 topics; of those, we selected 12 topics with the highest volume of tweets for analysis. Administration and access to vaccines were some of the major concerns of the public. Additionally, we classified the tweets in each topic into 1 of the 5 emotions and found fear to be the leading emotion in the tweets, followed by joy. Conclusions This research focused not only on negative emotions that may have led to vaccine hesitancy but also on positive emotions toward the vaccine. By identifying both positive and negative emotions, we were able to identify the public's response to the vaccines overall and to news events related to the vaccines. These results are useful for developing plans for disseminating authoritative health information and for better communication to build understanding and trust. 

   more » « less
5. Dependable Public Ledger for Policy Compliance, a Blockchain Based Approach

   Wu, Zhou; Williams, Andrew; Perouli, Debbie (July 2019, 39th IEEE International Conference on Distributed Computing Systems (ICDCS 2019))

   The ever increasing amount of personal data accumulated by companies offering innovative services through the cloud, Internet of Things devices and, more recently, social robots has started to alert consumers and legislative authorities. In the advent of the first modern laws trying to protect user privacy, such as the European Union General Data Protection Regulation, it is still unclear what are the tools and techniques that the industry should employ to comply with regulations in a transparent and cost effective manner. We propose an architecture for a public blockchain based ledger that can provide strong evidence of policy compliance. To address scalability concerns, we define a new type of off-chain channel that is based on general state channels and offers verification for information external to the blockchain. We also create a model of the business relationships in a smart home setup that includes a social robot and suggest a sticky policy mechanism to monitor cross-boundary policy compliance. 

   more » « less