skip to main content


Title: Challenges and Opportunities in Information Manipulation Detection: An Examination of Wartime Russian Media
NLP research on public opinion manipulation campaigns has primarily focused on detecting overt strategies such as fake news and disinformation. However, information manipulation in the ongoing Russia-Ukraine war exemplifies how governments and media also employ more nuanced strategies. We release a new dataset, VoynaSlov, containing 38M+ posts from Russian media outlets on Twitter and VKontakte, as well as public activity and responses, immediately preceding and during the 2022 Russia-Ukraine war. We apply standard and recently-developed NLP models on VoynaSlov to examine agenda setting, framing, and priming, several strategies underlying information manipulation, and reveal variation across media outlet control, social media platform, and time. Our examination of these media effects and extensive discussion of current approaches’ limitations encourage further development of NLP models for understanding information manipulation in emerging crises, as well as other real-world and interdisciplinary tasks.  more » « less
Award ID(s):
2142739
PAR ID:
10433105
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Empirical Methods in Natural Language Processing
Page Range / eLocation ID:
5209–5235
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Collecting public opinion data is challenging in the shadow of war. And yet accurate public opinion is crucial. Political elites rely on it and often attempt to influence it. Therefore, it is incumbent on researchers to provide independent and reliable wartime polls. However, surveying in wartime presents a distinctive set of challenges. We outline two challenges facing polling in war: under-coverage and response bias. We highlight these challenges in the context of the Russia– Ukraine war, drawing on original panel survey data tracing the attitudes of the same people prior to and after Russia’s full-scale invasion of Ukraine in 2022. We conclude with some lessons for those employing survey methods in wartime, and point to steps forward, in Ukraine and beyond. 
    more » « less
  2. Amidst growing concern over media manipulation, NLP attention has focused on overt strategies like censorship and “fake news”. Here, we draw on two concepts from the political science literature to explore subtler strategies for government media manipulation: agenda-setting (selecting what topics to cover) and framing (deciding how topics are covered). We analyze 13 years (100K articles) of the Russian newspaper Izvestia and identify a strategy of distraction: articles mention the U.S. more frequently in the month directly following an economic downturn in Russia. We introduce embedding-based methods for cross-lingually projecting English frames to Russian, and discover that these articles emphasize U.S. moral failings and threats to the U.S. Our work offers new ways to identify subtle media manipulation strategies at the intersection of agenda-setting and framing. 
    more » « less
  3. Research shows that people’s perceptions of historical violence shape many present-day outcomes. Yet it is also plausible that people emphasize or downplay certain events of the past based on how these resonate with their beliefs and identities today. With a population of diverse orientations involving Russia and Europe, Ukraine in 2019 was an important case for exploring how people’s present geopolitical orientations shaped perceptions of victimization in World War II. Drawing on a survey experiment, we find evidence for “motivated reasoning” among Western-oriented respondents, who emphasized their family’s suffering in World War II when faced with information that attributed blame to the Soviet regime. We find no evidence for motivated reasoning among the Russian-oriented respondents 
    more » « less
  4. The Russian-Ukrainian conflict spawned a high-intensity war that shattered decades of peace in Europe. The use of drones and social media elevates open-source intelligence as a critical strategic asset. However, information from these sources is sporadic, difficult to confirm, and prone to manipulation. Here, we use open-access spaceborne remote sensing data to probe the damage to infrastructure on and off the frontline at the city, region, and country-wide scales in Ukraine. Nighttime light data and Synthetic Aperture Radar images reveal widespread blackout and unveil the destruction of battleground cities, offering contrasted perspectives on the impact of the conflict. Optical satellite images capture extensive flooding along the Dnipro River in the aftermath of the breach of the Kakhovka dam. Leveraging visible, near-infrared, and microwave satellite data, we bring to light disruption of human activities, havoc in the environment, and the annihilation of entire cities during the protracted conflict. Open-source remote sensing can offer objective information about the nature and extent of devastation during military conflicts. 
    more » « less
  5. Ukraine is a very large and diverse country, and the least we can do amidst the massive trauma of Russia’s invasion is to acknowledge and respect its socio-cultural and geographic complexity. While there is strong evidence that Russia’s invasion of Ukraine has shifted public opinion towards the West, researchers have an obligation to convey the difficulties in gathering sensitive survey data in war zones and, thus, temper how data are generalized and represented in public discourse. This requires nuance when discussing the preferences of Ukrainians from all areas, including those in exile or living under Russian control, and greater efforts to communicate uncertainty. 
    more » « less