Search for: All records

We present the results of a survey fielded in June of 2022 as a lens to examine recent data reliability issues on Amazon Mechanical Turk. We contrast bad data from this survey with bad data from the same survey fielded among US workers in October 2013, April 2018, and February 2019. Application of an established data cleaning scheme reveals that unusable data has risen from a little over 2% in 2013 to almost 90% in 2022. Through symptomatic diagnosis, we attribute the data reliability drop not to an increase in bad faith work, but rather to a continuum of English proficiency levels. A qualitative analysis of workers’ responses to open-ended questions allows us to distinguish between low fluency workers, ultra-low fluency workers, satisficers, and bad faith workers. We go on to show the effects of the new low fluency work on Likert scale data and on the study’s qualitative results. Attention checks are shown to be much less effective than they once were at identifying survey responses that should be discarded. 

Educational prediction games use the popularity and engagement of fantasy sports as a success model to promote learning in other domains. Fantasy sports motivate players to stay up-to-date with relevant news and explore large statistical data sets, thereby deepening their domain understanding while potentially honing their data analysis skills. We conducted a study of fantasy sports players, and discovered that while some participants performed sophisticated data analysis to support their gameplay, far more relied on news and published commentary. We used results from this study to design a prototype prediction game, Fantasy Climate, which helps players move from intuitions and advice to consuming news and analyzing data by supporting a variety of activities essential to gameplay. Because news is a key component of Fantasy Climate, we evaluated two link-based interfaces to domain-related news, one geospatial and the other organized as a list. The evaluation revealed that news presentation has a strong effect on players' engagement and performance: players using the geospatial interface not only were more engaged in the game; they also made better predictions than players who used the list-based presentation. 

Has widespread news of abuse changed the public's perceptions of how user-contributed content from social networking sites like Facebook and LinkedIn can be used? We collected two datasets that reflect participants' attitudes about content ownership, privacy, and control, one in April 2018, while Cambridge Analytica was still in the news, and another in February 2019, after the event had faded from the headlines, and aggregated the data according to participants' awareness of the story, contrasting the attitudes of those who reported the greatest awareness with those who reported the least. Participants with the greatest awareness of the news story's details have more polarized attitudes about reuse, especially the reuse of content as data. They express a heightened desire for data mobility, greater concern about networked privacy rights, increased skepticism of algorithmically targeted advertising and news, and more willingness for social media platforms to demand corrections of inaccurate or deceptive content.