More Like this

1. Collective Obfuscation and Crowdsourcing

   Laufer, B.; Grupen, N. (August 2022, KDD MIS2-TrueFact and ICML DisCoML Workshops)

   Crowdsourcing technologies rely on groups of people to input information that may be critical for decision-making. This work examines obfuscation in the context of reporting technologies. We show that widespread use of reporting platforms comes with unique security and privacy implications, and introduce a threat model and corresponding taxonomy to outline some of the many attack vectors in this space. We then perform an empirical analysis of a dataset of call logs from a controversial, real-world reporting hotline and identify coordinated obfuscation strategies that are intended to hinder the platform's legitimacy. We propose a variety of statistical measures to quantify the strength of this obfuscation strategy with respect to the structural and semantic characteristics of the reporting attacks in our dataset. 

   more » « less
2. Bayesian Crowdsourcing with Constraints

   P. A. Traganitis, G. B. (January 2021, Proceedings of European Conf. on Machine Learning)

   null (Ed.)
3. Detecting adversaries in Crowdsourcing

   https://doi.org/10.1109/ICDM51629.2021.00174  

   Traganitis, Panagiotis A.; Giannakis, Georgios B. (November 2021, International Conference on Data Mining)
4. Crowdsourcing Perceptions of Gerrymandering

   https://doi.org/10.1609/hcomp.v10i1.21993  

   Kelly, Benjamin; Kang, Inwon; Xia, Lirong (October 2022, Proceedings of the AAAI Conference on Human Computation and Crowdsourcing)

   Gerrymandering is the manipulation of redistricting to influence the results of a set of elections for local representatives. Gerrymandering has the potential to drastically swing power in legislative bodies even with no change in a population’s political views. Identifying gerrymandering and measuring fairness using metrics of proposed district plans is a topic of current research, but there is less work on how such plans will be perceived by voters. Gathering data on such perceptions presents several challenges such as the ambiguous definitions of ‘fair’ and the complexity of real world geography and district plans. We present a dataset collected from an online crowdsourcing platform on a survey asking respondents to mark which of two maps of equal population distribution but different districts appear more ‘fair’ and the reasoning for their decision. We performed preliminary analysis on this data and identified which of several commonly suggested metrics are most predictive of the responses. We found that the maximum perimeter of any district was the most predictive metric, especially with participants who reported that they made their decision based on the shape of the districts. 

   more » « less
5. Identifying Dependent Annotators in Crowdsourcing

   https://doi.org/10.1109/IEEECONF56349.2022.10052052  

   Traganitis, Panagiotis A.; Giannakis, Georgios B. (October 2022, Asilomar Conference on Signals Systems and Computers)

   Crowdsourcing is the learning paradigm that aims to combine noisy labels provided by a crowd of human annotators. To facilitate this label fusion, most contemporary crowdsourcing methods assume conditional independence between different annotators. Nevertheless, in many cases this assumption may not hold. This work investigates the effects of groups of correlated annotators in multiclass crowdsourced classification. To deal with this setup, a novel approach is developed to identify groups of dependent annotators via second-order moments of annotator responses. This in turn, enables appropriate dependence aware aggregation of annotator responses. Preliminary tests on synthetic and real data showcase the potential of the proposed approach. 

   more » « less