Search for: All records

Crowdworkers depend on Amazon Mechanical Turk (AMT) as an important source of income and it is left to workers to determine which tasks on AMT are fair and worth completing. While there are existing tools that assist workers in making these decisions, workers still spend significant amounts of time finding fair labor. Difficulties in this process may be a contributing factor in the imbalance between the median hourly earnings ($2.00/hour) and what the average requester pays ($11.00/hour). In this paper, we study how novices and experts select what tasks are worth doing. We argue that differences between the two populations likely lead to the wage imbalances. For this purpose, we first look at workers' comments in TurkOpticon (a tool where workers share their experience with requesters on AMT). We use this study to start to unravel what fair labor means for workers. In particular, we identify the characteristics of labor that workers consider is of "good quality'' and labor that is of "poor quality'' (e.g., work that pays too little.) Armed with this knowledge, we then conduct an experiment to study how experts and novices rate tasks that are of both good and poor quality. Through our research we uncover that experts and novices both treat good quality labor in the same way. However, there are significant differences in how experts and novices rate poor quality labor, and whether they believe the poor quality labor is worth doing. This points to several future directions, including machine learning models that support workers in detecting poor quality labor, and paths for educating novice workers on how to make better labor decisions on AMT. 

Flexible, contingent, or 'agile,' working arrangements provide workers with greater autonomy over when, where, or how to fulfill their responsibilities. In search of increased productivity and reduced absenteeism, organizations have increasingly turned to flexible work arrangements. Although access to flexible work arrangements is more prevalent among high-skilled workers, in the form of flextime or co-working, the past decade has also witnessed growth of independent contractors, digital nomadism, digitally enabled crowdwork, online freelancing, and on-demand platform labor. Flexible work arrangements reduce commutes and can enable workers with care-responsibilities to stay in the workforce. Younger workers also see flexibility as a top priority when considering career opportunities. Flexible working arrangements can also be mutually beneficial, enabling organizations to scale dynamically. Specific skill sets can be immediately accessed by turning to freelancers to fill organizational gaps. A growing number of organizations and workers rely on short-term and project-based relationships, using online platforms such as Upwork or Fiverr to connect. However, flexible work arrangements often come entwined with precarity cloaked in emancipatory narratives. Fixed salaries and benefits have given way to hourly rates and quantified ratings. Flexible workers often face unpredictability and uncertainty as they carry more risk and responsibility, and are burdened with a great portion of administrative costs (that is, overhead) associated with organizational support systems. Flexible workers at Google, for instance, outnumber full time workers but face far more unpredictability. Current formulations consider organizations as relatively fixed 'containers', which encapsulate the work performed and the information and communications technology (ICT) systems used to perform it.12 However, flexible work arrangements take place outside of organizational containers. In this new sociotechnical dynamic, flexible workers interact with a diversity of digital tools that defy centralized, top-down standardization or governance. We capture this diversity of digital tools through the concept of Personal Digital Infrastructures (PDIs), which denote an individualized assemblage of tools and technologies, such as personal laptops, smartphones, cloud services, and applications brought together by workers to perform their work tasks. Yet, flexible workers constantly reconfigure their PDIs as the technology landscape, client-relationship, and task requirements shift. For flexible work arrangements to be mutually beneficial, PDI integration in ICT systems for work is increasingly necessary, beyond a narrow focus on enterprise systems supporting standard work. Our collective research on flexible work arrangements indicates that PDIs present non-trivial challenges, but a more effective design of ICT systems for work can facilitate the integration of these bottom-up infrastructures. The nuanced understanding of PDIs presented here highlights their interplay with flexible work arrangements across key dimensions (spatial, temporal, organizational, and technological) and suggests key priorities for technology and platform developers. 

The artificial intelligence (AI) industry has created new jobs that are essential to the real world deployment of intelligent systems. Part of the job focuses on labeling data for machine learning models or having workers complete tasks that AI alone cannot do. These workers are usually known as ‘crowd workers’—they are part of a large distributed crowd that is jointly (but separately) working on the tasks although they are often invisible to end-users, leading to workers often being paid below minimum wage and having limited career growth. In this chapter, we draw upon the field of human–computer interaction to provide research methods for studying and empowering crowd workers. We present our Computational Worker Leagues which enable workers to work towards their desired professional goals and also supply quantitative information about crowdsourcing markets. This chapter demonstrates the benefits of this approach and highlights important factors to consider when researching the experiences of crowd workers. 

Crowdsourcing markets provide workers with a centralized place to find paid work. What may not be obvious at first glance is that, in addition to the work they do for pay, crowd workers also have to shoulder a variety of unpaid invisible labor in these markets, which ultimately reduces workers' hourly wages. Invisible labor includes finding good tasks, messaging requesters, or managing payments. However, we currently know little about how much time crowd workers actually spend on invisible labor or how much it costs them economically. To ensure a fair and equitable future for crowd work, we need to be certain that workers are being paid fairly for ALL of the work they do. In this paper, we conduct a field study to quantify the invisible labor in crowd work. We build a plugin to record the amount of time that 100 workers on Amazon Mechanical Turk dedicate to invisible labor while completing 40,903 tasks. If we ignore the time workers spent on invisible labor, workers' median hourly wage was $3.76. But, we estimated that crowd workers in our study spent 33% of their time daily on invisible labor, dropping their median hourly wage to $2.83. We found that the invisible labor differentially impacts workers depending on their skill level and workers' demographics. The invisible labor category that took the most time and that was also the most common revolved around workers having to manage their payments. The second most time-consuming invisible labor category involved hyper-vigilance, where workers vigilantly watched over requesters' profiles for newly posted work or vigilantly searched for labor. We hope that through our paper, the invisible labor in crowdsourcing becomes more visible, and our results help to reveal the larger implications of the continuing invisibility of labor in crowdsourcing.