An official website of the United States government
Crowdsourcing platforms have traditionally been designed with a focus on workstation interfaces, restricting the flexibility that crowdworkers need. Recognizing this limitation and the need for more adaptable platforms, prior research has highlighted the diverse work processes of crowdworkers, influenced by factors such as device type and work stage. However, these variables have largely been studied in isolation. Our study is the first to explore the interconnected variabilities among these factors within the crowdwork community. Through a survey involving 150 Amazon Mechanical Turk crowdworkers, we uncovered three distinct groups characterized by their interrelated variabilities in key work aspects. The largest group exhibits a reliance on traditional devices, showing limited interest in integrating smartphones and tablets into their work routines. The second-largest group also primarily uses traditional devices but expresses a desire for supportive tools and scripts that enhance productivity across all devices, particularly smartphones and tablets. The smallest group actively uses and strongly prefers non-workstation devices, especially smartphones and tablets, for their crowdworking activities. We translate our findings into design insights for platform developers, discussing the implications for creating more personalized, flexible, and efficient crowdsourcing environments. Additionally, we highlight the unique work practices of these crowdworker clusters, offering a contrast to those of more traditional and established worker groups.
more »
« less
Mapping the Crowdsourcing Workforce in Latin America and the Caribbean
Research has primarily focused on understanding the perspectives and experiences of US based crowdworkers, leaving a notable gap to those within the Global South. To bridge this, we conducted a survey with 100 crowdworkers across 16 Latin American and Caribbean countries. To understand the tensions between the crowdworking economy abroad. In our study we found crowd work being the stepping stone to financial and professional independence among those we surveyed along with the tensions within data extraction among crowdworkers.
more »
« less
- Award ID(s):
- 2339443
- PAR ID:
- 10590840
- Publisher / Repository:
- Avances en Interacción Humano-Computadora
- Date Published:
- Journal Name:
- Avances en Interacción Humano-Computadora
- Volume:
- 9
- Issue:
- 1
- ISSN:
- 2594-2352
- Page Range / eLocation ID:
- 146 to 153
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Exposing students to low-quality assessments such as multiple-choice questions (MCQs) and short answer questions (SAQs) is detrimental to their learning, making it essential to accurately evaluate these assessments. Existing evaluation methods are often challenging to scale and fail to consider their pedagogical value within course materials. Online crowds offer a scalable and cost-effective source of intelligence, but often lack necessary domain expertise. Advancements in Large Language Models (LLMs) offer automation and scalability, but may also lack precise domain knowledge. To explore these trade-offs, we compare the effectiveness and reliability of crowdsourced and LLM-based methods for assessing the quality of 30 MCQs and SAQs across six educational domains using two standardized evaluation rubrics. We analyzed the performance of 84 crowdworkers from Amazon's Mechanical Turk and Prolific, comparing their quality evaluations to those made by the three LLMs: GPT-4, Gemini 1.5 Pro, and Claude 3 Opus. We found that crowdworkers on Prolific consistently delivered the highest-quality assessments, and GPT-4 emerged as the most effective LLM for this task. Our study reveals that while traditional crowdsourced methods often yield more accurate assessments, LLMs can match this accuracy in specific evaluative criteria. These results provide evidence for a hybrid approach to educational content evaluation, integrating the scalability of AI with the nuanced judgment of humans. We offer feasibility considerations in using AI to supplement human judgment in educational assessment.more » « less
-
This paper contributes to understanding the work of teaching the university geometry courses that are taken by prospective secondary teachers. We ask what are the tensions that instructors need to manage as they plan and teach these courses. And we use these tensions to argue that mathematics instruction in geometry courses for secondary teachers includes complexities that go beyond those of other undergraduate mathematics courses–an argument that possibly applies to other mathematics courses for teachers. Building on the notion that the work of teaching involves managing tensions, and relying on interviews of 32 instructors, we characterize 5 tensions (content, experiences, students, instructor, and institutions) that instructors of geometry for teachers manage in their work. We interpret these tensions as emerging from a dialectic between two normative understandings of instruction in these courses, using the instructional triangle to represent these.more » « less
-
null (Ed.)The Metaverse, a term coined in science fiction, is now being discussed seriously as a new form of infra-structure. The Metaverse is intended to make possible thematically interconnected immersive experiences. In this paper, we conceptualize the Metaverse as a meta design space. Within this space, designers create var-ious interconnected design spaces. We highlight how the key dimensions of human experience (time, space, actors, and artifacts) each introduce tensions for making decisions in those design spaces, and we highlight the transitions between design spaces. This conceptual language opens up this novel and emergent phenomenon both to those wishing to design new disruptive technologies and those seeking to improve existing platform strategies. We conclude by highlighting how the Metaverse will not only comprise immersive virtual experiences but also transitions between physical and virtual experiences.more » « less
-
Background: While advocates for integrating Computational Thinking (CT) into existing K12 classrooms have acknowledged and aimed to address various barriers to implementation, we contend that a more foundational issue—tensions between the epistemology of computing and those of existing disciplines—has largely been overlooked. Studies of contact between heterogeneous disciplinary perspectives in both pedagogical and real world professional settings point to other risks, and harms, that educators may need to consider as they attempt to integrate CT into their teaching. As such, designing for integrated CT pedagogies does not simply require addressing functional problems such as teacher professional learning and limited classroom time, but rather implicates complex epistemological navigations. Objective: This manuscript explores epistemic tensions between Computational Thinking (CT) and K12 humanities and arts disciplines and possibilities for their resolution. Method: Based on a Delphi study with 43 experts from three disciplines—language arts, social studies, and arts—as they engaged in 20 hours of focus group conversations exploring potential approaches to integrating CT these disciplines, analysis focused on identifying perceived epistemic tensions that can arise in the context of instruction and directions for their resolution. Findings: We found 5 epistemic tensions that are explored in detail: contextual reductionism, procedural reductionism, epistemic chauvinism, threats to epistemic identities, and epistemic convergence, as well as a number of potential directions for navigating them. Implications: The study’s findings provide insights that bear on both scholarship and pedagogical design aimed at promoting substantive interdisciplinary learning with CT, and, critically, navigating potential tensions that can arise within it.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.47756/aihc.y9i1.160
