skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Hiding in Plain Sight: Secondary Analysis of Data Records as a Method for Learning about Citizen Science Projects and Volunteers’ Skills
Title: Hiding in Plain Sight: Secondary Analysis of Data Records as a Method for Learning about Citizen Science Projects and Volunteers’ Skills
This paper is the culmination of several facilitated exercises and meetings between external researchers and five citizen science (CS) project teams who analyzed existing data records to understand CS volunteers’ accuracy and skills. CS teams identified a wide range of skill variables that were “hiding in plain sight” in their data records, and that could be explored as part of a secondary analysis, which we define here as analyses based on data already possessed by the project. Each team identified a small number of evaluation questions to explore with their existing data. Analyses focused on accurate data collection and all teams chose to add complementary records that documented volunteers’ project engagement or the data collection context to their analysis. Most analyses were conducted as planned, and included a range of approaches from correlation analyses to general additive models. Importantly, the results from these analyses were then used to inform the design of both existing and new CS projects, and to inform the field more broadly through a range of dissemination strategies. We conclude by sharing ways that others might consider pursuing their own secondary analysis to help fill gaps in our current understanding related to volunteer skills.  more » « less
Award ID(s):
1713424
PAR ID:
10476229
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Ubiquity Press
Date Published:
Journal Name:
Citizen Science: Theory and Practice
Volume:
7
Issue:
1
ISSN:
2057-4991
Page Range / eLocation ID:
35
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Citizen Science: Theory and Practice)
    This paper explores the assumptions that citizen science (CS) project leaders had about their volunteers’ science inquiry skill–proficiency overall, and then examines volunteers’ actual proficiency in one specific skill, scientific observation, because it is fundamental to and shared by many projects. This work shares findings from interviews with 10 project leaders related to two common assumptions leaders have about their volunteers’ skill proficiency: one, that volunteers can perform the necessary skills to participate at the start of a CS project, and therefore may not need training; and two, volunteer skill proficiency improves over time through involvement in the CS project. In order to answer questions about the degree of accuracy to which volunteers can perform the necessary skills and about differences in their skill proficiency based on experience and data collection procedures, we analyzed data from seven CS projects that used two shared embedded assessment tools, each focused on skills within the context of scientific observation in natural settings: Notice relevant features for taxonomic identification and record standard observations. This across-project and cross-sectional study found that the majority of citizen science volunteers (n = 176) had the necessary skill proficiency to collect accurate scientific observations but proficiency varied based on volunteer experience and project data collection procedures. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, Citizen Science: Theory and Practice)
    Fortson, Lucy; Crowston, Kevin; Kloetzer, Laure; Ponti, Marisa (Ed.)
    In the era of rapidly growing astronomical data, the gap between data collection and analysis is a significant barrier, especially for teams searching for rare scientific objects. Although machine learning (ML) can quickly parse large data sets, it struggles to robustly identify scientifically interesting objects, a task at which humans excel. Human-in-the-loop (HITL) strategies that combine the strengths of citizen science (CS) and ML offer a promising solution, but first, we need to better understand the relationship between human- and machine-identified samples. In this work, we present a case study from the Galaxy Zoo: Weird & Wonderful project, where volunteers inspected ~200,000 astronomical images—processed by an ML-based anomaly detection model—to identify those with unusual or interesting characteristics. Volunteer-selected images with common astrophysical characteristics had higher consensus, while rarer or more complex ones had lower consensus. This suggests low-consensus choices shouldn’t be dismissed in further explorations. Additionally, volunteers were better at filtering out uninteresting anomalies, such as image artifacts, which the machine struggled with. We also found that a higher ML-generated anomaly score that indicates images’ low-level feature anomalousness was a better predictor of the volunteers’ consensus choice. Combining a locus of high volunteer-consensus images within the ML learnt feature space and anomaly score, we demonstrated a decision boundary that can effectively isolate images with unusual and potentially scientifically interesting characteristics. Using this case study, we lay important guidelines for future research studies looking to adapt and operationalize human-machine collaborative frameworks for efficient anomaly detection in big data. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; ; et al (, PLOS ONE)
    Olu, Olushayo Oluseun (Ed.)
     
    more » « less
  4. ; ; ; (, ASEE Global Colloquium on Engineering Education)
    There are several changes anticipated in computer science (CS) education over the next decade, including updated student standards, rapidly changing impacts of artificial intelligence (AI), and an increasing number of school systems requiring a CS class for graduation. In order to prepare for these changes – as well as to address the equity issues that have plagued CS since its inception – we engaged in a project designed to reimagine content and pathways for high school CS education. As a collaborative project, we hosted multiple events for relevant parties (including K-12 educators and administrators, higher education faculty, industry professionals, state and district CS supervisors, and CS education researchers). These events were designed to collaboratively seek input for the creation of a series of reports recommending what a CS course that satisfies a high school graduation requirement should include, how that course should align with Advanced Placement (AP) and post-secondary CS instruction, and what pathways should exist for students after that introductory high school course. The portion of the project highlighted in this article contains an analysis of data collected from focus groups (n=21), interviews (n=10), and an in-person convening of participants from K-12, post-secondary, industry, and administrative roles (n=35). The data is centered on determining what CS content is essential for all high school students. Participants considered knowledge, skills, and dispositions across a range of CS and CS-adjacent topics and, through a variety of activities, described what new content should be taught when viewing through the lens of teaching CS to high school students in the year 2030 and what content should be prioritized. Our analysis sought to delineate and synthesize their sentiments. Six major priorities emerged from our analysis: societal impacts and ethical issues, algorithmic thinking, data and analysis, inclusive computing culture, AI, and career knowledge. The significance of our findings is that they present a broad overview of what a variety of relevant parties consider to be the most important CS content for high school students; this information is important for educators, administrators, and those who develop curriculum, standards, and/or teaching tools. 
    more » « less
  5. ; ; ; ; ; (, Science and Children)
    This article portrays how citizen science (CS) projects can be integrated into elementary classrooms to enhance students’ sensemaking skills and connect to real-world science problems. For the last several years, we have been involved in a study, Teacher Learning for Effective School-Based Citizen Science (TL4CS), that developed materials for elementary school teachers to engage their students in data collection, analysis, and interpretation for two existing CS projects: Community Collaborative Rain, Hail, and Snow Network (CoCoRaHS) and the Lost Ladybug Project (LLP). After piloting the TL4CS materials for two years, two teachers, Penny and Amy, share the ways they used the materials to create rich sensemaking experiences for their students. Penny used our TL4CS CoCoRaHS materials to make connections between their daily precipitation data and local weather phenomena, patterns in ecosystems, and student-created graphs. Amy used our TL4CS LLP materials to explore students’ questions about human impact on animals’ habitats and discover the importance of biodiversity in ecosystems. As demonstrated by Penny’s and Amy’s stories, the TL4CS materials can transform mere data collection for CS projects into opportunities for real-world connections and sensemaking in science classrooms. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email