An official website of the United States government
Abstract The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with machine learning classifications providing a rapid first-pass classification of the dataset and enabling tiered volunteer training, and volunteer-based classifications verifying the machine classifications, bolstering the machine learning training set and identifying new morphological classes of glitches. These classifications are now routinely used in studies characterizing the performance of the LIGO gravitational-wave detectors. Providing the volunteers with a training framework that teaches them to classify a wide range of glitches, as well as additional tools to aid their investigations of interesting glitches, empowers them to make discoveries of new classes of glitches. This demonstrates that, when giving suitable support, volunteers can go beyond simple classification tasks to identify new features in data at a level comparable to domain experts. The Gravity Spy project is now providing volunteers with more complicated data that includes auxiliary monitors of the detector to identify the root cause of glitches.
more »
« less
Understanding Confusion: A Case Study of Training a Machine Model to Predict and Interpret Consensus From Volunteer Labels
Citizen science has become a valuable and reliable method for interpreting and processing big datasets, and is vital in the era of ever-growing data volumes. However, there are inherent difficulties in the generating labels from citizen scientists, due to the inherent variability between the members of the crowd, leading to variability in the results. Sometimes, this is useful — such as with serendipitous discoveries, which corresponds to rare/unknown classes in the data — but it might also be due to ambiguity between classes. The primary issue is then to distinguish between the intrinsic variability in the dataset and the uncertainty in the citizen scientists’ responses, and leveraging that to extract scientifically useful relationships. In this paper, we explore using a neural network to interpret volunteer confusion across the dataset, to increase the purity of the downstream analysis. We focus on the use of learned features from the network to disentangle feature similarity across the classes, and the ability of the machines’ “attention” in identifying features that lead to confusion. We use data from Jovian Vortex Hunter, a citizen science project to study vortices in Jupiter’s atmosphere, and find that the latent space from the model helps effectively identify different sources of image-level features that lead to low volunteer consensus. Furthermore, the machine’s attention highlights features corresponding to specific classes. This provides meaningful image-level feature-class relationships, which is useful in our analysis for identifying vortex-specific features to better understand vortex evolution mechanisms. Finally, we discuss the applicability of this method to other citizen science projects.
more »
« less
- Award ID(s):
- 2006894
- PAR ID:
- 10569023
- Editor(s):
- Fortson, Lucy; Crowston, Kevin; Kloetzer, Laure; Ponti, Marisa
- Publisher / Repository:
- Ubiquity Press
- Date Published:
- Journal Name:
- Citizen Science: Theory and Practice
- Volume:
- 9
- Issue:
- 1
- ISSN:
- 2057-4991
- Page Range / eLocation ID:
- 41
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Data work is often completed by crowdworkers, who are routinely dehumanized, disempowered, and sidelined. We turn to citizen science to reimagine data work, highlighting collaborative relationships between citizen science project managers and volunteers. Though citizen science and traditional crowd work entail similar forms of data work, such as classifying or transcribing large data sets, citizen science relies on volunteer contributions rather than paid data work. We detail the work citizen science project managers did to shape volunteer experiences: aligning science goals, minimizing barriers to participation, engaging communities, communicating with volunteers, providing training and education, rewarding contributions, and reflecting on volunteer work. These management strategies created opportunities for meaningful work by cultivating intrinsic motivation and fostering collaborative work relationships but ultimately limited participation to specific data-related tasks. We recommend management tactics and task design strategies for creating meaningful work for invisible collar workers, an understudied class of labor in CSCW.more » « less
-
Building community with rural and underrepresented groups has been a challenge in the field of citizen science. At the University of Alaska Fairbanks, a team of scientists, educators, Extension professionals, and evaluators have joined efforts to take on this challenge across Alaska. The goals for Arctic Harvest-Public Participation in Scientific Research are to: 1) investigate how shifts in environmental conditions affect the fate of subsistence berries and timing of berry loss from plants in fall and winter across Alaska; and 2) improve the participation in and effectiveness of citizen science across diverse audiences, particularly at high latitudes where a high proportion of communities have populations underrepresented in STEM. We present the assets that collaboration across a land grant university brought to the table, and the Winterberry Citizen Science program design elements we have developed to engage our 1080+ volunteer berry citizen scientists ages three through elder across urban and rural, Indigenous and non-Indigenous, and formal and informal learning settings. Our interdisciplinary team developed and implemented a program that provides in-person or online support for berry monitoring and data collection, and accommodates different age levels and settings. We also developed and tested an innovative program model that weaves storytelling throughout the citizen science learning cycle, from berries stories from the larger community, to stories of the citizen science process, to stories developed from berry data being collected and applied to future scenarios in a changing climate. The variety of program modifications we created have been highly effective helping reach a variety of settings and age levels. In both informal and formal learning environments in our first two years of the program we have had 568 pre-K and elementary-aged (age 3-12), 424 secondary-aged (age 12-18) youth participants and 107 adults (ages 18+), with 44% of participants coming from groups underrepresented in STEM, and 100% of groups completing berry monitoring throughout the fall. These results highlight the importance of designing the citizen science program with cultural relevance, program delivery options, and relationships between participants and scientists, while remaining committed to making a substantial scientific contribution.more » « less
-
Accurate detection of skin lesions through computer-aided diagnosis has emerged as a critical advancement in dermatology, addressing the inefficiencies and errors inherent in manual visual analysis. Despite the promise of automated diagnostic approaches, challenges such as image size variability, hair artifacts, color inconsistencies, ruler markers, low contrast, lesion dimension differences, and gel bubbles must be overcome. Researchers have made significant strides in binary classification problems, particularly in distinguishing melanocytic lesions from normal skin conditions. Leveraging the “MNIST HAM10000” dataset from the International Skin Image Collaboration, this study integrates Scale-Invariant Feature Transform (SIFT) features with a custom convolutional neural network model called LesionNet. The experimental results reveal the model's robustness, achieving an impressive accuracy of 99.28%. This high accuracy underscores the effectiveness of combining feature extraction techniques with advanced neural network models in enhancing the precision of skin lesion detection.more » « less
-
Abstract Citizen science is personal. Participation is contingent on the citizens’ connection to a topic or to interpersonal relationships meaningful to them. But from the peer-reviewed literature, scientists appear to have an acquisitive data-centered relationship with citizens. This has spurred ethical and pragmatic criticisms of extractive relationships with citizen scientists. We suggest five practical steps to shift citizen-science research from extractive to relational, reorienting the research process and providing reciprocal benefits to researchers and citizen scientists. By virtue of their interests and experience within their local environments, citizen scientists have expertise that, if engaged, can improve research methods and product design decisions. To boost the value of scientific outputs to society and participants, citizen-science research teams should rethink how they engage and value volunteers.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.5334/cstp.731
