skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Against method: Exploding the boundary between qualitative and quantitative studies of science
Quantitative and qualitative studies of science have historically played radically different roles with opposing epistemological commitments. Using large-scale text analysis, we see that qualitative studies generate and value new theory, especially regarding the complex social and political contexts of scientific action, while quantitative approaches confirm existing theory and evaluate the performance of scientific institutions. Large-scale digital data and emerging computational methods could allow us to refigure these positions, turning qualitative artifacts into quantitative patterns into qualitative insights across many scales, heralding a new era of theory development, engagement, and relevance for scientists, policy-makers, and society.  more » « less
Award ID(s):
1800956
PAR ID:
10298865
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Quantitative Science Studies
Volume:
1
Issue:
3
ISSN:
2641-3337
Page Range / eLocation ID:
930 to 944
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Social‐ecological systems like fisheries provide food, livelihoods and recreation. However, lack of data and its integration into governance hinders their conservation and management. Stakeholders possess site‐specific knowledge crucial for confronting these challenges. There is increasing recognition that Indigenous and local knowledge (ILK) is valuable, but structural differences between ILK and quantitative archetypes have stalled the assimilation of ILK into fisheries management, despite acknowledged bias and uncertainty in scientific methods. Conducting a systematic review of fisheries‐associated ILK research (n = 397 articles), we examined how ILK is accessed, applied, distributed across space and species, and has evolved. We show that ILK has generated qualitative, semi‐quantitative and quantitative information for diverse taxa across 98 countries. Fisheries‐associated ILK research mostly targets small‐scale and artisanal fishers (70% of studies) and typically uses semi‐structured interviews (60%). We revealed large variability in sample size (n = 4–7638), predicted by the approach employed and the data generated (i.e. qualitative studies target smaller groups). Using thematic categorisation, we show that scientists are still exploring techniques, or ‘validating’ ILK through comparisons with quantitative scientific data (20%), and recording qualitative information of what fishers understand (40%). A few researchers are applying quantitative social science methods to derive trends in abundance, catch and effort. Such approaches facilitate recognition of local insight in fisheries management but fall short of accepting ILK as a valid complementary way of knowing about fisheries systems. This synthesis reveals that development and increased opportunities are needed to bridge ILK and quantitative scientific data. 
    more » « less
  2. Language-based text provide valuable insights into people’s lived experiences. While traditional qualitative analysis is used to capture these nuances, new paradigms are needed to scale qualitative research effectively. Artificial intelligence presents an unprecedented opportunity to expand the sale of analysis for obtaining such nuances. The study tests the application of GPT-4—a large language modeling—in qualitative data analysis using an existing set of text data derived from 60 qualitative interviews. Specifically, the study provides a practical guide for social and behavioral researchers, illustrating core elements and key processes, demonstrating its reliability by comparing GPT-generated codes with researchers’ codes, and evaluating its capacity for theory-driven qualitative analysis. The study followed a three-step approach: (1) prompt engineering, (2) reliability assessment by comparison of GPT-generated codes with researchers’ codes, and (3) evaluation of theory-driven thematic analysis on psychological constructs. The study underscores the utility of GPT’s capabilities in coding and analyzing text data with established qualitative methods while highlighting the need for qualitative expertise to guide GPT applications. Recommendations for further exploration are also discussed. 
    more » « less
  3. Tzovla, Eirini (Ed.)
    IntroductionProfessional development is essential for science teachers so they can stay current with educational trends, address diverse student needs, and integrate emerging scientific phenomena into their teaching practices. This study explores factors that motivate science teachers to self-select into multi-year-long professional development programs. MethodsThe study’s sample included 14 exemplary science teachers enrolled in multi-year programs and 58 teachers in one-year professional development programs. Quantitative data were collected from all participants using the Basic Psychological Need Satisfaction at Work Scale and qualitative data were gathered from the multi-year program participants through semi-structured interviews. ResultsGrounded in self-determination theory, the quantitative data revealed that exemplary science teachers reported low job satisfaction and significantly lower scores in therelatednessconstruct, indicating they felt undervalued in their workplaces compared to other participants. Qualitative findings supported these results, as teachers expressed their motivations for participating in multi-year professional development programs driven by a desire for personal growth, continuous learning, and a collaborative learning environment, as well as a lack of rigorous in-school professional development options. RecommendationsThe results inform recommendations for professional development organizers and educational leaders to prioritize offering sustained long-term professional development programs tailored for exemplary teachers so they can feel supported and valued. ConclusionThe study highlights the importance of tailoring professional development programs to the diverse needs of exemplary science teachers such that they feel they belong in the field of science education. 
    more » « less
  4. Assessing the impact of regional or statewide interventions in primary and secondary school (K-12) computer science (CS) education is difficult for a variety of reasons. Qualitative survey data provide only a limited view of impacts, but quantitative data can be notoriously difficult to acquire at scale from large numbers of classrooms, schools, or local educational authorities. In this paper, we use several publicly available data sources to glean insights into public high school CS enrollments across an entire U.S. state. Course enrollments with NCES course codes and local descriptors, school-level demographic data, and school geographic attendance boundaries can be combined to highlight where CS offerings persist and thrive, how CS enrollments change over time, and the ultimate quantitative impact of a statewide intervention. We propose a more appropriate level of data aggregation for these types of quantitative studies than has been undertaken in previous work while demonstrating the importance of a contextual aggregation process. The results of our disparate impact analysis for the first time quantify the impact of a statewide Exploring Computer Science (ECS) program rollout on economic groups across the region. Our blueprint for this analysis can serve as a template to guide and assess large-scale K-12 CS interventions wherever detailed project evaluation methods cannot scale to encompass the entire study area, especially in cases where attribute heterogeneity is a significant issue. 
    more » « less
  5. null (Ed.)
    In many academic fields, the number of papers published each year has increased significantly over time. Policy measures aim to increase the quantity of scientists, research funding, and scientific output, which is measured by the number of papers produced. These quantitative metrics determine the career trajectories of scholars and evaluations of academic departments, institutions, and nations. Whether and how these increases in the numbers of scientists and papers translate into advances in knowledge is unclear, however. Here, we first lay out a theoretical argument for why too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea. Then, we show data supporting the predictions of this theory. When the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon. Policy measures shifting how scientific work is produced, disseminated, consumed, and rewarded may be called for to push fields into new, more fertile areas of study. 
    more » « less