More Like this

1. An indicator of technical emergence

   Carley, S.F.; Newman, N.C.; Porter, A.L.; Garner, J. (April 2018, Scientometrics)

   Developing useful intelligence on scientific and technological emergence challenges those who would manage R&D portfolios, assess research programs, or manage innovation. Recently, the U.S. Intelligence Advanced Research Projects Activity Foresight and Understanding from Scientific Exposition Program has explored means to detect emergence via text analyses. We have been involved in positing conceptual bases for emergence, framing candidate indicators, and devising implementations. We now present a software script to generate a family of Emergence Indicators for a topic of interest. This paper offers some background, then discusses the development of this script through iterative rounds of testing, and then offers example findings. Results point to promising and actionable intelligence for R&D decision-makers. 

   more » « less
2. A Multi-match Approach to the Author Uncertainty Problem

   Carley, S.; Porter, A.L.; Youtie, J (January 2019, Journal of data and information science)

   Developing useful intelligence on scientific and technological emergence challenges those who would manage R&D portfolios, assess research programs, or manage innovation. Recently, the U.S. Intelligence Advanced Research Projects Activity Foresight and Understanding from Scientific Exposition Program has explored means to detect emergence via text analyses. We have been involved in positing conceptual bases for emergence, framing candidate indicators, and devising implementations. We now present a software script to generate a family of Emergence Indicators for a topic of interest. This paper offers some background, then discusses the development of this script through iterative rounds of testing, and then offers example findings. Results point to promising and actionable intelligence for R&D decision-makers. 

   more » « less
3. Extending a pretrained language model (BERT) using an ontological perspective to classify students’ scientific expertise level from written responses

   https://doi.org/10.1186/s43031-025-00149-5  

   Wang, Heqiao; Haudek, Kevin C; Manzanares, Amanda D; Romulo, Chelsie L; Royse, Emily A; Azzarello, Caterina B (December 2025, Disciplinary and Interdisciplinary Science Education Research)

   The complex and interdisciplinary nature of scientific concepts presents formidable challenges for students in developing their knowledge-in-use skills. The utilization of computerized analysis for evaluating students’ contextualized constructed responses offers a potential avenue for educators to develop personalized and scalable interventions, thus supporting the current teaching and learning of science. While prior research in artificial intelligence has demonstrated the effectiveness of algorithms, including Bidirectional Encoder Representations from Transformers (BERT), in tasks like automated classifications of constructed responses, these efforts have predominantly leaned towards text-level features, often overlooking the exploration of conceptual ideas embedded in students’ responses from a cognitive perspective. Despite BERT’s performance in downstream tasks, challenges may arise in domain-specific tasks, particularly in establishing knowledge connections between specialized and open domains. These challenges become pronounced in small-scale and imbalanced educational datasets, where the available information for fine-tuning is frequently inadequate to capture task-specific nuances and contextual details. The primary objective of the present study is to investigate the effectiveness of a pretrained language model, when integrated with an ontological framework aligned with a contextualized science assessment, in classifying students’ expertise levels in scientific explanation. Our findings indicate that while pretrained language models, such as BERT, contribute to enhanced performance in language-related tasks within educational contexts, the incorporation of identifying domain-specific terms and extracting and substituting with their associated sibling terms in sentences through ontology-based systems can significantly improve classification model performance. Further, we qualitatively examined student responses and found that, as expected, the ontology framework identified and substituted key domain-specific terms in student responses that led to more accurate predictive scores. The study explores the practical implementation of ontology in assessment evaluation to facilitate formative assessment and formulate instructional strategies. 

   more » « less
4. Empowering future scientists: mentors employ various strategies to engage students in professional science disciplinary literacy practices

   https://doi.org/10.1186/s43031-025-00124-0  

   Minocha, Trisha; Bhagatwala, Tanya; Mirzoyan, Gwendolyn; McDowell, Gary; Fankhauser, Sarah C (December 2025, Disciplinary and Interdisciplinary Science Education Research)

   Peer-review and publication are important parts of the scientific enterprise, and research has shown that engaging students in such scholarly practices helps build their sense of belonging and scientific identity. Yet, these disciplinary literacy skills and professional practices are often part of the hidden curriculum of science research, thus excluding students and others from fully understanding ways in which scientific knowledge is constructed, refined, and disseminated even though students are participating in such activities. Secondary students are increasingly involved in scientific research projects that include authentic disciplinary literacy components such as research proposals, posters, videos, and scientific research papers. More and more, students are also engaging in professional practice of publishing their scientific research papers through dedicated secondary science journals. How teachers and other mentors support the development of professional disciplinary literacies in students is critical to understand as part of supporting more student participation in research. To this end, we used a mixed-methods study of interviews and surveys to examine the experience and conceptions of the mentors (teachers and professional scientists) who guided pre-college students through the writing and publication of their scientific research projects. Analyzing our data from a lens of cognitive apprenticeship, we find that mentors encourage independence by primarily employing the method of “exploration”. We also find that mentors have divergent views on the value of publication within science, versus for student scientists specifically. Our findings suggest that mentors could work to explicitly reveal their own thinking within science writing to provide more sequenced support for student scientists. 

   more » « less
5. Online citizen science in higher education courses

   https://doi.org/10.1002/ecs2.70352  

   Cotey, Stacy R; Dunbar‐Wallis, Amy K; Golumbic, Yaela N; Mehrotra, Pankaj; Beamer, David A; Vance‐Chalcraft, Heather D (July 2025, Ecosphere)

   Climate change and biodiversity loss require us to engage the next generation of scientists in addressing global ecological issues. Introducing undergraduate students to citizen science allows them to learn scientific processes and content while contributing to real‐world applications. We conducted a systematic review of literature to (1) identify what types of undergraduate courses and institutions use citizen science, (2) list the projects and platforms that have been implemented in online courses in undergraduate education, (3) examine how students participated in the projects through online courses, and (4) summarize learning objectives and reported benefits of student participation. In all, 44 studies about the use of citizen science in undergraduate online courses were found in 25 papers in the published literature. The most common projects consisted of classification of species or natural history (e.g., iNaturalist), which could be done mainly online but with data collection completed at a location available to the student. Citizen science projects were incorporated into multiple course formats (e.g., lecture, lab) and class sizes, and students were most frequently asked to collect and submit data. The most frequently reported learning outcomes included increased student interest/engagement, improved appreciation for the relevance of science to the “real world,” and practice using the scientific process, but rigorous assessment data were lacking in papers. The use of citizen science in online courses and institutions appears to be increasing, and we encourage faculty using these approaches with students to publish on their efforts, providing details about their implementation, assessment, and course context. 

   more » « less