Search for: All records

The complex and interdisciplinary nature of scientific concepts presents formidable challenges for students in developing their knowledge-in-use skills. The utilization of computerized analysis for evaluating students’ contextualized constructed responses offers a potential avenue for educators to develop personalized and scalable interventions, thus supporting the current teaching and learning of science. While prior research in artificial intelligence has demonstrated the effectiveness of algorithms, including Bidirectional Encoder Representations from Transformers (BERT), in tasks like automated classifications of constructed responses, these efforts have predominantly leaned towards text-level features, often overlooking the exploration of conceptual ideas embedded in students’ responses from a cognitive perspective. Despite BERT’s performance in downstream tasks, challenges may arise in domain-specific tasks, particularly in establishing knowledge connections between specialized and open domains. These challenges become pronounced in small-scale and imbalanced educational datasets, where the available information for fine-tuning is frequently inadequate to capture task-specific nuances and contextual details. The primary objective of the present study is to investigate the effectiveness of a pretrained language model, when integrated with an ontological framework aligned with a contextualized science assessment, in classifying students’ expertise levels in scientific explanation. Our findings indicate that while pretrained language models, such as BERT, contribute to enhanced performance in language-related tasks within educational contexts, the incorporation of identifying domain-specific terms and extracting and substituting with their associated sibling terms in sentences through ontology-based systems can significantly improve classification model performance. Further, we qualitatively examined student responses and found that, as expected, the ontology framework identified and substituted key domain-specific terms in student responses that led to more accurate predictive scores. The study explores the practical implementation of ontology in assessment evaluation to facilitate formative assessment and formulate instructional strategies. 

Abstract Science identity, or one's sense of recognition and competence as a scientist, is an invaluable tool for predicting student persistence and success, but is understudied among undergraduates completing preparatory work for later studies in medicine, nursing, and allied health (“pre‐health career students”). In the United States, pre‐health career students make up approximately half of all biology students and, as professionals, play important roles in caring for an aging, increasingly diverse population, managing the ongoing effects of a pandemic, and navigating socio‐political shifts in public attitudes toward science and evidence‐based medicine. Pre‐health career students are also often members of groups marginalized and minoritized in STEM education, and generally complete their degrees in community college settings, which are chronically under‐resourced and understudied. Understanding these students' science identities is thus a matter of social justice and increasingly important to public health in the United States. We examined science identity and engagement among community college biology students using two scales established and validated for use with STEM students attending four‐year institutions. Exploratory and confirmatory factor analysis were used on two sub‐samples drawn from the pool of 846 participants to confirm that the factor structures functioned as planned among the new population. Science identity values were then compared between pre‐health career students (pre‐nursing and pre‐allied health) and other groups. Pre‐health career students generally reported interest and performance/competence on par with their traditional STEM, pre‐med, and pre‐dentistry peers, challenging popular assumptions about these students' interests and abilities. However, they also reported significantly lower recognition than traditional STEM and pre‐med/dentistry students. The implications for public health, researchers, and faculty are discussed. 

Interdisciplinary environmental and sustainability (IES) programs are different from other fields because they focus on a complex integration of humanities, social, and natural sciences concepts centered on the interactions of coupled human and natural systems. The interdisciplinary nature of IES programs does not lend itself to traditional discipline-specific concept inventory frameworks for critically evaluating preconceptions and learning. We discuss the results of the first phase of a research project to develop a next generation concept inventory for evaluating interdisciplinary concepts important for introductory IES courses. Using the Food-Energy-Water (FEW) Nexus (the intersections/interdependencies of food, energy, and water sectors) as our focus, we conducted a content analysis of eight representative college-level introductory environmental course syllabi and course materials (e.g., textbooks, journal articles, print media) to identify common interdisciplinary FEW Nexus concepts taught in introductory IES courses. Results demonstrate that all IES introductory course materials reference the FEW Nexus. Food, energy, and/or water resources as individual elements of the FEW Nexus are frequently described, but connections between these resource systems are included less often. Biology, energy systems, waste and pollution in the natural environment, agriculture, earth sciences and geology, climate change, behavioral social sciences, and economics concepts are most associated with FEW concepts, hinting at commonalities across IES topics that anchor systems thinking. Despite differences in IES programs, there appears to be some alignment between core concepts being taught at the FEW Nexus in introductory courses.