skip to main content


Title: Covariational reasoning and item context affect language in undergraduate mass balance written explanations
This article builds on the work of Scott et al. (Scott EE, Cerchiara J, McFarland JL, Wenderoth MP, Doherty JH. J Res Sci Teach 1: 37, 2023) and Shiroda et al. (Shiroda M, Fleming MP, Haudek KC. Front Educ 8: 989836, 2023) to quantitatively examine student language in written explanations of mass balance across six contexts using constructed response assessments. These results present an evaluation of student mass balance language and provide researchers and practitioners with tools to assist students in constructing scientific mass balance reasoning explanations.  more » « less
Award ID(s):
1660643
PAR ID:
10471768
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Advances in Physiology Education
Date Published:
Journal Name:
Advances in Physiology Education
Volume:
47
Issue:
4
ISSN:
1043-4046
Page Range / eLocation ID:
762 to 775
Subject(s) / Keyword(s):
constructed response context covariational reasoning mass balance student language
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Modèle Atmosphérique Régional (MAR) is a regional climate model that is fully coupled to a one-dimensional surface-atmosphere energy and mass transfer scheme, SISVAT (Soil Ice Snow Vegetation Atmosphere Transfer) (Fettweis et al., 2005, 2020; Lefebre et al., 2005). SISVAT employs a multilayered snowpack model, CROCUS, that simulates meltwater production, percolation, and refreeze (Brun et al., 1989), while also accounting for changes in albedo due to snow metamorphism (Brun et al., 1992). MAR has been extensively verified over the Greenland Ice Sheet and is therefore particularly well suited for analyses of Greenland ice sheet surface mass balance (Fettweis et al., 2011; Fettweis et al., 2020; Lefebre et al. 2005; Mattingly et al. 2020). Brun, E., Martin, E., Simon, V., Gendre, C., and Coléou, C. (1989). An energy and mass model of snow cover suitable for operational avalanche forecasting. Journal of Glaciology, 35, 333. https://doi.org/10.1017/S0022143000009254 Brun, E., David, P., Sudul, M., and Brunot, G. (1992). A numerical model to simulate snow-cover stratigraphy for operational avalanche forecasting. Journal of Glaciology, 38(128), 13–22. https://doi.org/10.3189/S0022143000009552 Fettweis, X., Gallée, H., Lefebre, F., and van Ypersele, J.-P. (2005). Greenland surface mass balance simulated by a regional climate model and comparison with satellite-derived data in 1990–1991. Climate Dynamics, 24(6), 623–640. https://doi.org/10.1007/s00382-005-0010-y Fettweis, X., Tedesco, M., van den Broeke, M., and Ettema, J. (2011). Melting trends over the Greenland ice sheet (1958–2009) from spaceborne microwave data and regional climate models. The Cryosphere, 5(2), 359–375. https://doi.org/10.5194/tc-5-359-2011 Fettweis, X., Hofer, S., Krebs-Kanzow, U., Amory, C., Aoki, T., Berends, C. J., et al. (2020). GrSMBMIP: intercomparison of the modelled 1980–2012 surface mass balance over the Greenland Ice Sheet. The Cryosphere, 14(11), 3935–3958. https://doi.org/10.5194/tc-14-3935-2020 Lefebre, F., Fettweis, X., Gallée, H., Van Ypersele, J.-P., Marbaix, P., Greuell, W., and Calanca, P. (2005). Evaluation of a high-resolution regional climate simulation over Greenland. Climate Dynamics, 25(1), 99–116. https://doi.org/10.1007/s00382-005-0005-8 Mattingly, K. S., Mote, T. L., Fettweis, X., van As, D., Van Tricht, K., Lhermitte, S., et al. (2020). Strong summer atmospheric rivers trigger Greenland ice sheet melt through spatially varying surface energy balance and cloud regimes. Journal of Climate, 33(16), 6809–6832. https://doi.org/10.1175/JCLI-D-19-0835.1 
    more » « less
  2. We address the question of consistency strength of certain filters and ultrafilters which fail to satisfy the Galvin property. We answer questions [Benhamou and Gitik, Ann. Pure Appl. Logic 173 (2022) 103107; Questions 7.8, 7.9], [Benhamou et al., J. Lond. Math. Soc. 108(1) (2023) 190–237; Question 5] and improve theorem [Benhamou et al., J. Lond. Math. Soc. 108(1) (2023) 190–237; Theorem 2.3].

     
    more » « less
  3. Hallucinations in large language models (LLMs), where they generate fluent but factually incorrect outputs, pose challenges for applications requiring strict truthfulness. This work proposes a multi-faceted approach to detect such hallucinations across various language tasks. We leverage automatic data annotation using a proprietary LLM, fine-tuning of the Mistral-7B-instruct-v0.2 model on annotated and benchmark data, role-based and rationale-based prompting strategies, and an ensemble method combining different model outputs through majority voting. This comprehensive framework aims to improve the robustness and reliability of hallucination detection for LLM generations. Code and data1 1 Introduction The modern natural language generation (NLG) (OpenAI et al., 2023; Touvron et al., 2023) landscape faces two interconnected challenges: firstly, current neural models have a tendency to produce f luent yet inaccurate outputs, and secondly, our evaluation metrics are better suited for assessing f luency rather than correctness(Bang et al., 2023; Guerreiro et al., 2023). This phenomenon, known as "hallucination," (Ji et al., 2023) where neural networks generate plausible-sounding but factually incorrect outputs, is a significant hurdle, especially for NLG applications that require strict adherence to correctness. For instance, in machine translation(Lee et al., 2019), producing a fluent translation that deviates from the source text’s meaning renders the entire translation pipeline unreliable. This issue may arise as LLMs are trained on vast amounts of data from the internet, which can contain inaccuracies, biases, and false information. Also, it may arise due improper representations learned during training even if good quality data is 1https://github.com/souvikdgp16/shroom_compos_mentis used. As a result, LLMs can sometimes hallucinate or fabricate details, especially when prompted to discuss topics outside their training data or make inferences beyond their capabilities. Hallucination detection (Liu et al., 2022), also known as factual verification or truthfulness evaluation, identifies and mitigates these hallucinations in the outputs of LLMs. This is an active area of research and development, as it is crucial for ensuring the reliability and trustworthiness of LLMgenerated content, particularly in high-stakes domains such as healthcare, finance, and legal applications. In this task, the primary focus will be to classify whether a generation is hallucinated. This work proposes a multi-faceted approach to detecting hallucinations in large language models. 
    more » « less
  4. This work-in-progress paper describes a collaborative effort between engineering education and machine learning researchers to automate analysis of written responses to conceptually challenging questions in mechanics. These qualitative questions are often used in large STEM classes to support active learning pedagogies; they require minimum calculations and focus on the application of underlying physical phenomena to various situations. Active learning pedagogies using this type of questions has been demonstrated to increase student achievement (Freeman et al., 2014; Hake, 1998) and engagement (Deslauriers, et al., 2011) of all students (Haak et al., 2011). To emphasize reasoning and sense-making, we use the Concept Warehouse (Koretsky et al., 2014), an audience response system where students provide written justifications to concept questions. Written justifications better prepare students for discussions with peers and in the whole class and can also improve students’ answer choices (Koretsky et al., 2016a, 2016b). In addition to their use as a tool to foster learning, written explanations can also provide valuable information to concurrently assess that learning (Koretsky and Magana, 2019). However, in practice, there has been limited deployment of written justifications with concept questions, in part, because they provide a daunting amount of information for instructors to process and for researchers to analyze. In this study, we describe the initial evaluation of large pre-trained generative sequence-to-sequence language models (Raffel et al., 2019; Brown et al., 2020) to automate the laborious coding process of student written responses. Adaptation of machine learning algorithms in this context is challenging since each question targets specific concepts which elicit their own unique reasoning processes. This exploratory project seeks to utilize responses collected through the Concept Warehouse to identify viable strategies for adapting machine learning to support instructors and researchers in identifying salient aspects of student thinking and understanding with these conceptually challenging questions. 
    more » « less
  5. Introduction This archive includes a tab-delimited (tsv) and comma-delimited (csv) version of the Discover Life bee species guide and world checklist (Hymenoptera: Apoidea: Anthophila). Discover Life is an important resource for bee species names and this update is from Draft-55, November 2020. Data were accessed and transformed into a tsv file in August 2023 using Global Biotic Interactions (GloBI) nomer software. GloBI now incorporates the Discover Life bee species guide and world checklist in its functionality for searching for bee interactions. Update! New Dataset also includes Subgenera Names A new, tab-delimited version of the Discover Life taxonomy as derived from Dorey et. al, 2023 can be found via Zenodo at https://doi.org/10.5281/zenodo.10463762. This version of the Discover Life world species guide and checklist includes subgeneric names. Citation Please cite the original source for this data as: Ascher, J. S. and J. Pickering. 2022.Discover Life bee species guide and world checklist (Hymenoptera: Apoidea: Anthophila).http://www.discoverlife.org/mp/20q?guide=Apoidea_species Draft-56, 21 August, 2022 nomer nomer is a command-line application for working with taxonomic resources offline. nomer incorporates many of the present taxonomic catalogs (e.g., catalog of life, ITIS, EOL, NCBI) and provides simple tools for comparing between resources or resolving taxonomic names based on one or more taxonomic name catalogs. Discover Life is in nomer version 0.5.1 and this full dataset can be recreated by installing nomer from https://github.com/globalbioticinteractions/nomer and running $ nomer list discoverlife > discoverlife.tsv Data Columns Discover Life provides a world name checklist and includes other names (synonyms and homonyms) that refer to the same species. In the tsv file, the provided name is both the accepted, or checklist name, or "other name." All names will be listed as a providedName. Below is an example subset of the transformed version of the data. providedExternalId= link to name on Discover Life providedName=an accepted or "other name" in the Discover Life bee checklist. "Other names" can be synonyms or homonyms. providedAuthorship=authorship for the providedName providedRank=rank of the providedName providedPath=higher taxonomy of the providedName. This will be the same as the accepted name or resolvedName relationName=relationship between the "other name" and the bee name in the Discover Life checklist. It may include itself resolvedExternalID=an accepted name in the Discover Life bee checklist resolvedExternalId=link to name on Discover Life resolvedAuthorship=authorship of the accepted, or checklist name resolvedRank=rank of the accepted, or checklist name resolvedPath=higher taxonomy of the accepted, or checklist name Changes No major changes to format in this version. References Jorrit Poelen, & José Augusto Salim. (2022). globalbioticinteractions/nomer: (0.2.11). Zenodo. https://doi.org/10.5281/zenodo.6128011 Poelen JH, Simons JD and Mungall CH. (2014). Global Biotic Interactions: An open infrastructure to share and analyze species-interaction datasets. Ecological Informatics. https://doi.org/10.1016/j.ecoinf.2014.08.005. Seltmann KC, Allen J, Brown BV, Carper A, Engel MS, Franz N, Gilbert E, Grinter C, Gonzalez VH, Horsley P, Lee S, Maier C, Miko I, Morris P, Oboyski P, Pierce NE, Poelen J, Scott VL, Smith M, Talamas EJ, Tsutsui ND, Tucker E (2021) Announcing Big-Bee: An initiative to promote understanding of bees through image and trait digitization. Biodiversity Information Science and Standards 5: e74037. https://doi.org/10.3897/biss.5.74037 Dorey, J.B., Fischer, E.E., Chesshire, P.R. et al. A globally synthesised and flagged bee occurrence dataset and cleaning workflow. Sci Data 10, 747 (2023). https://doi.org/10.1038/s41597-023-02626-w 
    more » « less