skip to main content


Title: Comparing study features is easy but identifying next steps is hard: Evaluating critical thinking through the Biology Lab Inventory of Critical Thinking in Ecology
Abstract

Critical thinking, which can be defined as the evidence‐based ways in which people decide what to trust and what to do, is an important competency included in many undergraduate science, technology, engineering, and mathematics (STEM) courses. To help instructors effectively measure critical thinking, we developed the Biology Lab Inventory of Critical Thinking in Ecology (Eco‐BLIC), a freely available, closed‐response assessment of undergraduate students' critical thinking in ecology. The Eco‐BLIC includes ecology‐based experimental scenarios followed by questions that measure how students decide on what to trust and what to do next. Here, we present the development of the Eco‐BLIC using tests of validity and reliability. Using student responses to questions and think‐aloud interviews, we demonstrate the effectiveness of the Eco‐BLIC at measuring students' critical thinking skills. We find that while students generally think like experts while evaluating what to trust, students' responses are less expert‐like when deciding on what to do next.

 
more » « less
Award ID(s):
1909602
NSF-PAR ID:
10412855
Author(s) / Creator(s):
 ;  ;  ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Ecology and Evolution
Volume:
13
Issue:
5
ISSN:
2045-7758
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Pamucar, Dragan (Ed.)
    Critical thinking is the process by which people make decisions about what to trust and what to do. Many undergraduate courses, such as those in biology and physics, include critical thinking as an important learning goal. Assessing critical thinking, however, is non-trivial, with mixed recommendations for how to assess critical thinking as part of instruction. Here we evaluate the efficacy of assessment questions to probe students’ critical thinking skills in the context of biology and physics. We use two research-based standardized critical thinking instruments known as the Biology Lab Inventory of Critical Thinking in Ecology (Eco-BLIC) and Physics Lab Inventory of Critical Thinking (PLIC). These instruments provide experimental scenarios and pose questions asking students to evaluate what to trust and what to do regarding the quality of experimental designs and data. Using more than 3000 student responses from over 20 institutions, we sought to understand what features of the assessment questions elicit student critical thinking. Specifically, we investigated (a) how students critically evaluate aspects of research studies in biology and physics when they are individually evaluating one study at a time versus comparing and contrasting two and (b) whether individual evaluation questions are needed to encourage students to engage in critical thinking when comparing and contrasting. We found that students are more critical when making comparisons between two studies than when evaluating each study individually. Also, compare-and-contrast questions are sufficient for eliciting critical thinking, with students providing similar answers regardless of if the individual evaluation questions are included. This research offers new insight on the types of assessment questions that elicit critical thinking at the introductory undergraduate level; specifically, we recommend instructors incorporate more compare-and-contrast questions related to experimental design in their courses and assessments. 
    more » « less
  2. Abstract

    Contemporary science is a field that is becoming increasingly computational. Today’s scientists not only leverage computational tools to conduct their investigations, they often must contribute to the design of the computational tools for their specific research. From a science education perspective, for students to learn authentic science practices, students must learn to use the tools of the trade. This necessity in science education has shaped recent K–12 science standards including the Next Generation Science Standards, which explicitly mention the use of computational tools and simulations. These standards, in particular, have gone further and mandated thatcomputational thinkingbe taught and leveraged as a practice of science. While computational thinking is not a new term, its inclusion in K–12 science standards has led to confusion about what the term means in the context of science learning and to questions about how to differentiate computational thinking from other commonly taught cognitive skills in science like problem-solving, mathematical reasoning, and critical thinking. In this paper, we propose a definition ofcomputational thinking for science(CT-S) and a framework for its operationalization in K–12 science education. We situate our definition and framework in Activity Theory, from the learning sciences, in order to position computational thinking as an input to and outcome of science learning that is mediated by computational tools.

     
    more » « less
  3. Engineers are called to play an important role in addressing the complex problems of our global society, such as climate change and global health care. In order to adequately address these complex problems, engineers must be able to identify and incorporate into their decision making relevant aspects of systems in which their work is contextualized, a skill often referred to as systems thinking. However, within engineering, research on systems thinking tends to emphasize the ability to recognize potentially relevant constituent elements and parts of an engineering problem, rather than how these constituent elements and parts are embedded in broader economic, sociocultural, and temporal contexts and how all of these must inform decision making about problems and solutions. Additionally, some elements of systems thinking, such as an awareness of a particular sociocultural context or the coordination of work among members of a cross-disciplinary team, are not always recognized as core engineering skills, which alienates those whose strengths and passions are related to, for example, engineering systems that consider and impact social change. Studies show that women and minorities, groups underrepresented within engineering, are drawn to engineering in part for its potential to address important social issues. Emphasizing the importance of systems thinking and developing a more comprehensive definition of systems thinking that includes both constituent parts and contextual elements of a system will help students recognize the relevance and value of these other elements of engineering work and support full participation in engineering by a diverse group of students. We provide an overview of our study, in which we are examining systems thinking across a range of expertise to develop a scenario-based assessment tool that educators and researchers can use to evaluate engineering students’ systems thinking competence. Consistent with the aforementioned need to define and study systems thinking in a comprehensive, inclusive manner, we begin with a definition of systems thinking as a holistic approach to problem solving in which linkages and interactions of the immediate work with constituent parts, the larger sociocultural context, and potential impacts over time are identified and incorporated into decision making. In our study, we seek to address two key questions: 1) How do engineers of different levels of education and experience approach problems that require systems thinking? and 2) How do different types of life, educational, and work experiences relate to individuals’ demonstrated level of expertise in solving systems thinking problems? Our study is comprised of three phases. The first two phases include a semi-structured interview with engineering students and professionals about their experiences solving a problem requiring systems thinking and a think-aloud interview in which participants are asked to talk through how they would approach a given engineering scenario and later reflect on the experiences that inform their thinking. Data from these two phases will be used to develop a written assessment tool, which we will test by administering the written instrument to undergraduate and graduate engineering students in our third study phase. Our paper describes our study design and framing and includes preliminary findings from the first phase of our study. 
    more » « less
  4. null (Ed.)
    The promotion of global sustainability within environmental science courses requires a paradigm switch from knowledge-based teaching to teaching that stimulates higher-order cognitive skills. Non-major undergraduate science courses, such as environmental science, promote critical thinking in students in order to improve the uptake of scientific information and develop the rational decision making used to make more informed decisions. Science, engineering, technology and mathematics (STEM) courses rely extensively on visuals in lectures, readings and homework to improve knowledge. However, undergraduate students do not automatically acquire visual literacy and a lack of intervention from instructors could be limiting academic success. In this study, a visual literacy intervention was developed and tested in the face-to-face (FTF) and online sections of an undergraduate non-major Introduction to Environmental Science course. The intervention was designed to test and improve visual literacy at three levels: (1) elementary—identifying values; (2) intermediate—identifying trends; and (3) advanced—using the data to make projections or conclusions. Students demonstrated a significant difference in their ability to answer elementary and advanced visual literacy questions in both course sections in the pre-test and post-test. Students in the face-to-face course had significantly higher exam scores and higher median assessment scores compared to sections without a visual literacy intervention. The online section did not show significant improvements in visual literacy or academic success due to a lack of reinforcement of visual literacy following the initial intervention. The visual literacy intervention shows promising results in improving student academic success and should be considered for implementation in other general education STEM courses. 
    more » « less
  5. Abstract

    Simulation models are increasingly used by ecologists to study complex, ecosystem‐scale phenomena, but integrating ecosystem simulation modeling into ecology undergraduate and graduate curricula remains rare. Engaging ecology students with ecosystem simulation models may enable students to conduct hypothesis‐driven scientific inquiry while also promoting their use of systems thinking, but it remains unknown how using hands‐on modeling activities in the classroom affects student learning. Here, we developed short (3‐hr) teaching modules as part of the Macrosystems EDDIE (Environmental Data‐Driven Inquiry & Exploration) program that engage students with hands‐on ecosystem modeling in the R statistical environment. We embedded the modules into in‐person ecology courses at 17 colleges and universities and assessed student perceptions of their proficiency and confidence before and after working with models. Across all 277 undergraduate and graduate students who participated in our study, completing one Macrosystems EDDIE teaching module significantly increased students' self‐reported proficiency, confidence, and likely future use of simulation models, as well as their perceived knowledge of ecosystem simulation models. Further, students were significantly more likely to describe that an important benefit of ecosystem models was their “ease of use” after completing a module. Interestingly, students were significantly more likely to provide evidence of systems thinking in their assessment responses about the benefits of ecosystem models after completing a module, suggesting that these hands‐on ecosystem modeling activities may increase students’ awareness of how individual components interact to affect system‐level dynamics. Overall, Macrosystems EDDIE modules help students gain confidence in their ability to use ecosystem models and provide a useful method for ecology educators to introduce undergraduate and graduate students to ecosystem simulation modeling using in‐person, hybrid, or virtual modes of instruction.

     
    more » « less