An official website of the United States government
The emphasis on conceptual learning and the development of adaptive instructional design are both emerging areas in science and engineering education. Instructors are writing their own conceptual questions to promote active learning during class and utilizing pools of these questions in assessments. For adaptive assessment strategies, these questions need to be rated based on difficulty level (DL). Historically DL has been determined from the performance of a suitable number of students. The research study reported here investigates whether instructors can save time by predicting DL of newly made conceptual questions without the need for student data. In this paper, we report on the development of one component in an adaptive learning module for materials science – specifically on the topic of crystallography. The summative assessment element consists of five DL scales and 15 conceptual questions This adaptive assessment directs students based on their previous performances and the DL of the questions. Our five expert participants are faculty members who have taught the introductory Materials Science course multiple times. They provided predictions for how many students would answer each question correctly during a two-step process. First, predictions were made individually without an answer key. Second, experts had the opportunity to revise their predictions after being provided an answer key in a group discussion. We compared expert predictions with actual student performance using results from over 400 students spanning multiple courses and terms. We found no clear correlation between expert predictions of the DL and the measured DL from students. Some evidence shows that discussion during the second step made expert predictions closer to student performance. We suggest that, in determining the DL for conceptual questions, using predictions of the DL by experts who have taught the course is not a valid route. The findings in this paper can be applied to assessments in both in-person, hybrid, and online settings and is applicable to subject matter beyond materials science.
more »
« less
Explicit Instruction of Scientific Uncertainty in an Undergraduate Geoscience Field-Based Course
Abstract Understanding and communicating uncertainty is a key skill needed in the practice of science. However, there has been little research on the instruction of uncertainty in undergraduate science education. Our team designed a module within an online geoscience field course which focused on explicit instruction around uncertainty and provided students with an uncertainty rating scale to record and communicate their uncertainty with a common language. Students then explored a complex, real-world geological problem about which expert scientists had previously made competing claims through geologic maps. Provided with data, expert uncertainty ratings, and the previous claims, students made new geologic maps of their own and presented arguments about their claims in written form. We analyzed these reports along with assessments of uncertainty. Most students explicitly requested geologists’ uncertainty judgments in a post-course assessment when asked why scientists might differ in their conclusions and/or utilized the rating scale unprompted in their written arguments. Through the examination of both pre- and post-course assessments of uncertainty and students’ course-based assessments, we argue that explicit instruction around uncertainty can be introduced during undergraduate coursework and could facilitate geoscience novices developing into practicing geoscientists.
more »
« less
- Award ID(s):
- 1839705
- PAR ID:
- 10366794
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- Science & Education
- Volume:
- 31
- Issue:
- 6
- ISSN:
- 0926-7220
- Page Range / eLocation ID:
- p. 1541-1566
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Grawe, Nathan D (Ed.)Many educators and professional organizations recommend Quantitative Reasoning as the best entry-level postsecondary mathematics course for non-STEM majors. However, novice and veteran instructors who have no prior experience in teaching a QR course often express their ignorance of the content to choose for this course, the instruction to offer students, and the assessments to measure student learning. We conducted a case study to investigate the initial implementation of an entry-level university quantitative reasoning course during fall semester, 2018. The participants were the course instructor and students. We examined the instructor’s motives and actions and the students’ responses to the course. The instructor had no prior experience teaching a QR course but did have 15 years of experience teaching student-centered mathematics. Data included course artifacts, class observations, an instructor interview, and students’ written reflections. Because this was a new course—and to adapt to student needs—the instructor employed his instructional autonomy and remained flexible in designing and enacting the course content, instruction, and assessment. His instructional decision making and flexible approach helped the instructor tailor the learning activities and teaching practices to the needs and interests of the students. The students generally appreciated and benefited from this approach, enjoyed the course, and provided positive remarks about the instructors’ practices.more » « less
-
Abstract Strong spatial skills are foundational in predicting students' performance in science, technology, engineering, and mathematics education. Decades of research have considered the relationship between thinking spatially and how scientists reason and solve problems. However, few studies have examined the factors that influence improvement in students' spatial thinking during their school science curricula. The present study investigates theThinkSpacecurricula—two middle school astronomy units designed to support students' ability to apply the spatial skill of perspective‐taking (PT) while learning to explain lunar phases (3 days) and the seasons (8 days). U.S. students in 6th and 8th grades (N = 877) across four districts participated in the study, completing assessments before and after theThinkSpacecurricula, along with an additional group of students in 6th and 7th grades (N = 172) who participated as a spatial control group. Data collection included multiple‐choice content assessments, PT skill assessments, and interviews (from a sub‐sample of 96 students), before and after instruction. After participating inThinkSpacecurricula, students demonstrated improved spatial thinking within the domain of astronomy, as measured by improved written content assessments, increased application of PT during conceptual interviews, and a general measurement of PT skill. Higher initial PT skill and higher gain in PT skill predicted greater improvement in students' astronomy understanding, even when accounting for their initial content knowledge. AlthoughThinkSpacestudents in all demographic groups improved PT skill post‐instruction, 8th graders (who were in districts with lower SES), and females were predicted to have smaller gains in their PT skill than the 6th graders (who were in districts with higher SES) and male students. These findings suggest that middle school students' spatial thinking in science can be improved during their middle school science curricula, but questions remain concerning how to reduce spatial‐learning gaps that are associated with gender and possibly SES.more » « less
-
Abstract It is important to understand how students reason in K-12 integrated STEM settings to better prepare teachers to engage their students in integrated STEM tasks. To understand the reasoning that occurs in these settings, we used the lens of collective argumentation, specifically attending to the types of warrants elementary students and their teachers provided and accepted in integrated STEM contexts and how teachers supported students in providing these warrants. We watched 103 h of classroom instruction from 10 elementary school teachers and analyzed warrants that occurred in arguments in mathematics, coding, and integrated contexts to develop a typology of warrants contributed in mathematics and coding arguments. We found that these students made their warrants explicit the majority of the time, regardless of the teacher’s presence or absence. When teachers were present, they supported argumentation in various ways; however, they offered less support in integrated contexts. Additionally, we found students relied more on visual observations in coding contexts than in mathematics or integrated contexts, where they often provided warrants based on procedures required to accomplish a task. These findings have implications for improving integrated STEM instruction through engaging students in argumentation.more » « less
-
While students can learn both chemistry content and inquiry practices by participating in course-based undergraduate research experiences (CUREs), it is well documented that prior experiences shape perception. We conducted a case study to investigate students’ first experiences with a CURE in an upper-division chemical engineering laboratory course, drawing upon interviews (n = 6), field notes, and written reflections (N = 31). We used discourse analysis to characterize students’ agency as they navigated their uncertainty and made sense of the authentic research. We found that students’ past experiences shaped their expectations about the CURE, with some wishing for traditional learning supports misaligned to CUREs. Offering students constrained yet consequential agency allowed them to recognize the authentic supports that were available, including help-seeking as itself a form of agency; understand failure as endemic to research rather than their own shortcoming; and position themselves as capable of navigating the uncertainty as a community of researchers. Our results suggest that instructors should model uncertainty and failure as endemic to research and position students as valued collaborators and support for overcoming abundant prior experiences with cookbook-style experiments.more » « less
