skip to main content

Title: You Are What You Assess: The Case for Emphasizing Chemistry on Chemistry Assessments
What we emphasize and reward on assessments signals to students what matters to us. Accordingly, a great deal of scholarship in chemistry education has focused on defining the sorts of performances worth assessing. Here, we unpack observations we made while analyzing what “success” meant across three large-enrollment general chemistry environments. We observed that students enrolled in two of the three environments could succeed without ever connecting atomic/molecular behavior to how and why phenomena happen. These environments, we argue, were not really “chemistry classes” but rather opportunities for students to gain proficiency with a jumble of skills and factual recall. However, one of the three environments dedicated 14–57% of points on exams to items with the potential to engage students in using core ideas (e.g., energy, bonding interactions) to predict, explain, or model observable events. This course, we argue, is more aligned with the intellectual work of the chemical sciences than the other two. If our courses assess solely (or largely) decontextualized skills and factual recall we risk (1) gating access to STEM careers on the basis of facility with skills most students will never use outside the classroom and (2) never allowing students to experience the tremendous predictive and explanatory more » power of atomic/molecular models. We implore the community to reflect on whether “what counts” in the courses we teach aligns with the performances we actually value. « less
Authors:
; ; ; ;
Award ID(s):
1726360
Publication Date:
NSF-PAR ID:
10290648
Journal Name:
Journal of chemical education
Volume:
98
Issue:
8
Page Range or eLocation-ID:
2490-2495
ISSN:
1050-6942
Sponsoring Org:
National Science Foundation
More Like this
  1. Within chemistry education, there are various curricular and pedagogical approaches that aim to improve teaching and learning in chemistry. Efforts to characterize these transformations have primarily focused on student reasoning and performance, and little work has been done to explore student perceptions of curricular and pedagogical transformations and whether these perceptions align with the transformational intent. To complement our previous work on the Organic Chemistry, Life, the Universe, and Everything (OCLUE) curriculum, we developed this exploratory study to determine if students had perceived the goals of the transformation. As in our previous research on OCLUE, we compared perceptions between OCLUE and a more traditional organic chemistry course. Using inductive and deductive qualitative methodologies, we analyzed student responses to three open-eneded questions focused on how students perceived they were expected to think, what they found most difficult, and how they perceived they were assessed. The findings were classified into three superodinate themes: one where students perceived they were expected to learn things as rote knowledge, such as memorization (“Rote Knowledge”), another where students perceived they were expected to use their knowledge (“Use of Knowledge”), and responses that used vague, generalized language, were uninformative, or did not address the questions asked (“Other”).more »Students in these two courses responded very differently to the open-ended questions with students in OCLUE being more likely to perceive they were expected to use their knowledge, while students in the traditional course reported rote learning or memorization more frequently. As the findings evolved, our interpretations and discussions were influenced by sociocultural perspectives and other cultural frameworks. We believe this approach can provide meaningful insights into transformational intent and certain features of classroom cultures.« less
  2. Abstract Background The growing understanding of the oppressive inequities that exist in postsecondary education has led to an increasing need for culturally relevant pedagogy. Researchers have found evidence that beliefs about the nature of knowledge predict pedagogical practices. Culturally relevant pedagogy supports students in ways that leverage students’ own cultures through three tenets: academic success, cultural competence, and sociopolitical consciousness. If STEM practitioners believe that their disciplines are culture-free, they may not enact culturally relevant pedagogy in their courses. We investigated how and in what forms 40 faculty from mathematics, physics, chemistry, and biology departments at Hispanic-Serving Institutions enacted culturally relevant pedagogy. We used the framework of practical rationality to understand how epistemological beliefs about the nature of their discipline combined with their institutional context impacted instructors’ decision to enact practices aligning with the three tenets of culturally relevant pedagogy. Results In total, 35 instructors reported using practices that aligned with the academic success tenet, nine instructors with the cultural competence tenet, and one instructor with the sociopolitical consciousness tenet. Instructors expressed and even lauded their disciplines’ separation from culture while simultaneously expressing instructional decisions that aligned with culturally relevant pedagogy. Though never asked directly, six instructors made statements reflectingmore »a “culture-free” belief about knowledge in their discipline such as “To me, mathematics has no color.” Five of those instructors also described altering their teaching in ways that aligned with the academic success tenet. The framework of practical rationality helped explain how the instructors’ individual obligation (to the needs of individual students) and interpersonal obligation (to the social environment of the classroom) played a role in those decisions. Conclusions Instructors’ ability to express two contradictory views may indicate that professional development does not have to change an instructor’s epistemological beliefs about their discipline to convince them of the value of enacting culturally relevant pedagogy. We propose departmental changes that could enable instructors to decide to cultivate students’ cultural competence and sociopolitical consciousness. Our findings highlight the need for future research investigating the impacts of culturally relevant pedagogical content knowledge on students’ experiences.« less
  3. Sacristán, A. I. ; Cortés-Zavala, J. C. ; Ruiz-Arias, P. M. (Ed.)
    What impact, if any, do interesting lessons have on the types of questions students ask? To explore this question, we used lesson observations of six teachers from three high schools in the Northeast who were part of a larger study. Lessons come from a range of courses, spanning Algebra through Calculus. After each lesson, students reported interest via lesson experience surveys (Author, 2019). These interest measures were then used to identify each teachers’ highest and lowest interest lessons. The two lessons per teacher allows us to compare across the same set of students per teacher. We compiled 145 student questions and identified whether questions were asked within a group work setting or part of a whole class discussion. Two coders coded 10% of data to improve the rubric for type of students’ questions (what, why, how, and if) and perceived intent (factual, procedural, reasoning, and exploratory). Factual questions asked for definitions or explicit answers. Procedural questions were raised when students looked for algorithms or a solving process. Reasoning questions asked about why procedures worked, or facts were true. Exploratory questions expanded beyond the topic of focus, such as asking about changing the parameters to make sense of a problem. Themore »remaining 90% of data were coded independently to determine interrater reliability (see Landis & Koch, 1977). A Cohen’s Kappa statistic (K=0.87, p<0.001) indicates excellent reliability. Furthermore, both coders reconciled codes before continuing with data analysis. Initial results showed differences between high- and low-interest lessons. Although students raised fewer mathematical questions in high-interest lessons (59) when compared with low-interest lessons (86), high-interest lessons contained more “exploratory” questions (10 versus 6). A chi-square test of independence shows a significant difference, χ2 (3, N = 145) = 12.99, p = .005 for types of students’ questions asked in high- and low-interest lessons. The high-interest lessons had more student questions arise during whole class discussions, whereas low-interest lessons had more student questions during group work. By partitioning each lesson into acts at points where the mathematical content shifted, we were able to examine through how many acts questions remained open. The average number of acts the students’ questions remained unanswered for high-interest lessons (2.66) was higher than that of low-interest lessons (1.68). Paired samples t-tests suggest that this difference is significant t(5)=2.58, p = 0.049. Therefore, student interest in the lesson did appear to impact the type of questions students ask. One possible reason for the differences in student questions is the nature of the lessons students found interesting, which may allow for student freedom to wonder and chase their mathematical ideas. There may be more overall student questions in low-interest lessons because of confusion, but more research is needed to unpack the reasoning behind student questions.« less
  4. This work in progress research paper considers the question, what kind of problems do engineering students commonly solve during their education? Engineering problems have been generally classified as ill-structured/open-ended or well-structured/closed-ended. Various authors have identified the characteristics of ill-structured problems or presented typologies of problems. Simple definitions state that well-structured problems are simple, concrete, and have a single solution, while ill-structured problems are complex, abstract, and have multiple possible solutions (Jonassen, 1997, 2000). More detailed classifications have been provided by Shin, Jonassen, and McGee (2003), Voss (2006), and Johnstone (2001). It is commonly understood that classroom problems are well-structured while workplace problems are ill-structured, but we cannot find any empirical data to confirm or deny this proposition. Engineers commonly encounter ill-structured problems such as design problems in the field therefore problem-solving skills are invaluable and should be taught in engineering courses. This research specifically looks at the types of problems present in the two most commonly used statics textbooks (Hibbeler, 2016; Beer, et al., 2019). All end-of-chapter problems in these textbooks were classified using Jonassen’s (2000) well-known typology of problem types. Out of 3,387 problems between both books, 99% fell into the algorithmic category and the remaining fell into themore »logic category. These preliminary results provide an understanding of the types of problems engineering students most commonly encounter in their classes. Prior research has documented that textbook example problems exert a strong influence on students' problem-solving strategies (Lee et al., 2013). If instructors only assign textbook problems, students in statics courses do not see any ill-structured problems at that stage in their education. We argue that even in foundational courses such as statics, students should be exposed to ill-structured problems. By providing opportunities for students to solve more ill-structured problems, students can become more familiar with them and become better prepared for the workforce. Moving forward, textbooks from several other courses will be analyzed to determine the difference between a fundamental engineering course such as statics and upper-level courses. This research will allow us to determine how the problem types differ between entry level and advanced classes and reveal if engineering textbooks primarily contain well-structured problems. Keywords: problem solving, textbooks, ill-structured problems« less
  5. ABSTRACT The global COVID-19 pandemic left universities with few options but to turn to remote learning. With much effort, STEM courses made this change in modality; however, many laboratory skills, such as measurement and handling equipment, are more difficult to teach in an online learning environment. A cohort of instructors who are part of the NSF RCN-UBE funded Sustainable, Transformative Engagement across a Multi-Institution/Multidisciplinary STEM (STEM 2 ) Network (a working group of faculty from two community colleges and three 4-year universities) analyzed introductory biology and chemistry courses to identify essential laboratory skills that students will need in advanced courses. Seven essential laboratory proficiencies were derived from reviewing disciplinary guiding documents such as AAAS’s Vision and Change in Undergraduate Biology Education, the American Society for Microbiology’s Recommended Curriculum Guidelines for Undergraduate Microbiology Education , and the American Chemical Society’s Guidelines for Chemistry : data analysis, scientific writing, proper handling and disposal of laboratory materials, discipline-specific techniques, measurement, lab safety and personal protective equipment, and interpersonal and collaborative skills. Our analysis has determined that some of these skills are difficult to develop in a remote online setting but could be recovered with appropriate interventions. Skill recovery procedures suggested are a skillsmore »“boot camp,” department and college coordinated club events, and a triage course. The authors recommend that one of these three recovery mechanisms be offered to bridge this skill gap and better prepare STEM students for upper-level science courses and the real world.« less