An official website of the United States government
Visual literacy is recognized as a threshold concept in biochemistry and molecular biology. However, a consensus on the optimal methods for teaching and evaluating remains elusive. For a decade, BioMolViz has strived to enhance biomolecular visualization assessment. Through workshops and online working groups, we guide instructors on how to probe biomolecular visual literacy using accessible images and questions, which are ultimately shared broadly through our online repository (the BioMolViz Library). Here, we present the final step of our assessment validation process which occurred during the 2022-2023 academic year. We engaged life science students from seven U.S.-based institutions in a pilot field test. Students responded to the multiple choice, multiple select and free response items, rated them on their perceived difficulty, and provided optional open-ended feedback. As we examined the data, we became curious about whether instructors viewed the difficulty level of the items similarly to students. We followed up with an instructor survey where respondents rated and commented on the difficulty of 14 assessment items that were administered to students in the pilot field test. Subsequently, we conducted a mixed methods study to analyze our quantitative and qualitative data. Our analysis revealed a statistically significant disparity between instructors' and students' perceptions of assessment difficulty. Notably, regression models suggest that students' performance predicts their perceived difficulty, with high-performing students finding the assessment generally easier than their lower-performing peers. This points to the crucial role of performance in shaping students' perceptions, while also indicating that instructors, on the whole, tended to view the assessment as less challenging than students. To gain deeper insights into these findings, we performed thematic coding of both student and instructor responses. Our analysis unveiled three pivotal themes in visual literacy assessment: (a) expectations about images guide student performance, (b) disparities exist in visual literacy problem solving, and (c) content knowledge can be both a help and hindrance in visualization. Importantly, these results have changed the way members of our team now approach teaching and evaluating biomolecular visualization skills in our own classrooms. We will share our revised approaches alongside results from our study and provide practical recommendations to aid educators in effectively teaching and evaluating visual literacy in their classrooms. This material is supported by the National Science Foundation (NSF) under grants RCN-UBE #1920270 and NSF-IUSE #1712268
more »
« less
Seeing Eye to Eye? Comparing Faculty and Student Perceptions of Biomolecular Visualization Assessments
While visual literacy has been identified as a foundational skill in life science education, there are many challenges in teaching and assessing biomolecular visualization skills. Among these are the lack of consensus about what constitutes competence and limited understanding of student and instructor perceptions of visual literacy tasks. In this study, we administered a set of biomolecular visualization assessments, developed as part of the BioMolViz project, to both students and instructors at multiple institutions and compared their perceptions of task difficulty. We then analyzed our findings using a mixed-methods approach. Quantitative analysis was used to answer the following research questions: (1) Which assessment items exhibit statistically significant disparities or agreements in perceptions of difficulty between instructors and students? (2) Do these perceptions persist when controlling for race/ethnicity and gender? and (3) How does student perception of difficulty relate to performance? Qualitative analysis of open-ended comments was used to identify predominant themes related to visual problem solving. The results show that perceptions of difficulty significantly differ between students and instructors and that students’ performance is a significant predictor of their perception of difficulty. Overall, this study underscores the need to incorporate deliberate instruction in visualization into undergraduate life science curricula to improve student ability in this area. Accordingly, we offer recommendations to promote visual literacy skills in the classroom.
more »
« less
- Award ID(s):
- 1920270
- PAR ID:
- 10506589
- Editor(s):
- Jodie Jenkinson, Susan Keen
- Publisher / Repository:
- MDPI
- Date Published:
- Journal Name:
- Education Sciences
- Volume:
- 14
- Issue:
- 1
- ISSN:
- 2227-7102
- Page Range / eLocation ID:
- 94
- Subject(s) / Keyword(s):
- biomolecular visualization visual literacy perception perceived difficulty assessment assessment difficulty assessment quality control
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A brief glance through molecular biology and biochemistry textbooks underscores the importance of interpreting visual images in the molecular life sciences. In fact, biomolecular visual literacy has been deemed a threshold concept, essential for student success in the field. As one example, grasping the information displayed in visual representations is a gateway to a deep understanding of structure-function relationships, a core concept in biology education. Despite much interest, few studies have examined the assessment of visual literacy skills in the area of biomolecules. Ten years ago, BioMolViz began an initiative to improve biomolecular visualization instruction and assessment, which focused on developing validated assessments to probe students' visual literacy skills. In 2023, we introduced the BioMolViz Library, a repository where instructors can access the instruments built by our community. A subset of these assessments were administered in classrooms in a pilot field test during the 2022–2023 academic year. We gained invaluable information from both quantitative and qualitative data collected. Lessons learned from this first classroom test guided the design of the 2023–2024 large-scale field testing we describe here with over ten partner institutions, high enrollment classes, and an increased number of items per survey. We present the results of our analysis of item difficulty, discrimination, and distractor analysis, alongside a robust analysis of the influence of gender and race/ethnicity on student performance. To improve the statistical power of the study, we exchanged open-ended written feedback for an increased number of assessment items administered on each survey. However, recognizing the value of student feedback obtained through a mixed methods analysis from our 2022–2023 study, we followed up with focus groups to explore the perceptions and problem solving process of both low- and high-performing students. We present the results of our assessment validation, including an analysis of the influence of learner level, gender identity, and race/ethnicity on performance. We include suggestions for equitable and inclusive assessment methods as we continue to strive to improve visual literacy instruction. This material is supported by the National Science Foundation (NSF) under grants RCN-UBE #1920270 and NSF-IUSE #1712268.more » « less
-
BioMolViz is a community of practice dedicated to improving biomolecular visualization instruction. Guided by a framework of learning objectives for biomolecular visualization skills, our initial project goal was to create a repository of validated assessments to evaluate students’ visual literacy. In 2018, the team was awarded one year of seed funding, which led to a four-year National Science Foundation (NSF) grant. This support allowed BioMolViz to flourish into a community of educators in professional development workshops and working groups, where teams of participants aimed to design effective and accessible assessments to evaluate students’ biomolecular visual literacy. As the project advanced, we piloted these items in classrooms across the United States. Through a small-scale classroom testing study, we compared student and instructor perceptions of assessment difficulty, while large-scale testing revealed performance patterns that reinforced the need to understand distinct student perspectives. This led us to evaluate students’ problem-solving strategies through surveys and semi-structured interviews. Based on this work, we argue that a reimagining of the curriculum can begin with faculty, but must include productive student partnerships to enact effective change. We offer our repository of visual literacy assessments, the BioMolViz Library, as an instructor resource shaped by the student perspective, and present a new instructor training resource recently produced by our working group. As we approach the close of our funding cycle, we offer our ideas and invite conversations on fostering long-term sustainability for our robust community of practice, under all future resource models.more » « less
-
null (Ed.)The promotion of global sustainability within environmental science courses requires a paradigm switch from knowledge-based teaching to teaching that stimulates higher-order cognitive skills. Non-major undergraduate science courses, such as environmental science, promote critical thinking in students in order to improve the uptake of scientific information and develop the rational decision making used to make more informed decisions. Science, engineering, technology and mathematics (STEM) courses rely extensively on visuals in lectures, readings and homework to improve knowledge. However, undergraduate students do not automatically acquire visual literacy and a lack of intervention from instructors could be limiting academic success. In this study, a visual literacy intervention was developed and tested in the face-to-face (FTF) and online sections of an undergraduate non-major Introduction to Environmental Science course. The intervention was designed to test and improve visual literacy at three levels: (1) elementary—identifying values; (2) intermediate—identifying trends; and (3) advanced—using the data to make projections or conclusions. Students demonstrated a significant difference in their ability to answer elementary and advanced visual literacy questions in both course sections in the pre-test and post-test. Students in the face-to-face course had significantly higher exam scores and higher median assessment scores compared to sections without a visual literacy intervention. The online section did not show significant improvements in visual literacy or academic success due to a lack of reinforcement of visual literacy following the initial intervention. The visual literacy intervention shows promising results in improving student academic success and should be considered for implementation in other general education STEM courses.more » « less
-
Abstract While molecular visualization has been recognized as a threshold concept in biology education, the explicit assessment of students' visual literacy skills is rare. To facilitate the evaluation of this fundamental ability, a series of NSF‐IUSE‐sponsored workshops brought together a community of faculty engaged in creating instruments to assess students' biomolecular visualization skills. These efforts expanded our earlier work in which we created a rubric describing overarching themes, learning goals, and learning objectives that address student progress toward biomolecular visual literacy. Here, the BioMolViz Steering Committee (BioMolViz.org) documents the results of those workshops and uses social network analysis to examine the growth of a community of practice. We also share many of the lessons we learned as our workshops evolved, as they may be instructive to other members of the scientific community as they organize workshops of their own.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/educsci14010094
