An official website of the United States government
Visual literacy is recognized as a threshold concept in biochemistry and molecular biology. However, a consensus on the optimal methods for teaching and evaluating remains elusive. For a decade, BioMolViz has strived to enhance biomolecular visualization assessment. Through workshops and online working groups, we guide instructors on how to probe biomolecular visual literacy using accessible images and questions, which are ultimately shared broadly through our online repository (the BioMolViz Library). Here, we present the final step of our assessment validation process which occurred during the 2022-2023 academic year. We engaged life science students from seven U.S.-based institutions in a pilot field test. Students responded to the multiple choice, multiple select and free response items, rated them on their perceived difficulty, and provided optional open-ended feedback. As we examined the data, we became curious about whether instructors viewed the difficulty level of the items similarly to students. We followed up with an instructor survey where respondents rated and commented on the difficulty of 14 assessment items that were administered to students in the pilot field test. Subsequently, we conducted a mixed methods study to analyze our quantitative and qualitative data. Our analysis revealed a statistically significant disparity between instructors' and students' perceptions of assessment difficulty. Notably, regression models suggest that students' performance predicts their perceived difficulty, with high-performing students finding the assessment generally easier than their lower-performing peers. This points to the crucial role of performance in shaping students' perceptions, while also indicating that instructors, on the whole, tended to view the assessment as less challenging than students. To gain deeper insights into these findings, we performed thematic coding of both student and instructor responses. Our analysis unveiled three pivotal themes in visual literacy assessment: (a) expectations about images guide student performance, (b) disparities exist in visual literacy problem solving, and (c) content knowledge can be both a help and hindrance in visualization. Importantly, these results have changed the way members of our team now approach teaching and evaluating biomolecular visualization skills in our own classrooms. We will share our revised approaches alongside results from our study and provide practical recommendations to aid educators in effectively teaching and evaluating visual literacy in their classrooms. This material is supported by the National Science Foundation (NSF) under grants RCN-UBE #1920270 and NSF-IUSE #1712268
more »
« less
Preparing, Visualizing, and Using Real-world Data in Introductory Courses
In recent years, interactive textbooks have gained prominence in an effort to overcome student reluctance to routinely read textbooks, complete assigned homeworks, and to better engage students to keep up with lecture content. Interactive textbooks are more structured, contain smaller amounts of textual material, and integrate media and assessment content. While these are an arguable improvement over traditional methods of teaching, issues of academic integrity and engagement remain. In this work we demonstrate preliminary work on building interactive teaching modules for data structures and algorithms courses with the following characteristics, (1) the modules are highly visual and interactive, (2) training and assessment are tightly integrated within the same module, with sufficient variability in the exercises to make it next to impossible to violate academic integrity, (3) a data logging and analytic system that provides instantaneous student feedback and assessment, and (4) an interactive visual analytic system for the instructor to see students’ performance at the individual, sub-group or class level, allowing timely intervention and support for selected students. Our modules are designed to work within the infrastructure of the OpenDSA system, which will promote rapid dissemination to an existing user base of CS educators. We demonstrate a prototype system using an example dataset.
more »
« less
- PAR ID:
- 10064382
- Date Published:
- Journal Name:
- ACM Special Interest Group on Computer Science Education 2018
- Page Range / eLocation ID:
- 676 to 677
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Data leakage remains a pervasive issue in machine learning (ML), especially when applied to science, leading to overly optimistic performance estimates and irreproducible findings. Despite its prevalence, data leakage receives limited attention in ML education, in part due to the lack of accessible, hands-on teaching resources. To address this gap, we developed interactive learning modules in which students reproduce examples from academic publications that are affected by data leakage, then repeat the evaluation without the data leakage error to see how the finding is affected. These modules were deployed by the authors in two introductory machine learning courses, enabling students to explore common forms of leakage and their impact on model reliability. Following their engagement with these materials, student feedback highlighted increased awareness of subtle pitfalls that can compromise machine learning workflows.more » « less
-
This work-in-progress paper presents an innovative practice of using oral exams to maintain academic integrity and promote student engagement in large-enrollment engineering courses during remote instruction. With the abrupt and widespread transition to distance learning and assessment brought on by the COVID-19 pandemic, there has been a registered upsurge in academic integrity violations globally. To address the challenge of compromised integrity, in the winter quarter of 2021 we have implemented oral exams across six mostly high-enrollment mechanical and electrical engineering undergraduate courses. We present our oral exam design parameters in each of the courses and discuss how oral exams relate to academic integrity, student engagement, stress, and implicit bias. We also address the challenge of scalability, as most of our oral exams were implemented in large classes, where academic integrity and student-instructor disconnection have generally gotten disproportionately worse during remote learning. Our survey results indicate that oral exams have positively contributed to academic integrity in our courses. Based on our preliminary study and experiences, we expect oral exams can be effectively leveraged to hinder cheating and foster academic honesty in students, even when in-person instruction and assessment resumes.more » « less
-
Mobile devices are being used profusely in the classrooms to improve passive learning environments and to enhance student comprehension. However, with respect to students’ active involvement in problem solving activities, the typical usage of the mobile devices in answering multiple choice and true/false questions is not adequate and the use of mobile devices need to be expanded to include dynamic and interactive problem-solving activities to better satisfy students’ learning needs. To facilitate such interactive problem solving using mobile devices, a comprehensive software environment is necessary. This paper details the design, deployment and evaluation of Mobile Response System (MRS) software that facilitates execution and assessment of multi-step in-class interactive problem-solving activities using mobile devices. MRS is an active learning tool, which engages students with the visual representation of a problem that spans on multiple screens, allows them to interact with that, and makes them realize the consequences of their actions instantly and visually. The immediate and automated grading feature of MRS enables a feedback-driven and evidence-based teaching methodology, which is important to improve the quality of classroom learning. MRS is designed to be independent of any interactive problem or its domain. Therefore, it allows easier integration of interactive activity Apps developed by others and can be used in any discipline. The results obtained from software metrics and runtime performance data verified the quality of the software. Additionally, the in-class assessment data verified that the MRS software is a helpful intervention for improving student comprehension and satisfaction.more » « less
-
Security is a critical aspect in the process of designing, developing, and testing software systems. Due to the increasing need for security-related skills within software systems, there is a growing demand for these skills to be taught in computer science. A series of security modules was developed not only to meet the demand but also to assess the impact of these modules on teaching critical cybersecurity topics in computer science courses. This full paper in the innovative practice category presents the outcomes of six security modules in a freshman-level course at two institutions. The study adopts a Model-Eliciting Activity (MEA) as a project for students to demonstrate an understanding of security concepts. Two experimental studies were conducted: 1) Teaching effectiveness of implementing cyber security modules and MEA projects, and 2) Students’ experiences in conceptual modeling tasks in problem-solving. In measuring the effectiveness of teaching security concepts with the MEA project, students’ performance, attitudes, and interests as well as the instructor’s effectiveness were assessed. For the conceptual modeling tasks in problem-solving, the results of student outcomes were analyzed. After implementing the security modules with the MEA project, students showed a great understanding of cybersecurity concepts and an increased interest in broader computer science concepts. The instructor’s beliefs about teaching, learning, and assessment shifted from teacher-centered to student-centered during their experience with the security modules and MEA project. Although 64.29% of students’ solutions do not seem suitable for real-world implementation, 76.9% of the developed solutions showed a sufficient degree of creativity.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3159450.3159616
