skip to main content

This content will become publicly available on December 1, 2022

Title: Connecting Activity Implementation Characteristics to Student Buy-In Toward and Utilization of Formative Assessments Within Undergraduate Biology Courses
Abstract While formative assessments (FAs) can facilitate learning within undergraduate STEM courses, their impact likely depends on many factors, including how instructors implement them, whether students buy-in to them, and how students utilize them. FAs have many different implementation characteristics, including what kinds of questions are asked, whether questions are asked before or after covering the material in class, how feedback is provided, how students are graded, and other logistical considerations. We conducted 38 semi-structured interviews with students from eight undergraduate biology courses to explore how various implementation characteristics of in-class and out-of-class FAs can influence student perceptions and behaviors. We also interviewed course instructors to provide context for understanding student experiences. Using thematic analysis, we outlined various FA implementation characteristics, characterized the range of FA utilization behaviors reported by students, and identified emergent themes regarding the impact of certain implementation characteristics on student buy-in and utilization. Furthermore, we found that implementation characteristics have combined effects on student engagement and that students will tolerate a degree of “acceptable discomfort” with implementation features that contradict their learning preferences. These results can aid instructor reflection and guide future research on the complex connections between activity implementation and student engagement within STEM disciplines.
Authors:
; ;
Award ID(s):
1610621
Publication Date:
NSF-PAR ID:
10310463
Journal Name:
Journal for STEM Education Research
Volume:
4
Issue:
3
ISSN:
2520-8705
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Implementing research-based teaching practices has been repeatedly cited as an important factor for student success in university mathematics courses. Many research-based practices increase the amount of student–student and/or student–instructor interaction. However, some instructors are hesitant to implement such practices because they anticipate their students reacting negatively to experiencing an interactive classroom. As part of a larger project studying introductory undergraduate mathematics courses in the United States, we investigated students’ perceptions of the helpfulness of various classroom characteristics, particularly those that require interaction.

    Results

    From analyzing quantitative student data, we found that students reported interactive classroom characteristics (e.g., group work) as less prevalent than other classroom characteristics (e.g., lecture). Moreover, the students tended to regard characteristics that they reported experiencing often as helpful for their learning. From analyzing qualitative data from student focus groups, we found that students considered several indicators when identifying if a characteristic was helpful for their learning. In particular, students suggested that they can identify a characteristic as helpful for their learning when it supported them in solving assigned problems and understanding why the procedures work, earning good grades, building on their knowledge or applying it in different contexts, and teaching others.

    Conclusions

    The key finding from ourmore »work is that students are likely to view classroom characteristics that they experience more often as more helpful for their learning and are less likely to view characteristics that they rarely experience as helpful for their learning. Students view the characteristics that they regularly experience as helping them to solve problems and understand why the procedures work, earn good grades, build on their knowledge or apply it in different contexts, and teach others. We discuss important implications for practice, policy, and research as it relates to both student and instructor buy-in for increasing interactions in class.

    « less
  2. This research paper studies the challenges that mathematics faculty and graduate teaching assistants (GTAs) faced when moving active and collaborative calculus courses from in-person to virtual instruction. As part of a larger pedagogical change project (described below), the math department at a public Research-1 university began transitioning pre-calculus and calculus courses to an active and collaborative learning (ACL) format in Fall 2019. The change began with the introduction of collaborative worksheets in recitations which were led by GTAs and supported by undergraduate learning assistants (LAs). Students recitation periods collaboratively solving the worksheet problems on whiteboards. When COVID-19 forced the rapid transition to online teaching, these ACL efforts faced an array of challenges. Faculty and GTA reflections on the changes to teaching and learning provide insight into how instructional staff can be supported in implementing ACL across various modes of instruction. The calculus teaching change efforts discussed in this paper are part of an NSF-supported project that aims to make ACL the default method of instruction in highly enrolled gateway STEM courses across the institution. The theoretical framework for the project builds on existing work on grassroots change in higher education (Kezar and Lester, 2011) to study the effect of communitiesmore »of practice on changing teaching culture. The project uses course-based communities of practice (Wenger, 1999) that include instructors, GTAs, and LAs working together to design and enact teaching change in the targeted courses alongside ongoing professional development for GTAs and LAs. Six faculty and five GTAs involved in the teaching change effort in mathematics were interviewed after the Spring 2020 semester ended. Interview questions focused on faculty and GTA experiences implementing active learning after the rapid transition to online teaching. A grounded coding scheme was used to identify common themes in the challenges faced by instructors and GTAs as they moved online and in the impacts of technology, LA support, and the department community of practice on the move to online teaching. Technology, including both access and capabilities, emerged as a common barrier to student engagement. A particular barrier was students’ reluctance to share video or participate orally in sessions that were being recorded, making group work more difficult than it had been in a physical classroom. In addition, most students lacked access to a tablet for freehand writing, presenting a significant hurdle for sharing mathematical notation when physical whiteboards were no longer an option. These challenges point to the importance of incorporating flexibility in active learning implementation and in the professional development that supports teaching changes toward active learning, since what is conceived for a collaborative physical classroom may be implemented in a much different environment. The full paper will present a detailed analysis of the data to better understand how faculty and GTA experiences in the transition to online delivery can inform planning and professional development as the larger institutional change effort moves forward both in mathematics and in other STEM fields.« less
  3. Evidence has shown that facilitating student-centered learning (SCL) in STEM classrooms enhances student learning and satisfaction [1]–[3]. However, despite increased support from educational and government bodies to incorporate SCL practices [1], minimal changes have been made in undergraduate STEM curriculum [4]. Faculty often teach as they were taught, relying heavily on traditional lecture-based teaching to disseminate knowledge [4]. Though some faculty express the desire to improve their teaching strategies, they feel limited by a lack of time, training, and incentives [4], [5]. To maximize student learning while minimizing instructor effort to change content, courses can be designed to incorporate simpler, less time-consuming SCL strategies that still have a positive impact on student experience. In this paper, we present one example of utilizing a variety of simple SCL strategies throughout the design and implementation of a 4-week long module. This module focused on introductory tissue engineering concepts and was designed to help students learn foundational knowledge within the field as well as develop critical technical skills. Further, the module sought to develop important professional skills such as problem-solving, teamwork, and communication. During module design and implementation, evidence-based SCL teaching strategies were applied to ensure students developed important knowledge and skills withinmore »the short timeframe. Lectures featured discussion-based active learning exercises to encourage student engagement and peer collaboration [6]–[8]. The module was designed using a situated perspective, acknowledging that knowing is inseparable from doing [9], and therefore each week, the material taught in the two lecture sessions was directly applied to that week’s lab to reinforce students’ conceptual knowledge through hands-on activities and experimental outcomes. Additionally, the majority of assignments served as formative assessments to motivate student performance while providing instructors with feedback to identify misconceptions and make real-time module improvements [10]–[12]. Students anonymously responded to pre- and post-module surveys, which focused on topics such as student motivation for enrolling in the module, module expectations, and prior experience. Students were also surveyed for student satisfaction, learning gains, and graduate student teaching team (GSTT) performance. Data suggests a high level of student satisfaction, as most students’ expectations were met, and often exceeded. Students reported developing a deeper understanding of the field of tissue engineering and learning many of the targeted basic lab skills. In addition to hands-on skills, students gained confidence to participate in research and an appreciation for interacting with and learning from peers. Finally, responses with respect to GSTT performance indicated a perceived emphasis on a learner-centered and knowledge/community-centered approaches over assessment-centeredness [13]. Overall, student feedback indicated that SCL teaching strategies can enhance student learning outcomes and experience, even over the short timeframe of this module. Student recommendations for module improvement focused primarily on modifying the lecture content and laboratory component of the module, and not on changing the teaching strategies employed. The success of this module exemplifies how instructors can implement similar strategies to increase student engagement and encourage in-depth discussions without drastically increasing instructor effort to re-format course content. Introduction.« less
  4. The drive to encourage young people to pursue degrees and careers in engineering has led to an increase in student populations in engineering programs. For some institutions, such as large public research institutions, this has led to large class sizes for courses that are commonly taken across multiple programs. While this decision is reasonable from an operational and resource management perspective, research on large classes have shown that students suffer decreased engagement, motivation and achievement. Instructors, on the other hand, report having difficulty establishing rapport with their students and a growing inability to monitor students’ learning gains and provide quality individualized feedback. To address these issues, our project draws from Lattuca and Stark’s Academic Plan model, which incorporates a thorough consideration of factors influencing curricular activities that can be applied at the course, program, and institutional levels, and assumes that instructors are key actors in curriculum development and revision. We aim to revitalize feedback loops to help instructors and departments continuously improve. Recognizing that we must understand both individual and systems level perspectives, we prioritize regular engagement between faculty and institutional support structures to collaboratively identify problems and systematically establish continuous improvement. In the first phase of this NSF IUSEmore »Institutional Transformation project, we focus on specifically prompting and studying the experiences of 8 instructors of foundational engineering courses usually taught in large class sizes across 4 different departments at a large public research institution. We collected qualitative data (semi-structured interviews, reflective journals, course-related documents) and quantitative data (student surveys and institution-provided transcript data) to answer research questions (e.g., what data do faculty teaching large foundational undergraduate engineering courses identify as being useful so that they may enhance students’ experiences and outcomes within the classes that they teach and across students’ multiple large classes?) at the intersection of learning analytics and faculty change. The data was used as a baseline to further refine data collection protocols, identify data that faculty consider meaningful and useful for managing large foundational engineering courses, and consider ways of productively leveraging institutional data to improve the learning experience in these courses. Data collection for the first phase is ongoing and will continue through the Spring 2018 semester. Findings for this paper will include high-level insights from Fall interviews with instructors as well as data visualizations created from the population-level data characterizing student performance in the foundational courses within the context of pre-college characteristics (e.g., SAT scores) and/or other academic outcomes (e.g., major switching within or out of engineer, degree attainment).« less
  5. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to these needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family tomore »attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees.« less