skip to main content


Title: Which evidence-based teaching practices change over time? Results from a university-wide STEM faculty development program
Abstract Background

There is overwhelming evidence that evidence-based teaching improves student performance; however, traditional lecture predominates in STEM courses. To provide support as faculty transform their lecture-based classrooms with evidence-based teaching practices, we created a faculty development program based on best practices, Consortium for the Advancement of Undergraduate STEM Education (CAUSE). CAUSE paired exploration of evidence-based teaching with support for classroom implementation over two years. Each year for three years, CAUSE recruited cohorts of faculty from seven STEM departments. Faculty met biweekly to discuss evidence-based teaching and receive feedback on their implementation. We used the PORTAAL observation tool to document evidence-based teaching practices (PORTAAL practices) across four randomly chosen class sessions each term. We investigated if the number of PORTAAL practices used or the amount of practices increased during the program.

Results

We identified identical or equivalent course offerings taught at least twice by the same faculty member while in CAUSE (n = 42 course pairs). We used a one-way repeated measures within-subjects multivariate analysis to examine the changes in average use of 14 PORTAAL practices between the first and second timepoint. We created heat maps to visualize the difference in number of practices used and changes in level of implementation of each PORTAAL practice. Post-hoc within-subjects effects indicated that three PORTAAL practices were significantly higher and two were lower at timepoint two. Use of prompting prior knowledge and calling on volunteers to give answers decreased, while instructors doubled use of prompting students to explain their logic, and increased use of random call by almost 40% when seeking answers from students. Heat maps indicated increases came both from faculty’s adoption of these practices and increased use, depending on the practice. Overall, faculty used more practices more frequently, which contributed to a 17% increase in time that students were actively engaged in class.

Conclusions

Results suggest that participation in a long-term faculty development program can support increased use of evidence-based teaching practices which have been shown to improve student exam performance. Our findings can help prioritize the efforts of future faculty development programs.

 
more » « less
Award ID(s):
1725149
NSF-PAR ID:
10363339
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
International Journal of STEM Education
Volume:
9
Issue:
1
ISSN:
2196-7822
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    An instructor’s conceptions of teaching and learning contribute to the establishment of learning environments that may benefit or hinder student learning. Previous studies have defined the continuum of teaching and learning conceptions, ranging from limited to complete, as well as the instructional practices that they help to inform (instructor-centered to student-centered), and the corresponding learning environments that these conceptions and practices establish, ranging from traditional to student-centered. Using the case of one STEM department at a research-intensive, minority serving institution, we explored faculty’s conceptions of teaching and learning and their resulting instructional practices, as well as uncovered their perspectives on the intradepartmental faculty interactions related to teaching. The study participants were drawn from both teaching-focused (called Professors of Teaching, PoTs) and traditional research (whom we call Research Professors, RPs) tenure-track faculty lines to identify whether differences existed amongst these two populations. We used interviews to explore faculty conceptions and analyzed syllabi to unveil how these conceptions shape instructional environments.

    Results

    Overall, PoTs exhibited complete conceptions of teaching and learning that emphasized student ownership of learning, whereas RPs possessed intermediate conceptions that focused more on transmitting knowledge and helping students prepare for subsequent courses. While both PoTs and RPs self-reported the use of active learning pedagogies, RPs were more likely to also highlight the importance of traditional lecture. The syllabi analysis revealed that PoTs enacted more student-centered practices in their classrooms compared to RPs. PoTs appeared to be more intentionally available to support students outside of class and encouraged student collaboration, while RPs focused more on the timeliness of assessments and enforcing more instructor-centered approaches in their courses. Finally, the data indicated that RPs recognized PoTs as individuals who were influential on their own teaching conceptions and practices.

    Conclusions

    Our findings suggest that departments should consider leveraging instructional experts who also possess a disciplinary background (PoTs) to improve their educational programs, both due to their student-centered impacts on the classroom environment and positive influence on their colleagues (RPs). This work also highlights the need for higher education institutions to offer appropriate professional development resources to enable faculty to reflect on their teaching and learning conceptions, aid in their pedagogical evolution, and guide the implementation of these conceptions into practice.

     
    more » « less
  2. Evidence has shown that facilitating student-centered learning (SCL) in STEM classrooms enhances student learning and satisfaction [1]–[3]. However, despite increased support from educational and government bodies to incorporate SCL practices [1], minimal changes have been made in undergraduate STEM curriculum [4]. Faculty often teach as they were taught, relying heavily on traditional lecture-based teaching to disseminate knowledge [4]. Though some faculty express the desire to improve their teaching strategies, they feel limited by a lack of time, training, and incentives [4], [5]. To maximize student learning while minimizing instructor effort to change content, courses can be designed to incorporate simpler, less time-consuming SCL strategies that still have a positive impact on student experience. In this paper, we present one example of utilizing a variety of simple SCL strategies throughout the design and implementation of a 4-week long module. This module focused on introductory tissue engineering concepts and was designed to help students learn foundational knowledge within the field as well as develop critical technical skills. Further, the module sought to develop important professional skills such as problem-solving, teamwork, and communication. During module design and implementation, evidence-based SCL teaching strategies were applied to ensure students developed important knowledge and skills within the short timeframe. Lectures featured discussion-based active learning exercises to encourage student engagement and peer collaboration [6]–[8]. The module was designed using a situated perspective, acknowledging that knowing is inseparable from doing [9], and therefore each week, the material taught in the two lecture sessions was directly applied to that week’s lab to reinforce students’ conceptual knowledge through hands-on activities and experimental outcomes. Additionally, the majority of assignments served as formative assessments to motivate student performance while providing instructors with feedback to identify misconceptions and make real-time module improvements [10]–[12]. Students anonymously responded to pre- and post-module surveys, which focused on topics such as student motivation for enrolling in the module, module expectations, and prior experience. Students were also surveyed for student satisfaction, learning gains, and graduate student teaching team (GSTT) performance. Data suggests a high level of student satisfaction, as most students’ expectations were met, and often exceeded. Students reported developing a deeper understanding of the field of tissue engineering and learning many of the targeted basic lab skills. In addition to hands-on skills, students gained confidence to participate in research and an appreciation for interacting with and learning from peers. Finally, responses with respect to GSTT performance indicated a perceived emphasis on a learner-centered and knowledge/community-centered approaches over assessment-centeredness [13]. Overall, student feedback indicated that SCL teaching strategies can enhance student learning outcomes and experience, even over the short timeframe of this module. Student recommendations for module improvement focused primarily on modifying the lecture content and laboratory component of the module, and not on changing the teaching strategies employed. The success of this module exemplifies how instructors can implement similar strategies to increase student engagement and encourage in-depth discussions without drastically increasing instructor effort to re-format course content. Introduction. 
    more » « less
  3. Dalby, Andrew R. (Ed.)
    Traditional teaching practices in undergraduate science, technology, engineering, and mathematics (STEM) courses have failed to support student success, causing many students to leave STEM fields and disproportionately affecting women and students of color. Although much is known about effective STEM teaching practices, many faculty continue to adhere to traditional methods, such as lecture. In this study, we investigated the factors that affect STEM faculty members’ instructional decisions about evidence-based instructional practices (EBIPs). We performed a qualitative analysis of semi-structured interviews with faculty members from the Colleges of Physical and Mathematical Sciences, Life Sciences, and Engineering who took part in a professional development program to support the use of EBIPs by STEM faculty at the university. We used an ecological model to guide our investigation and frame the results. Faculty identified a variety of personal, social, and contextual factors that influenced their instructional decision-making. Personal factors included attitudes, beliefs, and self-efficacy. Social factors included the influence of students, colleagues, and administration. Contextual factors included resources, time, and student characteristics. These factors interact with each other in meaningful ways that highlight the hyper-local social contexts that exist within departments and sub-department cultures, the importance of positive feedback from students and colleagues when implementing EBIPs, and the need for support from the administration for faculty who are in the process of changing their teaching. 
    more » « less
  4. Background/Context:

    Computer programming is rarely accessible to K–12 students, especially for those from culturally and linguistically diverse backgrounds. Middle school age is a transitioning time when adolescents are more likely to make long-term decisions regarding their academic choices and interests. Having access to productive and positive knowledge and experiences in computer programming can grant them opportunities to realize their abilities and potential in this field.

    Purpose/Focus of Study:

    This study focuses on the exploration of the kind of relationship that bilingual Latinx students developed with themselves and computer programming and mathematics (CPM) practices through their participation in a CPM after-school program, first as students and then as cofacilitators teaching CPM practices to other middle school peers.

    Setting:

    An after-school program, Advancing Out-of-School Learning in Mathematics and Engineering (AOLME), was held at two middle schools located in rural and urban areas in the Southwest. It was designed to support an inclusive cultural environment that nurtured students’ opportunities to learn CPM practices through the inclusion of languages (Spanish and English), tasks, and participants congruent to students in the program. Students learned how to represent, design, and program digital images and videos using a sequence of 2D arrays of hexadecimal numbers with Python on a Raspberry Pi computer. The six bilingual cofacilitators attended Levels 1 and 2 as students and were offered the opportunity to participate as cofacilitators in the next implementation of Level 1.

    Research Design:

    This longitudinal case study focused on analyzing the experiences and shifts (if any) of students who participated as cofacilitators in AOLME. Their narratives were analyzed collectively, and our analysis describes the experiences of the cofacilitators as a single case study (with embedded units) of what it means to be a bilingual cofacilitator in AOLME. Data included individual exit interviews of the six cofacilitators and their focus groups (30–45 minutes each), an adapted 20-item CPM attitude 5-point Likert scale, and self-report from each of them. Results from attitude scales revealed cofacilitators’ greater initial and posterior connections to CPM practices. The self-reports on CPM included two number lines (0–10) for before and after AOLME for students to self-assess their liking and knowledge of CPM. The numbers were used as interview prompts to converse with students about experiences. The interview data were analyzed qualitatively and coded through a contrast-comparative process regarding students’ description of themselves, their experiences in the program, and their perception of and relationship toward CPM practices.

    Findings:

    Findings indicated that students had continued/increased motivation and confidence in CPM as they engaged in a journey as cofacilitators, described through two thematic categories: (a) shifting views by personally connecting to CPM, and (b) affirming CPM practices through teaching. The shift in connecting to CPM practices evolved as students argued that they found a new way of learning mathematics, in that they used mathematics as a tool to create videos and images that they programmed by using Python while making sense of the process bilingually (Spanish and English). This mathematics was viewed by students as high level, which in turned helped students gain self-confidence in CPM practices. Additionally, students affirmed their knowledge and confidence in CPM practices by teaching them to others, a process in which they had to mediate beyond the understanding of CPM practices. They came up with new ways of explaining CPM practices bilingually to their peers. In this new role, cofacilitators considered the topic and language, and promoted a communal support among the peers they worked with.

    Conclusions/Recommendations:

    Bilingual middle school students can not only program, but also teach bilingually and embrace new roles with nurturing support. Schools can promote new student roles, which can yield new goals and identities. There is a great need to redesign the school mathematics curriculum as a discipline that teenagers can use and connect with by creating and finding things they care about. In this way, school mathematics can support a closer “fit” with students’ identification with the world of mathematics. Cofacilitators learned more about CPM practices by teaching them, extending beyond what was given to them, and constructing new goals that were in line with a sophisticated knowledge and shifts in the practice. Assigned responsibility in a new role can strengthen students’ self-image, agency, and ways of relating to mathematics.

     
    more » « less
  5. There have been numerous demands for enhancements in the way undergraduate learning occurs today, especially at a time when the value of higher education continues to be called into question (The Boyer 2030 Commission, 2022). One type of demand has been for the increased integration of subjects/disciplines around relevant issues/topics—with a more recent trend of seeking transdisciplinary learning experiences for students (Sheets, 2016; American Association for the Advancement of Science, 2019). Transdisciplinary learning can be viewed as the holistic way of working equally across disciplines to transcend their own disciplinary boundaries to form new conceptual understandings as well as develop new ways in which to address complex topics or challenges (Ertas, Maxwell, Rainey, & Tanik, 2003; Park & Son, 2010). This transdisciplinary approach can be important as humanity’s problems are not typically discipline specific and require the convergence of competencies to lead to innovative thinking across fields of study. However, higher education continues to be siloed which makes the authentic teaching of converging topics, such as innovation, human-technology interactions, climate concerns, or harnessing the data revolution, organizationally difficult (Birx, 2019; Serdyukov, 2017). For example, working across a university’s academic units to collaboratively teach, or co-teach, around topics of convergence are likely to be rejected by the university systems that have been built upon longstanding traditions. While disciplinary expertise is necessary and one of higher education’s strengths, the structures and academic rigidity that come along with the disciplinary silos can prevent modifications/improvements to the roles of academic units/disciplines that could better prepare students for the future of both work and learning. The balancing of disciplinary structure with transdisciplinary approaches to solving problems and learning is a challenge that must be persistently addressed. These institutional challenges will only continue to limit universities seeking toward scaling transdisciplinary programs and experimenting with novel ways to enhance the value of higher education for students and society. This then restricts innovations to teaching and also hinders the sharing of important practices across disciplines. To address these concerns, a National Science Foundation Improving Undergraduate STEM Education project team, which is the topic of this paper, has set the goal of developing/implementing/testing an authentically transdisciplinary, and scalable educational model in an effort to help guide the transformation of traditional undergraduate learning to span academics silos. This educational model, referred to as the Mission, Meaning, Making (M3) program, is specifically focused on teaching the crosscutting practices of innovation by a) implementing co-teaching and co-learning from faculty and students across different academic units/colleges as well as b) offering learning experiences spanning multiple semesters that immerse students in a community that can nourish both their learning and innovative ideas. As a collaborative initiative, the M3 program is designed to synergize key strengths of an institution’s engineering/technology, liberal arts, and business colleges/units to create a transformative undergraduate experience focused on the pursuit of innovation—one that reaches the broader campus community, regardless of students’ backgrounds or majors. Throughout the development of this model, research was conducted to help identify institutional barriers toward creating such a cross-college program at a research-intensive public university along with uncovering ways in which to address these barriers. While data can show how students value and enjoy transdisciplinary experiences, universities are not likely to be structured in a way to support these educational initiatives and they will face challenges throughout their lifespan. These challenges can result from administration turnover whereas mutual agreements across colleges may then vanish, continued disputes over academic territory, and challenges over resource allotments. Essentially, there may be little to no incentives for academic departments to engage in transdisciplinary programming within the existing structures of higher education. However, some insights and practices have emerged from this research project that can be useful in moving toward transdisciplinary learning around topics of convergence. Accordingly, the paper will highlight features of an educational model that spans disciplines along with the workarounds to current institutional barriers. This paper will also provide lessons learned related to 1) the potential pitfalls with educational programming becoming “un-disciplinary” rather than transdisciplinary, 2) ways in which to incentivize departments/faculty to engage in transdisciplinary efforts, and 3) new structures within higher education that can be used to help faculty/students/staff to more easily converge to increase access to learning across academic boundaries. 
    more » « less