Search for: All records

Abstract BackgroundNumerous studies show that active and engaging classrooms help students learn and persist in college, but adoption of new teaching practices has been slow. Professional development programs encourage instructors to implement new teaching methods and change the status quo in STEM undergraduate teaching, and structured observations of classrooms can be used in multiple ways to describe and assess this instruction. We addressed the challenge of measuring instructional change with observational protocols, data that often do not lend themselves easily to statistical comparisons. Challenges using observational data in comparative research designs include lack of descriptive utility for holistic measures and problems related to construct representation, non-normal distributions and Type-I error inflation for segmented measures. ResultsWe grouped 790 mathematics classes from 74 instructors using Latent Profile Analysis (a statistical clustering technique) and found four reliable categories of classes. Based on this grouping we proposed a simple proportional measure we called Proportion Non-Didactic Lecture (PND). The measure aggregated the proportions of interactive to lecture classes for each instructor. We tested the PND and a measure derived from the Reformed Teaching Observation Protocol (RTOP) with data from a professional development study. The PND worked in simple hypothesis tests but lacked some statistical power due to possible ceiling effects. However, the PND provided effective descriptions of changes in instructional approaches from pre to post. In tandem with examining the proportional measure, we also examined the RTOP-Sum, an existing outcome measure used in comparison studies. The measure is based on the aggregated items in a holistic observational protocol. As an aggregate measure we found it to be highly reliable, correlated highly with the PND, and had more statistical power than the PND. However, the RTOP measure did not provide the thick descriptions of teaching afforded by the PND. ConclusionsFindings suggest that useful dependent measures can be derived from both segmented and holistic observational measures. Both have strengths and weaknesses: measures from segmented data are best at describing changes in teaching, while measures derived from the RTOP have more statistical power. Determining the validity of these measures is important for future use of observational data in comparative studies. 

Teaching observations can be used in multiple ways to describe and assess instruction. We addressed the challenge of measuring instructional change with observational protocols, data that often do not lend themselves easily to statistical comparisons. We first grouped 790 mathematics classes using Latent Profile Analysis and found four reliable categories of classes. Based on the grouping we proposed a proportional measure called Proportion Non-Didactic Lecture (PND). The measure is the proportion of interactive to lecture classes for each instructor. The PND worked in simple hypothesis tests but lacked some statistical power due to possible scaler ceiling effects. The measure correlated highly with a dependent measure derived from the Reformed Teaching Observation Protocol (RTOP), a holistic observational measure. The PND also provided effective descriptions and visualizations of instructional approaches and how these changed from pre to post. 

Professional development has been identified as an effective way to increase college STEM instructors’ use of research-based instructional strategies (RBIS) known to benefit student learning and persistence in STEM. Yet only a few studies relate professional development experiences to later teaching behaviors of higher education instructors. This study of 361 undergraduate mathematics instructors, all of whom participated in multi-day, discipline-based workshops on teaching held in 2010–2019, examined the relationship between such participation and later use of RBIS. We found that instructors’ RBIS attitudes, knowledge, and skills strengthened after participating in professional development, and their self-reported use of RBIS became more frequent in the first year after the workshop. Applying the Theory of Planned Behavior as a conceptual framework, we used a structural equation model to test whether this theory could explain the roles of workshop participation and other personal, professional and contextual factors in fostering RBIS use. Findings indicated that, along with workshop participation, prior RBIS experience, class size, and course coordination affected RBIS use. That is, both targeted professional development and elements of the local context for implementation were important in supporting instructors’ uptake of RBIS—but, remarkably, both immediate and longer-term outcomes of professional development did not depend on other individual or institutional characteristics. In this study, the large sample size, longitudinal measurement approach, and consistency of the form and quality of professional development make it possible to distinguish the importance of multiple possible influences on instructors’ uptake of RBIS. We discuss implications for professional development and for institutional structures that support instructors as they apply what they learned, and we offer suggestions for the use of theory in future research on this topic. 

Faculty professional development is an important lever for change in supporting instructors to adopt research-based instructional strategies that engage students intellectually, foster learning-supportive attitudes and habits of mind, and strengthen their persistence in mathematics. Yet the literature contains few well-rationalized models for faculty development in higher education. We describe the rationale and design for a model for discipline-based faculty development to support instructional change, and we detail our implementation of this model as applied to intensive workshops on inquiry-based learning (IBL) in college mathematics. These workshops seek to foster post-secondary mathematics instructors’ adoption of IBL, to help them adapt inquiry approaches for their classrooms, and ultimately to increase student learning and persistence in science and mathematics. Based on observed faculty needs, four strands of activity help instructors develop a mental model for an IBL classroom, adapt that model to their teaching context, develop facilitation and task-design skills, and plan an IBL mathematics course. Evaluation data from surveys and observations illustrate participant responses to the workshop and its components. The model has been robust across 15 years of workshops implemented by three generations of workshop leaders and its features make it adaptive, strategic, and practical for other faculty developers. 

Observations are widely used in research and evaluation to characterize teaching and learning activities. Because conducting observations is typically resource intensive, it is important that inferences from observation data are made confidently. While attention focuses on interrater reliability, the reliability of a single-class measure over the course of a semester receives less attention. We examined the use and limitations of observation for evaluating teaching practices, and how many observations are needed during a typical course to make confident inferences about teaching practices. We conducted two studies based on generalizability theory to calculate reliabilities given class-to-class variation in teaching over a semester. Eleven observations of class periods over the length of a semester were needed to achieve a reliable measure, many more than the one to four class periods typically observed in the literature. Findings suggest practitioners may need to devote more resources than anticipated to achieve reliable measures and comparisons.