skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Using global observation protocols to inform research on teaching effectiveness and school improvement: Strengths and emerging limitations
Title: Using global observation protocols to inform research on teaching effectiveness and school improvement: Strengths and emerging limitations
An essential feature of many modern teacher observation protocols is their “global” approach to measuring instruction. Global protocols provide a summary evaluation of multiple domains of instruction from observers’ overall review of classroom processes.  Although these protocols have demonstrated strengths, including their comprehensiveness and advanced state of development, in this analysis we argue that global protocols also have inherent limitations affecting both research use and applied school improvement efforts.  Analyzing the Measures of Effective Teaching study data, we interrogate a set of five potential limitations of global protocols.  We conclude by discussing fine-grained measures of instruction, including tools that rely on automated methods of observation, as an alternative with the potential to overcome many of the fundamental limitations of global protocols.  more » « less
Award ID(s):
1735785
PAR ID:
10180388
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
education policy analysis archives
Volume:
28
ISSN:
1068-2341
Page Range / eLocation ID:
62
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, The Journal of experiential education)
    In this study, we report results from a novel coding of the Measures of Effective Teaching (MET) Study data that offers evidence on a set of teacher discourse measures in the domain of teacher support including: public praise vs. admonishment, autonomy support vs. controlling language, strategy suggestion vs. lack thereof, and discourse supporting (vs.undermining) learning mindsets. Novel coding of these constructs is paired with extant measures of instruction and achievement in the MET data. Several of the newly coded discourse measures have promising features, including high lesson- and teacher-level variability, and convergent and discriminant validity with existing protocols. We also report possible associations with change in achievement over two years. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, Tree Physiology)
    Pfautsch, Sebastian (Ed.)
    Given the pressing challenges posed by climate change, it is crucial to develop a deeper understanding of the impacts of escalating drought and heat stress on terrestrial ecosystems and the vital services they offer. Soil and plant water potential play a pivotal role in governing the dynamics of water within ecosystems and exert direct control over plant function and mortality risk during periods of ecological stress. However, existing observations of water potential suffer from significant limitations, including their sporadic and discontinuous nature, inconsistent representation of relevant spatio-temporal scales and numerous methodological challenges. These limitations hinder the comprehensive and synthetic research needed to enhance our conceptual understanding and predictive models of plant function and survival under limited moisture availability. In this article, we present PSInet (PSI—for the Greek letter Ψ used to denote water potential), a novel collaborative network of researchers and data, designed to bridge the current critical information gap in water potential data. The primary objectives of PSInet are as follows. (i) Establishing the first openly accessible global database for time series of plant and soil water potential measurements, while providing important linkages with other relevant observation networks. (ii) Fostering an inclusive and diverse collaborative environment for all scientists studying water potential in various stages of their careers. (iii) Standardizing methodologies, processing and interpretation of water potential data through the engagement of a global community of scientists, facilitated by the dissemination of standardized protocols, best practices and early career training opportunities. (iv) Facilitating the use of the PSInet database for synthesizing knowledge and addressing prominent gaps in our understanding of plants’ physiological responses to various environmental stressors. The PSInet initiative is integral to meeting the fundamental research challenge of discerning which plant species will thrive and which will be vulnerable in a world undergoing rapid warming and increasing aridification. 
    more » « less
  3. ; ; (, The Journal of the Acoustical Society of America)
    Intelligibility measures, which assess the number of words or phonemes a listener correctly transcribes or repeats, are commonly used metrics for speech perception research. While these measures have many benefits for researchers, they also come with a number of limitations. By pointing out the strengths and limitations of this approach, including how it fails to capture aspects of perception such as listening effort, this article argues that the role of intelligibility measures must be reconsidered in fields such as linguistics, communication disorders, and psychology. Recommendations for future work in this area are presented. 
    more » « less
  4. ; ; ; ; ; (, ASEE Annual Conference Proceedings)
    Over the past few years, education at all levels has been greatly disrupted by the COVID pandemic. For many schools, face-to-face interactions were reduced or restricted to only those activities that have been determined to be essential for student instruction. The pandemic has also had a great impact on teacher professional development programming, which traditionally has been delivered largely in face-to-face settings. This paper examines the implementation of a series of energy technology teacher professional development workshops that were delivered virtually using online meeting apps. The results of the workshops are presented including participant measures of learning gains, and feedback describing how faculty participants used the information gained to modify their curriculum and instruction. The discussion includes observations and recommended practices to promote the effective incorporation of tools and equipment for remotely delivered workshops. 
    more » « less
  5. ; ; (, International Journal of STEM Education)
    Abstract BackgroundNumerous studies show that active and engaging classrooms help students learn and persist in college, but adoption of new teaching practices has been slow. Professional development programs encourage instructors to implement new teaching methods and change the status quo in STEM undergraduate teaching, and structured observations of classrooms can be used in multiple ways to describe and assess this instruction. We addressed the challenge of measuring instructional change with observational protocols, data that often do not lend themselves easily to statistical comparisons. Challenges using observational data in comparative research designs include lack of descriptive utility for holistic measures and problems related to construct representation, non-normal distributions and Type-I error inflation for segmented measures. ResultsWe grouped 790 mathematics classes from 74 instructors using Latent Profile Analysis (a statistical clustering technique) and found four reliable categories of classes. Based on this grouping we proposed a simple proportional measure we called Proportion Non-Didactic Lecture (PND). The measure aggregated the proportions of interactive to lecture classes for each instructor. We tested the PND and a measure derived from the Reformed Teaching Observation Protocol (RTOP) with data from a professional development study. The PND worked in simple hypothesis tests but lacked some statistical power due to possible ceiling effects. However, the PND provided effective descriptions of changes in instructional approaches from pre to post. In tandem with examining the proportional measure, we also examined the RTOP-Sum, an existing outcome measure used in comparison studies. The measure is based on the aggregated items in a holistic observational protocol. As an aggregate measure we found it to be highly reliable, correlated highly with the PND, and had more statistical power than the PND. However, the RTOP measure did not provide the thick descriptions of teaching afforded by the PND. ConclusionsFindings suggest that useful dependent measures can be derived from both segmented and holistic observational measures. Both have strengths and weaknesses: measures from segmented data are best at describing changes in teaching, while measures derived from the RTOP have more statistical power. Determining the validity of these measures is important for future use of observational data in comparative studies. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email