skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


This content will become publicly available on January 1, 2026

Title: Improving the design of evaluations that include students, teachers, and schools: An empirical investigation of key design parameters. Improving the design of evaluations that include students, teachers, and schools: An empirical investigation of key design parameters
The learning environment is a complex environment for conducting evaluations. Students, teachers, and schools all play a role in shaping the environment, which can lead to differences in student achievement. Hence designing an evaluation with the capacity to detect the effectiveness of an educational intervention on student achievement requires careful planning. A key part of the planning is a statistical power analysis that considers the multilevel nature of the design. This study provides empirical estimates of design parameters necessary for planning adequately powered cluster randomized trials that include the student, teacher, and school level and are focused on reading, mathematics, or science achievement. The sample in our study includes administrative state datasets from Michigan, North Carolina, Kentucky, and Maryland for grades 3 through 8. The results showed that, with few exceptions, the variance in student test scores is larger between teachers within schools than between schools.  more » « less
Award ID(s):
2000388
PAR ID:
10623767
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
Sage
Date Published:
Journal Name:
AERA open
Volume:
11
Issue:
1
ISSN:
2332-8584
Page Range / eLocation ID:
1–20
Subject(s) / Keyword(s):
Achievement, design parameters, experimental design, hierarchical linear modeling, intraclass correlation, program evaluation, teacher research
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Science education integrates the study of and practices from the Next Generation Science Standards (NGSS). At the fundamental level, the pedagogy involves teaching and learning that emphasizes the use of scientific inquiry and the engineering design process to develop students’ problem-solving, critical thinking, and collaboration skills. Unfortunately, funding and professional development for teachers, which is essential to assure successful implementation of science lessons to increase the potential for student achievement, is lacking. Therefore, this NSF-funded science-education research project explored the development of a model that deepens the existing partnerships among grass-roots, non-profit community education organizations, K-12 public schools, and local university partners. Together, they worked collaboratively to develop systems where teachers could implement high-quality, place-based, NGSS-aligned science learning opportunities that actively engage students. This research project may lead to a future proposal for high-quality professional development for teachers, using the Teacher-to-Teacher professional development model, with the goal of impacting student achievement in science. The goals of this research project were to (1) develop a collaborative model that deepens community, public school, and university partnerships designed to support science educators and their students and (2) explore the current academic and social impact of the Teacher-to-Teacher professional development program as a possible solution for the development and implementation of high-quality, place-based, NGSS-aligned learning experiences for and with students. This presentation will focus on the components used to develop the partnership model with community partners, K-12 teachers and administrators, and university professors. Finally, the Teacher-to-Teacher (T2T) model and its new iteration, the Teacher-Plus-Community Partners T+CP Model will be shared for future development of place-based science learning experiences. 
    more » « less
  2. Engineering education has gained support in recent education reform efforts in the U.S. Although this is beneficial to students, it has become a significant challenge for teachers who do not have experience in engineering teaching, particularly in under-resourced schools. In this study, we characterized the nature of teacher educators' feedback used in an engineering design workshop. The findings showed that teacher educators frequently focused their feedback on the participants’ process, and none of them focused on personal evaluation, praise, or self-regulation – feedback that is common in classrooms. This study suggests the importance of teacher educators reflecting on their feedback during engineering design activities for effective engineering education. 
    more » « less
  3. null (Ed.)
    This qualitative study examined how a maker-based education workshop affected 20 pre-service STEM teachers’ views of the lesson planning process. Design is used as both an epistemological link between making and teaching practices as well as an analytical lens through which lesson planning could be interpreted and understood. The findings of this study suggest that pre-service teachers who have been introduced to maker-based principles and practices are able to imagine a lesson planning process that is more student-centered and active than the kind which they normally utilize. While there was a contrast between the content of making-based and traditional lesson planning processes, the pre-service teachers’ designs of these processes were largely the same: linear, verbal, and only occasionally reflective or iterative. These characteristics match those of novice designers. 
    more » « less
  4. Abstract Here, we describe the development, structure, and effectiveness of an outreach program, DrosoPHILA, that leverages the tools of our fly neurodevelopmental research program at the University of Pennsylvania to reinforce the biology curriculum in local public schools. DrosoPHILA was developed and is sustained by a continued collaboration between members of the Bashaw lab, experienced outreach educators, and teachers in the School District of Philadelphia. Since the program’s inception, we have collaborated with 18 teachers and over 2400 students. Student outcome data indicates significant positive attitude shifts around science identity and grade-appropriate knowledge gains. 
    more » « less
  5. Schools’ overt or explicit practices are a dominant lens through which education researchers and policymakers attempt to understand how schools are racially inequitable. Yet, Lewis and Diamond argue that contemporary racial inequalities are largely sustained through implicit factors, like institutional practices and structural inequalities. Ray’s framework on racialized organizations similarly outlines how our racialized sociopolitical structure becomes embedded in organizations, legitimating and perpetuating the racialized hierarchy. We apply illustrative cluster analysis techniques to rich data on schools, teachers, and students from the nationally representative High School Longitudinal Study of 2009 to find that structural inequities (e.g., student body, sector, average achievement) appear to be most salient in delineating the racialization of US high schools, whereas the characteristics of schools and teachers that are typically emphasized for closing racial inequities in educational outcomes (e.g., teacher qualifications, courses offered, stratification practices) are not salient differentiators across schools. 
    more » « less