skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Initial Steps in Developing Classroom Observation Rubrics Designed Around Instructional Practices that Support Equity and Access in Classrooms with Potential for “Success”
Background: The field of mathematics education has made progress toward generating a set of instructional practices that could support improvements in the learning opportunities made available to groups of students who historically have been underserved and marginalized. Studies that contribute to this growing body of work are often conducted in learning environments that are framed as “successful.” As a researcher who is concerned with issues of equity and who acknowledges the importance of closely attending to the quality of the mathematical activity in which students are being asked to engage, my stance on “successful learning environments” pulls from both Gutiérrez’s descriptions of what characterizes classrooms as aiming for equity and the emphasis on the importance of conceptually oriented goals for student learning that is outlined in documents like the Standards. Though as a field we are growing in our knowledge of practices that support these successful learning environments, this knowledge has not yet been reflected in many of the observational tools, rubrics, and protocols used to study these environments. In addition, there is a growing need to develop empirically grounded ways of attending to the extent to which the practices that are being outlined in research literature actually contribute to the “success” of these learning environments. Purpose: The purpose of this article is to explore one way of meeting this growing need by describing the complex work of developing a set of classroom observation rubrics (the Equity and Access Rubrics for Mathematics Instruction, EAR-MI) designed to support efforts in identifying and observing critical features of classrooms characterized as having potential for “success.” In developing the rubrics, I took as my starting place findings from an analysis that compared a set of classrooms that were characterized as demonstrating aspects of successful learning environments and a set of classrooms that were not characterized as successful. This paper not only describes the process of developing the rubrics, but also outlines some of the qualitative differences that distinguished more and less effective examples of the practices the rubrics are designed to capture. Research Design: In designing the rubrics, I engaged in multiple cycles of qualitative analyses of video data collected from a large-scale study. Specifically, I iteratively designed, tested, and revised the developing rubrics while consistently collaborating with, consulting with, and receiving feedback from different experts in the field of education. Conclusions: Although I fully acknowledge and recognize that there are several tensions and limitations of this work, I argue that developing rubrics like the EAR-MI is still worthwhile. One reason that I give for continuing these types of efforts is that it contributes to the work of breaking down forms of practice into components and identifying key aspects of specific practices that are critical for supporting student learning in ways that make potentially productive routines of action visible to and learnable by others, which may ultimately contribute to the development of more successful learning environments. I also argue that rubrics like the EAR-MI have the potential to support researchers in developing stronger evidence of the effectiveness of practices that prior research has identified as critical for marginalized students and in more accurately and concretely identifying and describing learning environments as having potential for “success.”  more » « less
Award ID(s):
1908481
PAR ID:
10424978
Author(s) / Creator(s):
Date Published:
Journal Name:
Teachers College Record: The Voice of Scholarship in Education
Volume:
124
Issue:
11
ISSN:
0161-4681
Page Range / eLocation ID:
179 to 217
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Reflection is often cited as a critical component of effective teaching, but the term itself and its related practices often remain ambiguous. Reflecting on one's teaching is an important exercise to better understand the approaches to and success towards creating inclusive classrooms. Therefore, engineering educators must become aware of reflective practices to be able to employ them in their work. We explored essential elements of highly effective reflection practices for equity-minded educators in a workshop where faculty participants learned about three reflective practices: (i) personal reflection, (ii) reflective engagement with colleagues, and (iii) reflection with students. Through collaboration with others, attendees evaluated various reflection techniques, discussed case studies, and considered supports and barriers to how purposeful reflection can support equity-minded engineering practitioners. From this workshop, a Community of Practice of faculty was formed to analyze individual reflective practices, identify practices applicable to their classrooms, and work together to employ reflection in seven classrooms across our college. In this practice paper, we evaluate each of the above reflective practices and their utility in contextualizing more equitable curricula in a variety of course types. Additionally, we provide an engineering education framework for using reflection to understand the classroom environment educators create and its impact on equitable student learning. This practice paper presents reflections from the workshop and outcomes from the Community of Practice activities to inform equity-minded reflective instruction in engineering. 
    more » « less
  2. This study examines the utilization of cognitive interviews longitudinally over a one-year period to collectively trace raters’ response processes as they interpreted and scored with observational rubrics designed to measure teaching practices that promote equity and access in elementary and middle school mathematics classrooms. We draw on four rounds of cognitive interviews (totaling 14 interviews) that involved four raters at purposeful time points spread over the year. Findings reported in this study focus on raters’ responses about one rubric, positioning students as competent. The findings point to the complexities of utilizing observational rubrics and the need to track response processes longitudinally at multiple time points during data collection in order to attend to rater calibration and the reliability and validity of resulting rubric scores. 
    more » « less
  3. In this essay, I explore some of the insights provided in a set of three manuscripts that focus on centering equity in peer review, authored by Bancroft, Ryoo and Miles, Nkrumah and Mutegi, and Marshall and Salter. I consider various aspects of their arguments, highlighting implications for the procedures and norms of journals and funding organizations and questions for further consideration. Drawing on their findings and analyses, I discuss various recommendations, such as the need to change the rules and norms of peer review to be more equitable, to ensure that reviews are free from race, ethnicity, gender, and other kinds of identity-related biases, to work towards equitable distribution of the resources, such as advising, mentoring, and valuable feedback, that support fair reviewing, and to establish criteria and rubrics that support research that is conducted in collaboration with communities marginalized in science education. In addition, I raise issues for further consideration, including the evolving relationship between “equity” and “merit” with regard to peer review and the need for centering equity in ways that allow for discussion, debate, and development of the field. 
    more » « less
  4. Minoritized and underrepresented students have historically experienced prejudice and discrimination within and outside of their classrooms, negatively impacting their educational outcomes. Research has illustrated that student academic and social success can be improved through instructors creating inclusive classroom environments that facilitate a sense of belonging. The impact of creating more inclusive environments is well-studied, however actionable guidance on how to do this, especially in more technical disciplines such as engineering, is lacking. This study aims to address this gap by developing an inclusive engineering classroom practices menu along with accompanying tools for faculty seeking to improve their classrooms. The first year of this study, as detailed at ASEE’s Annual Conference in June 2022, saw the development of the inclusive engineering classroom practices menu as well as the pilot of the inclusive learning communities for faculty across three partner institutions. The student and faculty assessment plans were also curated and involved both a student and faculty survey as well as the opportunity for students and faculty to engage in short-format interviews. This presentation will focus on the survey and interview data that has been collected in the second year of the project and the website that has been developed to further engage faculty and other institutions and partners interested in the study. This second year of this study will also see the creation of a decision matrix to aid faculty and instructors to further promote and support the implementation of inclusive practices in engineering classrooms. The continued refinement of the menu and creation of both the website and decision matrix are the next steps in the development of an inclusive classrooms toolkit that can be used across all engineering classrooms and curriculums. 
    more » « less
  5. null (Ed.)
    Today’s classrooms are remarkably different from those of yesteryear. In place of individual students responding to the teacher from neat rows of desks, one more typically finds students working in groups on projects, with a teacher circulating among groups. AI applications in learning have been slow to catch up, with most available technologies focusing on personalizing or adapting instruction to learners as isolated individuals. Meanwhile, an established science of Computer Supported Collaborative Learning has come to prominence, with clear implications for how collaborative learning could best be supported. In this contribution, I will consider how intelligence augmentation could evolve to support collaborative learning as well as three signature challenges of this work that could drive AI forward. In conceptualizing collaborative learning, Kirschner and Erkens (2013) provide a useful 3x3 framework in which there are three aspects of learning (cognitive, social and motivational), three levels (community, group/team, and individual) and three kinds of pedagogical supports (discourse-oriented, representation-oriented, and process-oriented). As they engage in this multiply complex space, teachers and learners are both learning to collaborate and collaborating to learn. Further, questions of equity arise as we consider who is able to participate and in which ways. Overall, this analysis helps us see the complexity of today’s classrooms and within this complexity, the opportunities for augmentation or “assistance to become important and even essential. An overarching design concept has emerged in the past 5 years in response to this complexity, the idea of intelligent augmentation for “orchestrating” classrooms (Dillenbourg, et al, 2013). As a metaphor, orchestration can suggest the need for a coordinated performance among many agents who are each playing different roles or voicing different ideas. Practically speaking, orchestration suggests that “intelligence augmentation” could help many smaller things go well, and in doing so, could enable the overall intention of the learning experience to succeed. Those smaller things could include helping the teacher stay aware of students or groups who need attention, supporting formation of groups or transitions from one activity to the next, facilitating productive social interactions in groups, suggesting learning resources that would support teamwork, and more. A recent panel of AI experts identified orchestration as an overarching concept that is an important focus for near-term research and development for intelligence augmentation (Roschelle, Lester & Fusco, 2020). Tackling this challenging area of collaborative learning could also be beneficial for advancing AI technologies overall. Building AI agents that better understand the social context of human activities has broad importance, as does designing AI agents that can appropriately interact within teamwork. Collaborative learning has trajectory over time, and designing AI systems that support teams not just with a short term recommendation or suggestion but in long-term developmental processes is important. Further, classrooms that are engaged in collaborative learning could become very interesting hybrid environments, with multiple human and AI agents present at once and addressing dual outcome goals of learning to collaborate and collaborating to learn; addressing a hybrid environment like this could lead to developing AI systems that more robustly help many types of realistic human activity. In conclusion, the opportunity to make a societal impact by attending to collaborative learning, the availability of growing science of computer-supported collaborative learning and the need to push new boundaries in AI together suggest collaborative learning as a challenge worth tackling in coming years. 
    more » « less