How Can Educators Exchange High-Quality Feedback with Various Forms of Evidence of Practice?: An Analysis of Afterschool Educator Professional Growth Through Noticing

investment is compounded by unique challenges facing afterschool education regarding high professional turnover (Afterschool Alliance, 2022). Thus, STEM-focused PL for afterschool educators is not only difficult to offer but also a speculative investment given the extent of staff turnover. Yet, as afterschool settings are unique among U.S. out-ofschool time (OST) programs in serving youth and families identifying with racial, socioeconomic, and linguistic groups systemically marginalized in STEM (Afterschool Alliance, 2021), it is important to support afterschool educators to develop inclusive STEM facilitation skills.

Since 2018, After school Coaching for Reflective Educators in STEM (ACRES) has addressed this need by supporting afterschool educators nationally in developing STEM facilitation skills. Unique among other PL for OST educators, ACRES instills fundamental principles of STEM facilitation, rather than providing singular examples of specific STEM activities (Brasili & Allen, 2019). This focus is intentional so that educators can easily apply the skills they develop to whatever STEM activities suit their programs. ACRES was designed to meet the needs and capitalize on the affordances of OST STEM learning programs-especially afterschool-in geographic areas where training for informal educators is especially inaccessible, such as rural communities or urban centers (Cook et al., 2021).

A crucial component of ACRES's PL model is using participants' videos of their practice with program youth to stimulate small-group reflection on a focal STEM facilitation skill. However, coaches have found that many circumstances render this request to provide a video of STEM facilitation with program youth unrealistic for educators. Some attend the PL program while their programs are not in session, others are new to their career and do not yet have youth-facing experience, and still others encounter restrictions against video recording program youth. In such cases, educators contribute other forms of evidence to receive peer feedback, such as videos of STEM facilitation with family or audio-only recordings with program youth. Given that ACRES was designed to build professional growth through noticing evidence of STEM facilitation in peers' videos, coaches wondered how these variations still supported the focal skills of the PL and how they could leverage this diversity to enrich their coaching. This need drives the motivation of our study as we consider the extent to which these diverse forms of evidence can be capitalized upon as valuable opportunities for peers to notice and reflect on the use of focal skills. We explore two research questions (RQs):

(1) How does engaging in noticing using diverse forms of evidence of practice in a PL community support OST educators' awareness of STEM focal facilitation practices? (2) How does the form of evidence that educators provide in PL sessions (e.g., video of practice in their programs, video of their practice with family) shape the aspects of STEM facilitation that peers notice and respond to?

These research questions respond to a need to understand how OST educators' professional growth can be supported by both (1) responding to existing literature on the value of noticing through evidence of practice and (2) acknowledging the realities of OST education that necessitates creative variations of typical exercises of sharing in PL for classroom educators. RQ2 particularly supports PL providers in identifying strategies to support educators in developing STEM facilitation skills grounded in their own experiences by identifying and capitalizing on assets of diverse forms of evidence that may be introduced in PL communities.

Support for video-based evidence of practice for educator re&ection

As noted above, ACRES coaches ask that educators bring video evidence of practice to ACRES sessions (see Program Background, below). This request for video evidence is based on Clarke and Hollingsworth's (2002) suggestion that video-based reflection a) supports educators' awareness of classroom activity that may be difficult to notice "in the moment;" b) creates possibilities for viewing unique educational contexts for critical reflection on personal choices; and c) creates a shared entry-point for conversations within a community of practice. Recent work (Buchbinder et al., 2021) further tests the capabilities of videobased reflection, exploring how 360-degree video of teaching can stimulate novel teacher "noticing" by providing multiple viewpoints for examining practice. Such work builds on the affordances of video-based PL, illustrating how the qualities of the video can open or restrict access points for reflection. This complements other work suggesting reflection using personal videos, as opposed to videos of others' practice, yields a higher sense of connection with the practice in the video (Seidel et al., 2011), and may be more beneficial for reflection (Zhang et al., 2011). However, in comparing personal videos, videos of peers, and published videos as shared material in professional development for elementary teachers, Zhang et al. (2011) noted that personal videos created some constraints to reflection personally or in peer groups, including technical limitations of freely available equipment, difficulties in encouraging learners to behave naturally in front of the camera, and awareness that peers may refrain from providing overly critical feedback. These incompatible findings illustrate a need for a context-specific study of the relative affordances and limitations of forms of evidence of practice in PL conversations. While such questions have been explored in formal education, understanding the use of evidence of practice-and the benefits of variations of evidence-has not been studied in PL for afterschool educators. The transferability of existing studies to afterschool is suspect, considering that afterschool educators do not have schedules that provide easy access to youth throughout the calendar year (e.g., educators in summer programs), and often the educators themselves come to the profession with backgrounds that have not included formal training in youth activity facilitation or STEM education (Brasili & Allen, 2019). Furthermore, the institutional structure complicates video permissions in contrast to the K-12 context where caregivers may annually opt-in to photo/video releases. Moreover, much of the existing research on outcomes associated with using videos in PL was published before COVID-19 (Sablij et al., 2021). Since then, educators have had far more encounters with themselves on video. Understanding the crucial characteristics of evidence of practice may guide PL providers serving the diverse landscape of afterschool education in defining what sources of evidence they ask participating educators to contribute to a virtually facilitated PL program.

ACRES program structure

Participating in an ACRES module entails attending three virtual, synchronous training sessions with a small cohort of approximately three to five peers (see overview in Figure 1). In the first session, coaches 1) introduce educators to the STEM facilitation skill (e.g., APQ); 2) create space for educators to practice noticing and offering feedback on the skill using samples of practice from video archivesfoot_0 ; and 3) lead an activity in which educators experience the skill as learners through collaborative, hands-on activities with other participants in their cohort.

Given the inherent challenge in hands-on collaboration with virtual colleagues, ACRES program organizers thoughtfully developed activities for the third component of the first session to allow all participants to engage using readily available household materials. The most commonly used activity in the APQ module is building a water filter. Program organizers selected this activity, adapted from Click2Science (n.d.), for its inclusivity across a) materials available to participants, b) participants' experience with STEM and STEM facilitation, and c) background knowledge participants can contribute to a discussion of pedagogical and real-world applications. In welcome e-mails that coaches send before the session, they ask participants to gather materials to make dirty water and a filter. In the session, participants make their "dirty water" and then strategize their filter construction in breakout rooms. The coach then models the APQ skill in the whole group as the groups attempt to filter their water and share observations.

Before the second and third sessions, coaches ask educators to create short, 3--5-min video recordings of their practice of APQ STEM facilitation with youth. Educators are encouraged to practice the skill in the context of their afterschool programs with whatever content or activities make sense for their audience; they are not assigned a specific activity or content focus beyond the request that it is STEMrelated. For educators who express they are brand new to STEM facilitation or are nervous about selecting an activity, the coach invites them to adopt the water filter activity. As noted above, while the request is to contribute a video of STEM facilitation with program youth, coaches encourage educators to bring variant forms of evidence of practice if needed. Coaches do not mandate a specific alternative, though they may offer strategies that other educators have taken, such as filming with family members.

Peers and coaches discuss evidence of practice in the synchronous sessions using a sharing order designated by the coach. At their turn in the sharing order, educators first introduce the context of the activity to orient peers-for instance, describing the learners in the video and the STEM content of the activity. After peers view or listen to the evidence of practice, they take turns-according to a speaking order given by the coach-suggesting "strengths" and then "opportunities for growth" related to APQ. The coach also offers feedback, usually after the peer educators share their ideas. Coaches introduce the language of "strengths" and "opportunities for growth" to structure this feedback exchange and remind educators to use it. The coach then asks the educator to make a summative reflection of the feedback they heard.

Derivation of the ACRES program model

The ACRES model is rooted in Clarke and Hollingsworth's (2002) articulation of "professional growth." This model articulates how educator growth moves of "enactment" and "reflection" mediate experiences across domains of external experience (e.g., the PL), practice (e.g., the classroom environment with learners), consequence (e.g., the observed learning outcomes), and personal awareness (e.g., knowledge of, belief in, and/or attitudes toward the practice). Clarke et al. (2013) specifically point to the role video can play in this model by "facilitating both teacher reflection and teacher action" (p. 94). For instance, as illustrated in Figure 2, an educator may record the practice of a new STEM facilitation skill, such as APQ, with learners. Then, they review the video to reflect on learner outcomes associated with the skill. This will lead to a) the educator reflecting on their beliefs about the APQ skill to support STEM learning generally and b) the enactment of practice based on those reflections. These reflections, along with the video itself, may be brought back to the learning community to be discussed, leading to further enactment of practice built from peer conversations.

Theoretical framework

Our analysis is grounded in teacher noticing (van Es & Sherin, 2002, 2008). Noticing acknowledges the social and contextual complexities of teaching activity in which teachers 1) attend to some phenomenon in the learning environment, 2) draw from their training to think about the phenomenon, and 3) apply their training to the specific phenomenon at hand (ibid.) Noticing aligns with the professional growth model that undergirds ACRES, particularly in the interplay of enactment and reflection across the professional growth process. In ACRES, this process unfolds as educators interact with a cohort of peers who share reflections on one another's enactment of STEM facilitation skills. As described above, coaches scaffold educator peers to engage in noticing as they 1) attend to the use of APQ in the shared evidence of practice; 2) identify skills in APQ during STEM facilitation that the coach shared in the first session; and 3) connect their observations of their peer's practice to the training-leveraging this connection to offer feedback on strengths and opportunities for growth. Research has explored noticing in teacher education candidates (e.g., Benedict-Chambers & Aram, 2017;Johnson & Mawyer, 2019) and in-service teachers (e.g., Criswell et al., 2022;Krist et al., 2023). However, we were unable to find research exploring noticing with afterschool educators. One exception (Francisco & Maher, 2011) reports on a program from over 10 years ago and explores the noticing of elementary and middle grades teachers who interned in an afterschool program-thus possessing the background of teacher education atypical among afterschool educators. Research in formal education contexts illustrates how video reflection can be a useful tool in both documenting and shifting noticing. In M. J. Luna and Sherin (2017), elementary mathematics teachers who participated in a video club shifted what they attended to in watching and discussing videos of each others' practice. Early reflections concentrated on learners and classroom climate and were superficially descriptive, while later reflections were more cognizant of educator activity and mathematical thinking and interpreted rather than simply described what unfolded. The authors suggested engagement with the video club supported shifts in what teachers noticed in ways that aligned with more awareness of their actions and specifically supported mathematical learning. In Zummo et al. (2022), early-career secondary science teachers used video annotation tools (VATs) to reflect on "moments where you are satisfied with your instructional decisions in relation to your goals and moments where you see room for growth" (p. 748). Researchers analyzed the VATs to explore the relationships between noticing of learners' thinking, teachers' practice, and science content. Indeed, research such as this grounded the development of the ACRES model that recognizes the importance of video of practice for feedback exchange on STEM facilitation skills, as video provides opportunities to notice the actions of learners, the actions of educators, and the use of pedagogy and science content.

While this prior work illustrates the potential of video evidence to support noticing, other researchers have explored how manipulating evidence sources can elicit different noticing attention. Lam and Chan (2020) interviewed preservice science teachers about what they noticed in short, produced videos. While all examples of practice were the same media type, educators' noticing differed according to the particularities of the video content, for instance, with more attention to a learner response in a familiar classroom situation compared to an unfamiliar one. In a deviation from video evidence, M. Luna and Selmer (2021) uniquely focus on teachers' noticing of learner thinking elicited using artifacts learners produce. The authors discuss how artifacts create a noticing context that can be uniquely formative and inform teachers' awareness of particular pedagogical responses they need to make to support learners' developing thinking. This research collectively illustrates that qualities of the source educators are prompted to "notice" may lend themselves to draw attention to certain aspects of learner engagement. Such insights resonated with our experience in ACRES, as our national coaches have shared with us anecdotal evidence of cohort educators contributing unique evidence of their practice diverging from the model grounded in video evidence-especially as our program scaled up to include coaches and educators across unique contexts. This inspired the present study to explore how educator peers attend to evidence of APQ during STEM facilitation, connect the practice with skill learning, and draw connections between the learning and practice across the diverse forms of evidence of practice we see frequently shared in cohorts instead of videos with program youth.

Participants

Our analysis includes the observations of six cohorts led by two of ACRES's most experienced coaches, Coach B and Coach E. They have each led more cohorts than any other current coach (i.e., over 20 unique cohorts) and have coached multiple ACRES module types. They are staff members for ACRES's parent organization, the Maine Mathematics and Science Alliance (MMSA), and attend regular, weekly meetings with ACRES researchers and program organizers. We included cohorts if two or more educators participated in all sessions, allowing us to observe feedback exchanged among peers. During the registration process, participants were prompted-separately-to consent to a) audio and video release and b) to participate in research. We only included in this study cohorts where all participants consented to both options. We did not compensate participants for allowing their videos to be used in research. Our analysis includes six unique cohorts with 28 total educators, ranging from two to eight educators in each cohort. Educators' reported years of experience in OST ranged from 0 to 24 years, and their reported experience facilitating STEM activities for youth ranged between 0 and 17 years.

Data gathering and organization

Our primary analysis relies on observational data from sessions 2 and 3-when educators bring evidence of their STEM facilitation practice for feedback. Because sessions are held virtually and are restricted to access by educators and their coaches, we rely on recorded videos of the 2-h Zoom sessions. We uploaded videos to Otter.ai for transcription. One researcher reviewed the transcript alongside the audio and video to clarify mistranscribed words, attribute speakers, and add nonverbal communication (e.g., nodding).

In our analysis, we attended to portions of the transcript occurring during the feedback components of the session-i.e., where educators discussed their peers' STEM facilitation practice by offering noticings of "strengths" and "opportunities for growth." We did not review the transcripts of evidence educators provided directly-instead using the feedback that was exchanged in response to the evidence as our primary data. We consulted transcripts of STEM facilitation practice itself to clarify the context of the activity that educators shared. While we recognize the value of these data as a primary data source, we are exploring them in a separate inquiry and consider their inclusion beyond the scope of the present manuscript.

Data sense-making

Our data sensemaking is summarized in the timeline in Figure 3. We directed our sensemaking to a) understanding the objects of noticing that each evidence form afforded, as suggested by educators' feedback to one another on "strengths" and "opportunities for growth" in practicing APQ, and b) comparing the noticing expressed by educators with the APQ module's learning objectives to understand the extent to which diverse forms of evidence aligned with program objectives. Our analysis began after we had collected the recordings of all cohorts. We began exploring the data using emergent coding (Miles et al., 2019) to ground our understanding in the language of what peer educators noticed about samples of practice shared by their peers (Charmaz, 2017). In other words, we considered their feedback as indications of what they attend to in the learning environment and how they apply their training to understand and further their learning about APQ during STEM facilitation (van Es & Sherin, 2002, 2008). Because educators were coached to comment on their peers' actions that they noticed by offering "strengths" and "opportunities for growth," we used gerund coding (Miles et al., 2019).

To leverage the unique perspectives both authors contributed to the inquiry, we collaborated on coding rather than seeking consensus (O'Connor & Joffe, 2020). Once we completed coding, we collaboratively identified connections between the codes through axial, or pattern coding (Miles et al., 2019;Strauss & Corbin, 1990). Miles et al. (2019) describe pattern coding as a "second cycle method" that can identify a "bigger picture" by collecting first cycle codes (e.g., our gerund codes) into "a smaller number of categories, themes, or concepts " (Miles et al. (2019, p. 79, emphasis in original). Since the overarching goal of the APQ module is for educators to recognize and act on opportunities to ask purposeful questions during STEM facilitation, we considered these axial codes as tentative categories of noticing outcomes of participating in the APQ sessions (i.e., what educators become aware of in the enactment of questioning practices during STEM facilitation)aligning with RQ1.

We shared the list of gerund codes and inductively derived focal skills (i.e., axial codes) with Coach B, Coach E, and other coaches on the ACRES team to provide feedback on a) the extent the actions described in the codes are skills of APQ communicated during sessions; b) whether any actions reflected in the codes misaligned with their objectives in the APQ module, and c) the relative significance they attribute to each of the outcomes we proposed with our axial codes. We reviewed this feedback to further develop our findings and their interpretation by contextualizing these responses within our data. For instance, the coach's feedback may indicate an outcome we identified was not directly related to their APQ objectives. Thus, we could differentiate between skills of APQ that are directly coached and those that coaches perceive as indirectly communicated in their presentation of APQ (RQ1). This allows us to explore how different forms of evidence support explicit objectives. Table 1 indicates our gerund codes, axial codes, and summary of coach feedback. We include the coach's feedback in Table 1 to indicate the alignment of each outcome with the stated objectives of APQ (RQ1). To develop our findings and address the purpose of evaluating evidence sources across cohorts (i.e., RQ 2), we examined code lists for each outcome disaggregated by evidence sources we saw represented in the data (see Table 2 for a list of the evidence sources and their descriptions). As noted above, educators themselves decided on these forms of evidence of practice based on what made sense for their current context. For instance, The educator submits a recorded video of facilitating STEM practices in their organization's learning setting.

Researcher positionality

Heidi taught high-school science for 7 years. During this time, she recalls being complimented by a supervisor on her questioning strategies. This recognition motivated her to continue to attend to questioning in the classroom and notice how her questioning promoted learning. Since beginning her research career, Heidi has been personally and professionally interested in how "science talk" between youth and adults builds science identity for both educators and learners. She especially attunes how interaction contexts relate to talking behavior between individuals. In reviewing video recordings of ACRES programs, this lens guided Heidi to notice the different types of evidence educators brought and the ways the cohort conversations seemed to be responsive to those different sources of evidence.

Kate is a versatile researcher with a background in journalism and a Master of Fine Arts Degree in Creative Writing. Her research passionately delves into the intersection of communication, storytelling, and science, mainly focusing on elevating participant voices. Her research often centers on rural communities and accessible STEM opportunities, with a special emphasis on afterschool educators. In analyzing the videos for the project, her unique background provided a lens through which she could identify nuanced attributes about how participants provided feedback to each other.

Outcome 1: eliciting desired youth outcomes

As noted in Table 1, ACRES coaches found the activities we associated with this outcome (e.g., "encouraging youth's deeper thinking) to align well with their coaching goals. Unsurprisingly, then, educators attended to this outcome across all of the forms of evidence shared. However, educators attuned their feedback to the opportunities for noticing each form of evidence afforded. For instance, peers occasionally said videos of authentic practice 2 did not clearly convey youths' utterances. This obfuscation often occurred because educators-serving the dual role of "educator" and "videographer"--did not carry a camera with them as they circulated to learner workgroups-instead leaving their camera in the corner of the room while they worked. As such, peers were not able to notice youth verbal responses and instead attended to body language. For example, here Educator Jfoot_2 commented on visual indicators of youth involvement and comfort in a peer's video showing youth exploring a robotics program: I love how she actually got on the floor . . . and that she was willing to be at their level more and make them comfortable. It's hard, too because it [the video] skipped a couple of times, but the kids seemed more involved this time and comfortable with her being lower.

Across evidence sources, many educators referenced practices that broadened the participation of all youth in the activity. In the coach-provided videos, educators pointed to specific opportunities they saw in the video where the filmed educator could involve more learners. For instance, in a video where youth were building cars, Educator E noticed:

One student used an eraser as a brake. And rather than telling the kids around the room what that was for it would have been really helpful for her to ask. And I think that the students might then have been able to consider different ideas and conversations. That might be helpful then to that educator to kind of see where they were thinking.

Here, Educator E provided feedback based on what was visibly available in the video. With other evidence sources, however, the suggestions for group engagement tended instead to project to hypothetical scenarios or ideas for "next time." This was especially true for contrived video since the video only included one or two learners. For instance, Educator N recommended to a peer who shared a video of a water tension activity with a single youth that, "for a large group, I think this experiment is really good for having them talk to each other a lot. And asking each other questions." While Educator N evoked the STEM facilitation activity of encouraging youth discussion amongst themselves, the suggestion was based more on what she noticed about the water tension activity and its potential than on the behavior of the youth himself, given that the context in the video did not lend itself to youth-to-youth interactions. Educator N's suggestion was offered only after nudging from Coach B when Educator N said they could not identify opportunities for growth "when you're talking with just one [learner]."

In contrast, in audio of authentic practice, peers keyed on phrases they heard, such as in this audio clip in which a peer was facilitating a chromatography activity:

It was very evident that it wasn't just you standing up there, that you're having them involved in that process, and giving them ways to think about different things that were connected. I love when people say, "Tell me about that.". . .So I thought it was really great how you incorporated those purposeful questions into their reactions.

Here, Educator C pieced together a mental picture of the classroom based on what they noticed auditorily, filling in visual gaps with their familiarity with OST learning dynamics. In this way, they connected their noticing not only to the instruction of the cohort but also to their experiential knowledge as an educator. This phenomenon is similar to the moves peers took with video of authentic practice accompanied by poor audio quality-they noticed the evidence of APQ afforded by the snapshot of practice and surmised what was not discernible by building on their familiarity with their peers' learning contexts.

Outcome 2: attending to youth contributions

Actions associated with Outcome 2 concerned educators' responses to contributions youth offer. As noted in Table 1, Coaches B and E considered this outcome, along with Outcome 1, as a focal objective in their coaching of APQ-meaning that we expected this outcome to be well realized in educators' noticing.

While we saw evidence of practices within Outcome 2 across evidence sources, we particularly noticed differences between contrived video (e.g., video facilitating a STEM activity with a spouse) and evidence that included program youth (i.e., coach-provided video, video of authentic practice, and audio of authentic practice). In contrived video, "opportunities" peers offered referenced likely differences in how youth in a program may respond compared to the familiar learner. For instance, Educator O advised a peerwhose video engaged their spouse in a potential energy activity-to consider "maybe even waiting a bit before saying 'that's a great observation' and see if anything else comes up. Especially with kids [it] might take a second." In contrast, Educator M's suggestion to a peer's video of authentic practice built on what Educator M had the opportunity to see during a chromatography activity with program youth. They advised "Taking a pause and asking them to think about what they see." because "they're so eager, and you want to ride that wave of eagerness. But I think that having them slow and pause could have some benefits." While in both instances the advice is to provide space for learners' thinking, in the authentic video, Educator M noticed in the video (i.e., "wave of eagerness") the socialemotional learning (SEL) effects of how the educator responded to youth contributions. In contrast, in the contrived video, the suggestion draws from the noticing of a lack of wait time but without the coincidental awareness of the expected consequence of its absence in the learner.

While the importance of "wait time" appeared in noticings across evidence sources, its mention in the contrived video was usually accompanied by suggestions the evidence offered little to improve upon and even self-criticism from the educator offering the feedback of their own capabilities. For instance, Educator T offered wait time as an opportunity while clearly grasping for suggestions and taking the appearance of flawlessness in their peer's contrived video with their spouse as a comparative judgment on Educator T's own performance.

I guess the only thing I would think of was just wait time because I thought it was excellent . . . It makes me want to do better next time because I'm thinking, "oh, when mine comes this is gonna be a joke."

Evidence that included learning contexts with program youth (i.e., video or audio of authentic practice, coach-provided video) provided more in-depth follow-up questions for peers to note and make connections to their learning about APQ. In coach-provided videos, peers noticed questions about the scientific process. In responding to a video of an educator engaged with their students in exploring properties of ice, Educator R noted "I really liked when she asked clarification questions like, 'what do you mean by effect?'." The follow-up questions in authentic practice also attended to SEL objectives, which we did not observe in contrived practice or coach-provided video. For instance, Educator J noticed how questioning during an engineering activity pointed to awareness of both the presence of questions supporting youth thinking and the absence of questions that would conflict with SEL objectives: I like how she didn't say that his design was not going to work. She was very encouraging like, "Well, why won't this happen? Why is this one gonna work and this one won't?" While we observed this kind of attention in both video and audio of authentic practice, peers who viewed video evidence also commented on the effectiveness of follow-up questioning to bring about visible responses from youth. For instance, Educator O noticed in a peer's facilitation of an engineering task that "those follow-up questions really got her [the student] thinking so much that she changed her mind, in the minute, and that was cool to see on video."

Outcome 3: goal-supported classroom

Peers also commented on the extent the evidence they observed leveraged APQ to align with the goals of the classroom. While Coaches B and E indicated that this use of questioning was not a focal point of their modules, they expected educators would recognize this outcome as they incorporated APQ into their STEM facilitation with youth. Educators' noticing aligning with this outcome was most frequently demonstrated in sharing descriptions of planned or reflected practice, such as lesson plans-formats that were not originally intended vehicles for exchanging feedback within the ACRES model. We suggest the preponderance of attention to this outcome within these sources occurred because the educator described a complete activity rather than a snapshot of practice-structuring their description in a full-lesson narrative for sharing with their peers. This structure allowed peers to comment on the full organization of the activity in ways that were not as conducive in other sharing contexts. However, this feedback often deviated from the learning goal of APQ, as educator's attention was diverted to the activities and their organization rather than the facilitation moves supporting STEM engagement. For instance, Educator S responded to a peer's lesson summary in testing water leakage when a bag was punctured by commenting, "I think there should be a stage at the beginning where you present the concept and you have the kids make a hypothesis."

As in the previously discussed outcomes, feedback on contrived video addressed the transferability "with kids," especially in terms of, seemingly, perfectly executed plans. Educator O wondered of a peer's activity with their spouse, "It felt like your questions were very prepared . . . Working with actual kids, [you] might not have the questions right there like your husband did." While Educator O noticed the strength in learner engagement supporting the classroom goal, they also recognized the contextual realities of the shared sample of practice may have predisposed this outcome.

Coach-provided videos are unique among evidence sources in that the educator being viewed is not present in the cohort to provide context on the activity. Thus, many comments, particularly on opportunities, to this evidence source included a caveat that the suggestion the participant offered may have been addressed but was not visible. Or, participants reasoned the video may have appeared especially strong because the editing (e.g., splicing together clips) provided a full narrative of what took place in the classroom.

Noting this feature, educators often commented on the strength they noticed in the example educator's ability to use questioning to connect the activity to larger goals. Educator R commented on an ice melting experiment: I like how she checked in to be like, "so how are you going to be measuring the changes in your ice throughout this experiment? And how are we quantifying this?" So they weren't just doing the experiment. They knew what they were looking for.

Uniquely, in videos of authentic practice, educators seemed to have more material to notice and build upon conveying the realities of afterschool learning and how APQ can contribute to larger learning goals. For instance, Educator B related to his peer's context by contrasting what is presented as "ideal" in training videos to the realities of afterschool STEM facilitation:

It's tricky because there's not always room to have another activity happening, especially if you're the only one trying to run the main activity. So I think somehow, some way . . . keeping the other ones who aren't participating busy, so that the ones that want to do the STEM project, or the main project, can actually do it . . . But that's a mystery that I still cannot solve . . . because a lot of the videos, especially in trainings, they're very orchestrated, everyone's doing the project. And I'm like, "but that's not how it happened." However, as in the excerpt above, feedback in response to this kind of context often deviated from the APQ objectives to build skills in STEM facilitation through purposeful questioning, digressing into sharing ideas about how to respond to youth needs noticeable in their visible behaviors. Tentatively, we noticed a stronger alignment of these objectives in instances of audio of authentic practice. For instance, Educator M said their peer was "developing the lesson" with their learners and "asking them to think about different opportunities. So even though you were leading them, you were also asking open-ended questions along the way." Although youth voices were frequently inaudible, the proximity of the recording device to the educator enabled it to pick up on a series of questions the educator used to guide youth into a goal-aligned activity.

Outcome 4: knowledge of learners

Coaches B and E expected that demonstrating and utilizing knowledge of youth would emerge from the practice of APQ, as educators realized how knowledge of students can inform effective learner-centered questioning approaches driving STEM engagement. As indicated in the prior three Outcomes, we often saw that educators drew upon their own knowledge of learners, generally and in their programs specifically, to build bridges between APQ and phenomena they could notice in their peers' practice. However, we also saw instances across evidence sources participants noticing educators demonstrating, or having the opportunity to demonstrate, knowledge of learners. In these cases, educators often grounded their feedback from their own experience, pairing their experience with the questioning frame of APQ presented in the learning module. Specifically, participants provided feedback to their peers about the extent to which interactions with youth, as evidenced by their sample of practice, reflected awareness of learners' intellectual and developmental needs and contexts. In experiencing contrived video, peers again attended to differences in facilitating STEM activities with family or other familiar learners compared to with youth in a program. In doing so, they drew from developmental knowledge acquired from their experience and connected that with the degree questioning was shown to be effective according to the evidence shared. For instance, Educator J reflected on a video her peer shared building a water filter with an older son:

Because it was her son, she was comfortable with him. But if you're [aged] five, four or even older, you're not going to understand it as much just because there were kind of very, very open-ended questions, or they were very vague questions.

We expected participants would less readily suggest their peers did not effectively incorporate knowledge of learners in their authentic interactions because we suspected this type of critique would carry a heftier (and negative) social consequence. Thus, we anticipated we would see more evidence of this kind of noticing in coach-provided videos, such as in commentary from Educator R, who keyed in on the appropriateness of a question she heard to support inclusivity of all learners. Here, the educator is trying to spark youths' interest in robots:

When he asked "who has a bike?," instead of asking something like that, which, for the kids can then help them indicate who doesn't or does have something, maybe be like, "All right, we've seen a tire on things like bikes, cars, buses. How do they work?" Or, "what's the material on those?." Because that's something that I feel like more kids could just be like, "Yeah, I've seen something like that within my community, even if I don't have it."

Peers noticed in their fellow cohort members' practice instances where educators' knowledge of learners was a strength. Educator B attended to the reactions of learners observed in a video of authentic practice when their engineering approach failed to yield desired results:

[The youth] didn't feel bad. They're just like, "Oh, it broke. Let's try again." Which is good to see. Because a lot of times, at least the kids I have here, they're . . . prone to the feelings of failure. And if it doesn't work the first time, they're usually just like, "Nah, I'm done.". . .But it looks like her questions were also like, "Okay, well, it didn't work, but we can try it again." And they had motivation to want to do it again.

Educators responded to audio of authentic practice similarly-integrating what they knew about youth and had learned from the APQ module to reflect on their peers' strengths in attending to unique youth capabilities. This noticing was demonstrated in how educators attended to notable verbalizations. Educator H, for instance, was struck by an instance when her peer asked a youth "can you describe 'weird'?" during an activity learning about neurons:

I think one of my favorite parts was can you describe "weird," because you're validating their thinking and also pushing them. Kids get stuck, I think with a little bit of limited vocabulary, but to push them to elaborate on their thinking is a really strong opportunity.

When participants provided descriptions of planned or reflected practice, feedback centered on ways background knowledge can inform planning. Educator H suggested "thinking ahead to, "what are those background knowledge questions that you could ask?" And having a list and thinking maybe where you'll go depending on the kids' response." Peers who provided feedback on this type of evidence often acknowledged they were filling in some evidence gaps with their own experience, such as offering "it's kind of hard to do this without seeing a a video" when a peer described a learning activity their learners found frustrating and offered "things that I've noticed can help reduce frustrations."

Discussion & recommendations

The present work contributes to the research base on how evidence of practice may be leveraged in educator PL to build awareness of skills through noticing and responding to peers' execution of focal skills (e.g., Zhang et al., 2011). We consider the extent to whichwithin a particular PL model that relies on educator noticing of peer practice to build professional growth (Clarke & Hollingsworth, 2002)-objects of noticing that educators can attend to in diverse forms of evidence may (or may not) align with the module learning objectives. We do not address directly whether the program's learning objectives are met for individual educators. Rather, we explore their feedback as indicators of what is available to notice in shared evidence of practice to understand how diverse contributions can be leveraged to support program goals.

Our work is situated within the unique realities of afterschool education, which stymies one-size-fits-all approaches to sharing evidence of practice to support professional growth. We draw from afterschool educators' ingenuity in sharing diverse samples of their STEM facilitation practice and their utilization of these varied forms of evidence to notice educator actions they recognize as "strengths" or "opportunities," ideally in the context of the focal learning skill. This focus on afterschool educators and forms of evidence they have selfselected differs from previous research on noticing focused on pre-service (e.g., Benedict-Chambers & Aram, 2017;Johnson & Mawyer, 2019) or classroom educators (e.g., Criswell et al., 2022;Krist et al., 2023) and one type of evidence, such as a peer video, that has been assigned for feedback sessions.

We identified four "Outcomes," or categories of activities educators noticed in their peer's practice. In examining the extent to which diverse forms of evidence contributed to noticing activities within these outcome categories, we recognized that, while outcomes were largely realized across evidence sources, the nature of noticing differed according to the evidence. Educators appeared to tailor their feedback to align with the opportunities for observing each type of evidence provided-and engaged in the professional growth process on which ACRES is grounded (Clarke & Hollingsworth, 2002)-but they supplemented their noticing with their experientially derived expertise, especially in offering peers "opportunities for growth." No evidence source perfectly articulated youth and educator verbalization, showed visible reactions, and reflected the realities of youth interactions in afterschool programs. However, these "imperfections" appeared to create space for participating educators to build on what they noticed and introduce external sources of information in their exchanges. Each form of evidence enriched the feedback process by highlighting different aspects of STEM facilitation (e.g., body language awareness afforded by video with poor audio, identification of strong follow-up questions in video of contrived practice), contributing to the realization of focal skills through varied peer observations and insights. Some unanticipated forms of evidence of practice, such as lesson plans and descriptions of activities, actually did more to facilitate the ancillary skill of connecting to broader goals than did the recommended video. While we note that the attention to this outcome typically deviated from program goals, we also see this tendency as an opportunity for building APQ competencies more attentive to larger program goals. Recently, some of our newly-trained coaches have revised their offering of APQ to begin with a discussion of lesson plans to establish alignment of questioning before recording the use of APQ with youth.

Opportunities for future research

Though research cited in earlier sections indicates opportunities inherent in diverse forms of evidence (e.g, student artifacts in M. Luna & Selmer, 2021), we uniquely consider contexts in which diverse forms of evidence are shared that differ between members within a single cohort. Our research suggests PL thoughtfully designed with the unique noticing opportunities of evidence of practice in mind may enrich conversations around the STEM facilitation skill by drawing out the most salient features and creating a natural window through which participants may leverage their own experience to inform their noticing. Current work involving session data across module types (e.g., APQ, Make Math Engaging) is undertaking this inquiry, using a Funds of Knowledge framework (Moll et al., 2006) to consider how diverse forms of evidence of practice yields ways of engaging unique educator expertise and backgrounds-a phenomenon that we argue to be especially relevant given the diversity of the afterschool educator workforce.

Opportunities for practice

In Table 3, we organize the practical implications of our findings in terms of "strengths" and "limitations" of each evidence source (RQ2). We recommend that educators use this resource to enhance participation opportunities for their educators and to look at the limitations of each form of evidence to direct focus on the areas where that particular source excels. For instance, given that participants may not readily notice inattention to learner background from their peers' practice, coaches may direct participants' attention to this skill in produced videos as a conscious way to introduce the feedback practice and build some conversation guidance to translating the skill to feedback with peers. Or, coaches may Peers may be reluctant to suggest that their colleagues made choices that reflected poor understanding of youth in their programs Audio quality may be relatively poor, forcing attention to other cues to "fill in the gaps," especially regarding youth responses Contrived video

Provides strong examples of the practice under "ideal" circumstances (e.g., high knowledge of students, unbounded time for activity) for peers to notice.

Feedback tends to delve into hypothetical situations, especially feedback on "opportunities for growth," rather than things that are actually noticed in the evidence Peers may struggle to notice opportunities and take this di@culty as a judgment on their capabilities Coach-provided video

Provides specific instances of group work for peers to notice Educators may be more willing to suggest lack of alignment with youth background Without greater context of the classroom, the educator, and the youth, peers may be less able to provide feedback on youth social and emotional reactions Audio of authentic practice

Educators may be more likely to notice language and phrasing Peers may be more likely to voice their experiences as they fill in mental visual details

In filling in details from their own experience, peers' noticing deviates from the sharing educator's actual experience

Descriptions of future or past teaching activities

The "full picture" of the activity helps to highlight feedback on goal alignment Peers may be more prone to give feedback that does not directly address APQ and the performance of the educator.

find purposefully cueing participants to attend to verbalizations in audio evidence, or body language in videos with poor audio quality, may support educators who bring this type of evidence in recognizing the educative value their contributions bring to the group. While previous work (e.g., Kleinknecht & Schneider, 2013) has explored differences in noticing based on who shares evidence, this table uniquely advises facilitation action based on the various forms of evidence educators may realistically provide. This resource may be especially useful for fully virtual PL or other programs where budget and logistic limitations necessitate coaches to adapt facilitation based on educator experience according to what evidence of practice is available to each educator. As indicated in Table 3, each evidence source provides opportunities for peers to attend to salient aspects of practice; identify skills of APQ that are indicated; and connect their observations to the APQ training and ideas for improved practice (see Figure 2). Allowing educators to self-select the types of evidence they may bring to PL sessions for peer feedback may alleviate barriers to participation (i.e. not having permission to record youth or scheduling issues) while still allowing coaches to provide PL that can meet stated outcomes. For instance, Figure 4 indicates the model of professional growth (Clarke & Hollingsworth, 2002) in the context of sharing videos with low-quality audio-a common choice for educators in ACRES. In this model, the external experience and practice are constant, but the observations and awareness that inform subsequent practice and participation in the PL are enhanced particularly with awareness of visual cues-a point of noticing that we did not observe in evidence with higher quality audio, where peers' attention appeared to be more attuned to verbalizations.

The limitations indicated in the table indicate aspects of coaching that could be strengthened. For instance, we saw that participants tended to arrive at negative self-evaluations of their practice when viewing their peers' contrived videos. This judgment threatens the ACRES program's goals of cultivating a supportive and brave space for educator growth. Thus, coaches may consider how the strengths of evidence sources may be leveraged to address growth opportunities. In this example, coaches may frame sharing a contrived video as an opportunity to explore possibilities afforded by in-depth, one-on-one conversations with a learner.