Across higher education, many institutions are looking at a changing landscape that could feature lower enrollments and will require necessary changes to address the potential challenges [1,2]. In 2020, 45.2% of department chairs surveyed reported facing some form of threat to the continuation of their physics program [3]. While these types of threats may not be experienced by all physics programs, many departments still feature unwelcoming environments for students [4][5][6][7]. To create more supportive undergraduate programs, many physics departments across the country must undergo substantial changes. For departments who embrace change, this is an opportunity to better support students and strengthen their programs. To meet the needs of physics programs across the U.S. and support them in pursuing these change efforts, the American Association of Physics Teachers and the American Physical Society established the Effective Practices for Physics Programs (EP3) initiative.
The Effective Practices for Physics Programs (EP3) Guide [8] is designed to empower program leaders to create and sustain effective departmental change. The EP3 Guide is a collection of knowledge and proven good practice drawn from the experience of the physics community to help departments engage with improving their programs. Core principles underlying use of the EP3 Guide are that successful and sustainable change efforts are highly intentional, rooted in a clear sense of departmental mission, grounded in data, and broadly inclusive of members of the departmental community.
While physicists are trained in research, and sometimes teaching, they rarely have training in how to effectively lead change efforts. To address this gap, the EP3 Initiative developed the Departmental Action Leadership Institute (DALI) to support faculty members in implementing changes to their undergraduate programs built on change theories outlined in organizational management and change literature [9,10]. Each participating program chooses a pair of faculty members to act as change leaders, who attend an in-person kickoff workshop followed by ongoing virtual meetings for one year to learn about and implement effective change practices. These pairs lead cross-constituency (faculty, staff, students, etc.) teams in their departments following the Departmental Action Team (DAT) model [11,12]. The DALI supports the change leaders in many aspects of change work and leadership that we have explored previously [13].
To date, there have been five DALI cohorts, which have included 43 physics faculty members from 22 physics programs (one program selected only one change leader). The participating physics programs are smaller sized programs, with an average of about six full-time faculty members. Cohorts 1 and 2 had virtual kickoffs due to COVID, while cohorts 3, 4, and 5 had in person kickoffs. Cohorts 3 and 4 ran concurrently and shared a joint kickoff event. Corbo and Craig served as the co-facilitators for cohorts 1, 2, and 5. For cohorts 3 and 4, they were joined by three additional cofacilitators: two members of cohort 1 and one of the EP3 Guide editorial directors. The content meetings included a mix of DALI curriculum and discussing "problems of practice" [14]-specific challenges that the change leaders are facing that they bring to the group for advice and reflection.
The DALI guides change leaders through a planned "curriculum." Based on earlier work by the DAT Project [15] and the EP3 Cycle of Reflection and Action [16], the DALI envisions the arc of change projects as a developmental cycle with various stages supporting the ultimate goal of a successful and sustainable change effort. The stages can be encapsulated by three big questions that drive the DALI curriculum: Where are we trying to go? (e.g., developing shared vision, goals, desired outcomes), Where are we now? (e.g., understanding the current departmental context, opportunities, constraints), and How are we going to get there? (e.g., using change models to build, implement, assess, and reflect on action plans).
A challenge facing departments engaged with the DALI is that while direct support lasts for a single academic year significant change efforts in higher education typically require multiple years to envision, plan, and implement. The DALI curriculum does address the full cycle of a successful and sustainable change effort, but individual DATs will typically be only part way through the full cycle when the DALI ends; change leaders must then lead their department's change efforts to completion on their own. We do not yet know what impact this may have on the likely success of the local departmental change initiatives.
We have investigated the ways in which change leaders describe their typical departmental practices in comparison to the new cultural practices that are taking shape in their DATs. We found that the change leaders grew in their awareness of different change practices and that departments previously approached change through ad hoc, individualistic, and faculty-centric efforts [17]. In contrast, the DALI-supported DATs have embraced using data to better understand their departments' challenges [18] and include students as members of their teams [19]. However, the expectations are sometimes at odds between faculty and students on these teams which can lead to differences in their understanding of what motivated change and how long change takes [20].
Similar to a community of transformation, the DALI provides a new philosophy for participants, creates space to engage in new practice, and connects change leaders into a network of peers [21]. Preliminary findings from the first cohort of change leaders identified outcomes for change leaders that focused on their level of preparedness in leading different aspects of effective change [13]. In the first cohort all change leaders felt well prepared to build teams and shared vision with team members, but only a few were prepared to interface with other stakeholders outside of the team. This continued work is, in part, inspired by discussions with others in the PER community about the types of departments that are involved in DALI and how department size may influence change processes. In this paper, we aim to document the outcomes after a total of five cohorts of DALI. This study is guided by the following research questions: (1) What kinds of departments participate in DALI? (2) What outcomes do DALI change leaders experience?
Turpen who discussed methodology and results. These results were then shared with Corbo and Craig. Data was gathered from AIP physics program statistics [22], IPEDS data sets, the DALI applications, and university websites. We aim to characterize the departments who participated in DALI by their size and other aspects of their institutional context.
We used an AIP data set to characterize the undergraduate physics program size by the number of graduates reported in 2021-2022 [22]. This is the time period in which DALI started and thus is a good metric for comparing program sizes. Some departments who were part of DALI did not report their data to AIP (N=3) or were left out of this data set for other reasons (e.g., physics program does not exist, non-US university) (N=2). To estimate the number of full time physics faculty members for each department, we used their self-reported numbers in the DALI applications. The number of full time faculty members includes any physics and/or astronomy faculty members. Other disciplinary faculty members, such as chemistry or biology, were not counted towards the estimate of full time physics faculty members.
The AIP data set identifies the highest physics degree offered at the university as reported by program chairs [22]. We used IPEDS data to identify private versus public designation. In addition to applications, we used university websites to gather confirmatory evidence about what disciplines made up their department. Any discipline other than physics or astronomy was counted as "multidisciplinary." These are reported as descriptive statistics in our results section.
Our second research question aims to draw attention to the outcomes for participants in DALI. To answer this question, we use the DALI exit survey that is distributed at the end of each cohort. As cohort 5 is not yet complete, only four cohorts are represented in this data set and analysis.
This exit survey was developed to assess the self-reported outcomes for change leaders. It was developed both as a tool to triangulate other research findings and serve as formative feedback for DALI facilitators. The survey aims to measure how well the change leaders feel they progressed along the four strands of DALI as described earlier: carrying out the change effort (eight items), developing a high functioning team (six items), building positive relationships (five items), and growing as change agents (five items). In our analysis, we will refer to these sets of assessment items as themes. The survey items for each theme were designed based on aspects of change work outlined in the DALI curriculum and have remained consistent throughout the four DALI cohorts.
The survey is divided into five sections: (i) a comparison between level of preparedness to lead change efforts pre-DALI and post-DALI, (ii) a self-evaluation of the level of success that their DAT has achieved in each area of change, (iii) an evaluation of the level of support they received from DALI in each area, (iv) a self-assessment of their current and ideal DAT implementation, and (v) a short answer section that asks them to describe their efforts holistically. Note, the "pre-DALI" answers represent participants' retrospective feelings of their preparedness before the year-long DALI activities.
The overall response rate for this survey was 76% (28/37). The response rates of cohort 1 (100%, 10/10) and cohort 2 (80%, 8/10) were substantially higher than the response rates of cohorts 3 and 4 (59%, 10/17). The survey was administered during DALI meetings for cohort 1 and cohort 2, but asynchronously for cohorts 3 and 4. Because of this difference in response rates, we primarily consider overall outcomes indicated by this survey instrument rather than describe outcomes that are different between cohorts.
In this paper, we present results from survey sections (i), (ii), and (iii). In these sections, change leaders were presented with a statement and could indicate their level of satisfaction with the associated activity. This was a five point scale with responses of "Not at all," "Scarcely," "Moderately," "Largely," and "Completely." Each of these we associate with a value of 1 to 5 to calculate shifts in the responses for section (i). For section (ii), focused on how satisfied the change leaders were with the success of their DATs, an additional option of "Not yet attempted" was included. We use theme when referring to a group of items which speaks to a DALI strand, and aspect when referring to the result from a single item which speaks to a specific component of change work.
To test significance shifts in change leaders' reported levels of preparedness to lead change pre-DALI to post-DALI (section (i) of the survey) we use a Wilcoxon signed rank test, which is appropriate for paired-responses of ordinal nonparametric data [23]. To investigate which DALI strand shows the most and least improvement, we investigate both the average absolute gain on each item, as well as the total increase in the number of change leaders who reported "largely" or "completely" levels of preparedness, which we interpret as reflecting confidence in this strand of change management.
Participation in DALI is diverse with the median size of the physics faculty as reported in DALI application is six faculty members with a wide variance across departments, as seen FIG. 2. Stacked bar chart of the number of physics programs who graduate the number of bachelor degrees in 2020-2021. In gray are all U.S. physics programs that are not in DALI. In black are the physics programs in DALI.
in Figure 1. For departments with three or less full-time faculty members, typically all faculty members were active in their DAT. For departments that have between four and size full-time faculty members a majority of them were involved in their DAT. For these two groups of departments, the DATs would have been able to involve a majority of faculty members in discussions about change work. For the departments that had greater than seven full-time faculty members, there is a need to focus on external discussions with others in the department about the DAT's work.
The median number of bachelor's degrees awarded in the 2021-2022 academic year was 4 (Interquartile range [2,11]) across the DALI departments. While this is a small number of physics graduates per year compared to the types of contexts PER historically studies [24], the distribution within the DALI cohort is representative of the full population of U.S. physics departments, as shown in Figure 2.
Of the 22 departments who participated in DALI, seven of the departments were multidisciplinary. The other disciplines represented in these departments included chemistry, biology, mathematics, and engineering. Most of the departments offer a bachelor's as the highest degree (N bach = 14), with just a couple offering master's as the highest degree (N mas = 2), and a few offering Ph.D. as the highest physics degree (N P HD = 5). One participating department did not offer any form of physics degree at the time of participating. A majority of the departments in DALI are situated in public institutions (N pub = 14) rather than private institutions (N priv = 8). These characteristics may play important roles when considering how to pursue change work and how different stakeholders orient to that work.
Across the different cohorts, change leaders report feeling more prepared for many different aspects of change leadership. These are measured by section (i) in the exit ticket survey. The survey items were developed along the four DALI strands: (a) carrying out the change effort, (b) developing a high functioning team, (c) building positive relationships, and (d) growing as change agents. The change leaders reported their level of preparedness for each item comparing pre-DALI to post-DALI. We found that for every item, the shift in the distribution of responses was significantly different according to the Wilcoxon signed-rank test (p < 0.001).
We averaged item responses within each theme and calculated the absolute gain of the clustered theme as well as the average change in the number of participants answering "largely" or "completely" (4 or 5 on the 5-point scale) prepared for that aspect of change leadership. Within each theme, there was an average gain of at least one point on the five point scale ("carrying out the change effort": 1.29±0.13, "developing a high functioning team": 1.20 ± 0.25, "building positive relationships": 0.98 ± 0.09, and "growing as change agents": 1.37 ± 0.26). Additionally, as seen in Figure 3, the percentage of change leaders who felt "largely" or "completely" prepared to take on the different aspects of change increased dramatically from pre-DALI to post-DALI.
These results show that DALI successfully supports change leaders in feeling prepared to lead their change efforts within their departments across multiple aspects of change work. Almost all of the change leaders (26 or 27 out of 28) felt that they were largely or completely prepared in these aspects: "building a shared vision among your team," "coming to a consensus on the team's goals," and "gathering feedback from DAT members." This indicates that change leaders felt that they could create collaborative environments within their DATs, an essential part of the DALI approach to change work. Although there are clear successes of DALI highlighted by these results, they also point to some areas for improvement.
Within the theme "building positive relationships," there were less gains due to both the larger number of respondents who felt prepared beforehand, and the smaller number of respondents who felt prepared post-DALI. There are many aspects of effective change that require relationships outside of the DAT itself. Compared to the other themes, change leaders felt that they were less prepared to do these tasks.
Within the theme "developing a high functioning team," the aspects of "managing conflict" and "motivating team members" were reported as the least improved aspects, while other aspects were more improved, leading to a high variance within this theme. The change leaders may not have had direct experience with these aspects during DALI. Alternatively, the change leaders may have felt they did poorly and are not prepared to handle these aspects again in the future.
In section (ii) of the survey, change leaders reported feeling satisfied with DAT's level of success, on average, across different aspects of change. However, there were particular items in which many change leaders report not yet attempting those aspects of change work. Within the theme of "building positive relationships," "navigating external resistance to the change efforts" and "reporting back to the department and engaging in sense-making about the change efforts with stakeholders outside the DAT" were among the least attempted aspects. This strand is the same one in which change leaders feel the least prepared. Although DALI can provide resources for navigating these external relationships, it has a limited capacity to support DATs when DATs do not reach this stage in their change effort within the yearlong timespan of DALI.
In section (iii) of the survey, change leaders reported that DALI is very supportive across the different aspects of change work. In alignment with the other findings presented here, the theme that DALI could most improve on average is "building positive relationships." Although this strand had the least amount of reported support, change leaders still felt that DALI at least "moderately" supported them in enacting all of the associated aspects of effective change.
DALI participants represent a variety of physics programs. While these physics programs are smaller than those historically investigated in PER, they are representative of the population of physics departments across the country. This draws attention to an important distinction when both researching and pursuing change in physics programs. DALI well supports these smaller programs, however it is yet unclear how exactly this may differ in larger physics departments.
Overall, DALI has been successful at supporting change leaders in many aspects of change work. Yet it is clear that there are aspects of this work associated with later stages of the change process that DALI does not support as completely. This is supported both by the experiences of DALI facilitators and the results of our exit survey. To address this issue, Corbo and Craig have created the "DALI Dashboard," which helps the change leaders track the progress of their DATs through the stages of a successful change effort. The Dashboard is effectively a "change workbook" that helps change leaders organize, monitor, and record their DAT's progress. The DALI Dashboard is being prepared for eventual publication in the hopes that it will be useful to other change efforts in the higher education community.
It is important to keep in mind the limitations of this work. First, we do not have responses from all DALI participants and those that chose not to participate may have unique experiences that are not captured here. Second, at the time of writing this, cohort 5 has not yet taken the survey, and thus their experiences are not reflected in our findings of DALI impacts. Finally, our survey methodology relies on retrospective reporting from participants about their feelings "pre-DALI," which may not accurately capture their initial feelings coming in and are influenced by their experiences within DALI.
Future research intends to examine the sustainability and success of some of these local departmental change efforts. We also encourage additional case-study based research on how DALI-supported DATs function within physics programs with larger faculty and student populations.