skip to main content

Title: Development of the Cooperative Adoption Factors Instrument to measure factors associated with instructional practice in the context of institutional change
Abstract Background

Many institutional and departmentally focused change efforts have sought to improve teaching in STEM through the promotion of evidence-based instructional practices (EBIPs). Even with these efforts, EBIPs have not become the predominant mode of teaching in many STEM departments. To better understand institutional change efforts and the barriers to EBIP implementation, we developed the Cooperative Adoption Factors Instrument (CAFI) to probe faculty member characteristics beyond demographic attributes at the individual level. The CAFI probes multiple constructs related to institutional change including perceptions of the degree of mutual advantage of taking an action (strategic complements), trust and interconnectedness among colleagues (interdependence), and institutional attitudes toward teaching (climate).

Results

From data collected across five STEM fields at three large public research universities, we show that the CAFI has evidence of internal structure validity based on exploratory and confirmatory factor analysis. The scales have low correlations with each other and show significant variation among our sampled universities as demonstrated by ANOVA. We further demonstrate a relationship between the strategic complements and climate factors with EBIP adoption through use of a regression analysis. In addition to these factors, we also find that indegree, a measure of opinion leadership, correlates with EBIP adoption.

Conclusions

The CAFI more » uses the CACAO model of change to link the intended outcome of EBIP adoption with perception of EBIPs as mutually reinforcing (strategic complements), perception of faculty having their fates intertwined (interdependence), and perception of institutional readiness for change (climate). Our work has established that the CAFI is sensitive enough to pick up on differences between three relatively similar institutions and captures significant relationships with EBIP adoption. Our results suggest that the CAFI is likely to be a suitable tool to probe institutional change efforts, both for change agents who wish to characterize the local conditions on their respective campuses to support effective planning for a change initiative and for researchers who seek to follow the progression of a change initiative. While these initial findings are very promising, we also recommend that CAFI be administered in different types of institutions to examine the degree to which the observed relationships hold true across contexts.

« less
Authors:
; ; ; ; ; ; ; ; ; ; ;
Award ID(s):
1726503 1726409 1726330
Publication Date:
NSF-PAR ID:
10368839
Journal Name:
International Journal of STEM Education
Volume:
9
Issue:
1
ISSN:
2196-7822
Publisher:
Springer Science + Business Media
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Efforts to achieve improved student outcomes in STEM are critically reliant on the success of reform efforts associated with teaching and learning. Reform efforts include the transformation of course-based practices, community values, and the institutional policies and structures associated with teaching and learning in higher education. Enacting change is a complex process that can be guided by change theories that describe how and why a desired change takes place. We analyzed the utility of a theory-based change model applied in a higher education setting. Our results provide guidance for change efforts at other institutions.

    Results

    Use of the CACAO model to guide the transformation of STEM instruction at a large public university resulted in changes to faculty teaching practices and department culture consistent with the vision defined for the project. Such changes varied across STEM departments in accordance with the emergent nature of project activities at the department level. Our application of the CACAO model demonstrates the importance of (1) creating a vision statement (statement of desired change or end-state); (2) attending to different levels of the organization (e.g., individuals, departments, and colleges); (3) working with change agents who are situated to be effective at different organizational levels; and (4)more »employing strategies to meet the needs and interests of faculty at different stages of adoption with respect to the desired change.

    Conclusion

    Our work, which demonstrates the utility of the CACAO model for change and captures its key elements in a matrix, provides a potential foundation for others considering how to frame and study change efforts. It reinforces the value of using change theories to inform change efforts and creates a structure that others can build on and modify, either by applying our CACAO matrix in their own setting or by using the matrix to identify elements that connect to other change theories. We contribute to the growing body of literature which seeks to understand how change theories can be useful and generalizable beyond a single project.

    « less
  2. null (Ed.)
    In 2016, 10 universities launched a Networked Improvement Community (NIC) aimed at increasing the number of scholars from Alliances for Graduate Education and the Professoriate (AGEP) populations entering science, technology, engineering, and mathematics (STEM) faculty careers. NICs bring together stakeholders focused on a common goal to accelerate innovation through structured, ongoing intervention development, implementation, and refinement. We theorized a NIC organizational structure would aid understandings of a complex problem in different contexts and accelerate opportunities to develop and improve interventions to address the problem. A distinctive feature of this NIC is its diverse institutional composition of public and private, predominantly white institutions, a historically Black university, a Hispanic-serving institution, and land grant institutions located across eight states and Washington, DC, United States. NIC members hold different positions within their institutions and have access to varied levers of change. Among the many lessons learned through this community case study, analyzing and addressing failed strategies is as equally important to a healthy NIC as is sharing learning from successful interventions. We initially relied on pre-existing relationships and assumptions about how we would work together, rather than making explicit how the NIC would develop, establish norms, understand common processes, and manage changing relationships.more »We had varied understandings of the depth of campus differences, sometimes resulting in frustrations about the disparate progress on goals. NIC structures require significant engagement with the group, often more intensive than traditional multi-institution organizational structures. They require time to develop and ongoing maintenance in order to advance the work. We continue to reevaluate our model for leadership, climate, diversity, conflict resolution, engagement, decision-making, roles, and data, leading to increased investment in the success of all NIC institutions. Our NIC has evolved from the traditional NIC model to become the Center for the Integration of Research, Teaching and Learning (CIRTL) AGEP NIC model with five key characteristics: (1) A well-specified aim, (2) An understanding of systems, including a variety of contexts and different organizations, (3) A culture and practice of shared leadership and inclusivity, (4) The use of data reflecting different institutional contexts, and (5) The ability to accelerate infrastructure and interventions. We conclude with recommendations for those considering developing a NIC to promote diversity, equity, and inclusion efforts.« less
  3. Abstract Background

    Active learning used in science, technology, engineering, and mathematics (STEM) courses has been shown to improve student outcomes. Nevertheless, traditional lecture-orientated approaches endure in these courses. The implementation of teaching practices is a result of many interrelated factors including disciplinary norms, classroom context, and beliefs about learning. Although factors influencing uptake of active learning are known, no study to date has had the statistical power to empirically test the relative association of these factors with active learning when considered collectively. Prior studies have been limited to a single or small number of evaluated factors; in addition, such studies did not capture the nested nature of institutional contexts. We present the results of a multi-institution, large-scale (N = 2382 instructors;N = 1405 departments;N = 749 institutions) survey-based study in the United States to evaluate 17 malleable factors (i.e., influenceable and changeable) that are associated with the amount of time an instructor spends lecturing, a proxy for implementation of active learning strategies, in introductory postsecondary chemistry, mathematics, and physics courses.

    Results

    Regression analyses, using multilevel modeling to account for the nested nature of the data, indicate several evaluated contextual factors, personal factors, and teacher thinking factors were significantly associated with percent of class time lecturing when controlling formore »other factors used in this study. Quantitative results corroborate prior research in indicating that large class sizes are associated with increased percent time lecturing. Other contextual factors (e.g., classroom setup for small group work) and personal contexts (e.g., participation in scholarship of teaching and learning activities) are associated with a decrease in percent time lecturing.

    Conclusions

    Given the malleable nature of the factors, we offer tangible implications for instructors and administrators to influence the adoption of more active learning strategies in introductory STEM courses.

    « less
  4. Abstract Background

    Repeated calls to diversify the population of students earning undergraduate degrees in science, technology, engineering, and mathematics (STEM) fields have noted the greater diversity of community college students and their potential to thus have an impact on the racial/ethnic composition of 4-year degree earners. In this paper, we investigate barriers and supports to Black women’s success in STEM, using longitudinal interview data with seven Black women who were enrolled at community colleges and stated an interest in majoring in STEM at 4-year institutions.

    Results

    Our findings highlight a contrast between community colleges and universities. At community colleges, Black women were able to form supportive relationships with professors and peers, downplayed the potential of racism and sexism to derail their STEM ambitions, and saw little to no impact of bias on their educational experiences. Those students who transferred characterized university climates very differently, as they struggled to form supportive relationships and experienced racism and sexism from professors and peers.

    Conclusions

    We conclude using Patricia Hill Collins’ Domains of Power framework to categorize students’ experiences, then end with recommendations for change that will result in less alienating experiences for Black women, among other minoritized students.

  5. Amid increasing demands from students and the public, universities have recently reinvigorated their efforts to increase the number of faculty from underrepresented populations. Although a myriad of piecemeal programs targeting individual recruitment and development have been piloted at several institutions, overall growth in faculty diversity remains almost negligible and highly localized. To bring about genuine change, we hypothesize a consortia approach that links individuals to hiring opportunities within a state university system might be more effective. Here we present a case study describing the progress of the NSF-funded Alliances for Graduate Education and the Professoriate (AGEP) PROMISE Academy Alliance, a consortium within the University System of Maryland (USM) collaborating to develop, implement, self-study, evaluate, and disseminate a unique postdoc-to-faculty conversion model in the biomedical sciences. The initiative centers on diversifying faculty across five institutions in the USM, including teaching-focused institutions, comprehensive universities, research institutions, and professional schools. Components of this approach include (1) enhanced recruiting and hiring practices to attract outstanding postdoctoral scholars from underrepresented backgrounds, (2) multi-institutional networking and professional development, and (3) facilitated processes to transition (or “convert”) postdocs into tenure-track positions at their postdoctoral institution or another institution in the state system. This model is distinct frommore »more deficit-based approaches because it goes beyond focusing on building the individual’s skills to enter the professoriate. This program restructures the traditionally short-term nature of postdoctoral employment and incorporates a pathway to a tenure-track professorship at the same institution or within the same statewide system where the postdoc is trained. This multi-institutional model leverages collaboration and distinct institutional strengths to create cross-institutional support, advocacy, and policy. Importantly, it uses a decentralized financial structure that makes this approach distinctly replicable. Recognizing the immediate need for more collaborative approaches to diversify faculty and a lack of literature about such approaches, this case study describes the development of, and potential benefits of, a state university system, as well as the qualitative lessons learned from self-study, internal evaluation, external evaluation, and NSF site visits. The AGEP PROMISE Academy can serve as a model for replication at other university systems hoping to diversify their faculty.« less