More Like this

1. ABET & Engineering Accreditation - History, Theory, Practice: Initial Findings from a National Study on the Governance of Engineering Education

   Akera, A. ( January 2019 , ASEE Annual Conference proceedings)

   Who and by what means do we ensure that engineering education evolves to meet the ever changing needs of our society? This and other papers presented by our research team at this conference offer our initial set of findings from an NSF sponsored collaborative study on engineering education reform. Organized around the notion of higher education governance and the practice of educational reform, our open-ended study is based on conducting semi-structured interviews at over three dozen universities and engineering professional societies and organizations, along with a handful of scholars engaged in engineering education research. Organized as a multi-site, multi-scale study, our goal is to document differences in perspectives and interest the exist across organizational levels and institutions, and to describe the coordination that occurs (or fails to occur) in engineering education given the distributed structure of the engineering profession. This paper offers for all engineering educators and administrators a qualitative and retrospective analysis of ABET EC 2000 and its implementation. The paper opens with a historical background on the Engineers Council for Professional Development (ECPD) and engineering accreditation; the rise of quantitative standards during the 1950s as a result of the push to implement an engineering science curriculum appropriate tomore »the Cold War era; EC 2000 and its call for greater emphasis on professional skill sets amidst concerns about US manufacturing productivity and national competitiveness; the development of outcomes assessment and its implementation; and the successive negotiations about assessment practice and the training of both of program evaluators and assessment coordinators for the degree programs undergoing evaluation. It was these negotiations and the evolving practice of assessment that resulted in the latest set of changes in ABET engineering accreditation criteria (“1-7” versus “a-k”). To provide an insight into the origins of EC 2000, the “Gang of Six,” consisting of a group of individuals loyal to ABET who used the pressure exerted by external organizations, along with a shared rhetoric of national competitiveness to forge a common vision organized around the expanded emphasis on professional skill sets. It was also significant that the Gang of Six was aware of the fact that the regional accreditation agencies were already contemplating a shift towards outcomes assessment; several also had a background in industrial engineering. However, this resulted in an assessment protocol for EC 2000 that remained ambiguous about whether the stated learning outcomes (Criterion 3) was something faculty had to demonstrate for all of their students, or whether EC 2000’s main emphasis was continuous improvement. When it proved difficult to demonstrate learning outcomes on the part of all students, ABET itself began to place greater emphasis on total quality management and continuous process improvement (TQM/CPI). This gave institutions an opening to begin using increasingly limited and proximate measures for the “a-k” student outcomes as evidence of effort and improvement. In what social scientific terms would be described as “tactical” resistance to perceived oppressive structures, this enabled ABET coordinators and the faculty in charge of degree programs, many of whom had their own internal improvement processes, to begin referring to the a-k criteria as “difficult to achieve” and “ambiguous,” which they sometimes were. Inconsistencies in evaluation outcomes enabled those most discontented with the a-k student outcomes to use ABET’s own organizational processes to drive the latest revisions to EAC accreditation criteria, although the organization’s own process for member and stakeholder input ultimately restored much of the professional skill sets found in the original EC 2000 criteria. Other refinements were also made to the standard, including a new emphasis on diversity. This said, many within our interview population believe that EC 2000 had already achieved much of the changes it set out to achieve, especially with regards to broader professional skills such as communication, teamwork, and design. Regular faculty review of curricula is now also a more routine part of the engineering education landscape. While programs vary in their engagement with ABET, there are many who are skeptical about whether the new criteria will produce further improvements to their programs, with many arguing that their own internal processes are now the primary drivers for change.« less
2. Building Your Dream Team for Change

   https://doi.org/10.18260/1-2--32489  

   Margherio, Cara ; Doten-Snitker, Kerice ; Litzler, Elizabeth ; Williams, Julia ; Andrijcic, Eva ; Mohan, Sriram ( June 2020 , 2019 ASEE Virtual Annual Conference Content Access)

   his panel paper presents research on connecting theory to practice and the lessons learned in a change project, with a focus on team formation during the early stages of change making. An important yet often overlooked step in any change project is pulling together individuals to form a competent and efficient team. A functional change-making team requires a variety of complementary skill sets, which may come from different disciplinary backgrounds and/or different prior experiences. Kotter (1996) uses the term “guiding coalition” to refer to an effective change-making team. He identifies four key characteristics of guiding coalitions: position power, expertise, credibility, leadership. Kotter also goes on to examine the importance of trust and a common goal. In a review of the literature on guiding coalitions, Have, Have, Huijsmans, and Otto (2017) found that though the concept of a guiding coalition is widely advocated in the literature, only one study showed a moderate correlation between the existence of a guiding coalition and the success of a change process (Abraham, Griffin, & Crawford, 1999). Have et al. (2017) conclude that while the literature provides little evidence to the value of a guiding coalition, it does provide evidence that Kotter’s characteristics of a guidingmore »coalition (position power, expertise, credibility, leadership skills, trust in leadership, and setting common goals) individually have positive effects on the outcomes of a change project. However, we don’t know how these characteristics interact. This analysis of team building and complementary skill sets emerges from our participatory action research with the NSF REvolutionizing engineering and computer science Departments (RED) teams to investigate the change process within STEM higher education. The research-to-practice cycle is integral to our project; data gathered through working with the RED teams provides insights that are then translated into applied, hands-on practices. We utilize an abductive analysis approach, a qualitative methodology that moves recursively between the data and theory-building to remain open to new or contradictory findings, keeping existing theory in mind while not developing formal hypotheses (Timmermans & Tavory, 2012). We find that many of the teams have learned lessons in the early stages of the change process around the guiding coalition characteristics, and our analysis builds on the literature by examining how these characteristics interact. For example, the expertise of the social scientists and education researchers help discern which change strategies have supporting evidence and fit the context, in addition to what is reasonable for planning, implementation, and evaluation. The results presented in this paper connect theory to practice, clarifying practices for building effective change-making teams within higher education.« less
3. An Inquiry into the Use of Intercoder Reliability Measures in Qualitative Research

   Wilson-Lopez, A. ; Minichiello, A. ; Green, T. ( July 2019 , ASEE Annual Conference proceedings)

   In this theory paper, we set out to consider, as a matter of methodological interest, the use of quantitative measures of inter-coder reliability (e.g., percentage agreement, correlation, Cohen’s Kappa, etc.) as necessary and/or sufficient correlates for quality within qualitative research in engineering education. It is well known that the phrase qualitative research represents a diverse body of scholarship conducted across a range of epistemological viewpoints and methodologies. Given this diversity, we concur with those who state that it is ill advised to propose recipes or stipulate requirements for achieving qualitative research validity and reliability. Yet, as qualitative researchers ourselves, we repeatedly find the need to communicate the validity and reliability—or quality—of our work to different stakeholders, including funding agencies and the public. One method for demonstrating quality, which is increasingly used in qualitative research in engineering education, is the practice of reporting quantitative measures of agreement between two or more people who code the same qualitative dataset. In this theory paper, we address this common practice in two ways. First, we identify instances in which inter-coder reliability measures may not be appropriate or adequate for establishing quality in qualitative research. We query research that suggests that the numerical measure itselfmore »is the goal of qualitative analysis, rather than the depth and texture of the interpretations that are revealed. Second, we identify complexities or methodological questions that may arise during the process of establishing inter-coder reliability, which are not often addressed in empirical publications. To achieve this purposes, in this paper we will ground our work in a review of qualitative articles, published in the Journal of Engineering Education, that have employed inter-rater or inter-coder reliability as evidence of research validity. In our review, we will examine the disparate measures and scores (from 40% agreement to 97% agreement) used as evidence of quality, as well as the theoretical perspectives within which these measures have been employed. Then, using our own comparative case study research as an example, we will highlight the questions and the challenges that we faced as we worked to meet rigorous standards of evidence in our qualitative coding analysis, We will explain the processes we undertook and the challenges we faced as we assigned codes to a large qualitative data set approached from a post positivist perspective. We will situate these coding processes within the larger methodological literature and, in light of contrasting literature, we will describe the principled decisions we made while coding our own data. We will use this review of qualitative research and our own qualitative research experiences to elucidate inconsistencies and unarticulated issues related to evidence for qualitative validity as a means to generate further discussion regarding quality in qualitative coding processes.« less
4. What Most Facilitates Thriving for Undergraduate Engineering Students? A Rank Order Investigation of Engineering Experts

   Gesun, Julianna ; Rizzo, Julia ( August 2022 , ASEE annual conference exposition proceedings)

   This research paper explores engineering experts' perceptions of the most important factors of thriving for undergraduate engineering students. Faculty, staff, and members of the engineering education system play a vital role in creating environments, and forming relationships with students conducive to thriving. The study in this paper builds upon prior work on engineering thriving that identified 147 factors developed from a literature review, refined with expert consultation. Out of the long list of factors, little is known regarding the most important factors that can serve as a starting point for engineering experts with limited resources to create environments and relationships that support more thriving engineering students. In this paper, we analyze ranked order data to investigate the most important internal thriving competencies. Participants include 47 engineering experts i.e., engineering administrators, professors, staff, and advisers. To find which competencies were perceived as most important to engineering thriving, each expert was asked to generate and define up to ten competencies that they considered to be most important, then rank these competencies in order of importance. During data analysis, ranked competencies were scored on a reverse ordinal points basis, with the most important rankings receiving 10 points and the least important rankings receivingmore »1 point. Overall, the top five most important competencies were Communication/Listening Skills (overall score = 104), Help-seeking/ Resourcefulness (overall score = 104), Teamwork (overall score = 97), Time Management (overall score = 96), and Resilience (overall score = 95). Findings from this study highlight the importance of intrapersonal, social, and behavioral competencies, providing a starting point for future work developing a survey of thriving for engineering students. Furthermore, these findings provide a greater insight into which high-impact competencies engineering faculty, staff, and administrators can focus on when creating environments conducive to student thriving and interacting directly with students when teaching, supporting, advising, and mentoring.« less
5. Interdisciplinary Teamwork Challenges in a Design Competition Team

   Beddoes, Kacey ; Nicewonger, Todd E. ( December 2019 , Australasian Association for Engineering Education Annual Conference)

   Communication and collaboration are key components of engineering work (Trevelyan, 2014), and teamwork, including interdisciplinary teamwork, is increasingly seen as an important component of engineering education programs (Borrego, Karlin, McNair, & Beddoes, 2013; Male, Bush, & Chapman, 2010, 2011; Paretti, Cross, & Matusovich, 2014; Purzer, 2011). Employers and education researchers alike advocate teamwork as a means of developing skills that engineering graduates need (Purzer, 2011), and accreditation bodies consider the ability to both lead and function on teams as an important outcome for engineering graduates (Engineers Australia, 2017). However, “despite the clear emphasis on teamwork in engineering and the increasing use of student team projects, our understanding of how best to cultivate and assess these learning outcomes in engineering students is sorely underdeveloped (McGourty et al., 2002; Shuman, Besterfield-Sacre, & McGourty, 2005)” (Borrego et al., 2013, p. 473). In order to contribute to the current conversation on interdisciplinary teamwork in engineering education, and to advance understandings of how best to cultivate teamwork learning outcomes, this paper discusses the most common teamwork challenges and presents boundary negotiating artifacts as a conceptual framework for addressing them. Drawing on data from long-term ethnographic observations of a design competition project, and the challenges studentsmore »experienced, we utilise findings from a systematic literature review and the conceptual framework of boundary negotiating artifacts to present a case study of how boundary negotiating artifacts can support important teamwork constructs.« less