skip to main content

Title: ABET's Maverick Evaluators and the Limits of Accreditation as a Mode of Governance in Engineering Education
This paper reflects on the significance of ABET’s “maverick evaluators” and what it says about the limits of accreditation as a mode of governance in US engineering education. The US system of engineering education operates as a highly complex system, where the diversity of the system is an asset to robust knowledge production and the production of a varied workforce. ABET Inc., the principal accreditation agency for engineering degree programs in the US, attempts to uphold a set of professional standards for engineering education using a voluntary, peer-based system of evaluation. Key to their approach is a volunteer army of trained program evaluators (PEVs) assigned by the engineering professional societies, who serve as the frontline workers responsible for auditing the content, learning outcomes, and continuous improvement processes utilized by every engineering degree program accredited by ABET. We take a look specifically at those who become labeled “maverick evaluators” in order to better understand how this system functions, and to understand its limitations as a form of governance in maintaining educational quality and appropriate professional standards within engineering education. ABET was established in 1932 as the Engineers’ Council for Professional Development (ECPD). The Cold War consensus around the engineering sciences led to more » a more quantitative system of accreditation first implemented in 1956. However, the decline of the Cold War and rising concerns about national competitiveness prompted ABET to shift to a more neoliberal model of accountability built around outcomes assessment and modeled after total quality management / continuous process improvement (TQM/CPI) processes that nominally gave PEVs greater discretion in evaluating engineering degree programs. However, conflicts over how the PEVs exercised judgment points to conservative aspects in the structure of the ABET organization, and within the engineering profession at large. This paper and the phenomena we describe here is one part of a broader, interview-based study of higher education governance and engineering educational reform within the United States. We have conducted over 300 interviews at more than 40 different academic institutions and professional organizations, where ABET and institutional responses to the reforms associated with “EC 2000,” which brought outcomes assessment to engineering education, are extensively discussed. The phenomenon of so-called “maverick evaluators” reveal the divergent professional interests that remain embedded within ABET and the engineering profession at large. Those associated with Civil and Environmental Engineering, and to a lesser extent Mechanical Engineering continue to push for higher standards of accreditation grounded in a stronger vision for their professions. While the phenomenon is complex and more subtle than we can summarize in an abstract, “maverick evaluators” emerged as a label for PEVs who interpreted their role, including determinations about whether certain content “appropriate to the field of study,” utilizing professional standards that lay outside of the consensus position held by the majority of the member of the Engineering Accreditation Commission. This, conjoined with the engineers’ epistemic aversion to uncertainty and concerns about the legal liability of their decisions, resulted in a more narrow interpretation of key accreditation criteria. The organization then designed and used a “due-process” reviews process to discipline identified shortcomings in order to limit divergent interpretations. The net result is that the bureaucratic process ABET built to obtain uniformity in accreditation outcomes, simultaneously blunts the organization’s capacity to support varied interpretations of professional standards at the program level. The apparatus has also contributed to ABET’s reputation as an organization focused on minimum standards, as opposed to one that functions as an effective driver for further change in engineering education. « less
Authors:
 ;  ;  ;  ;  ;  ;  
Award ID(s):
1656125
Publication Date:
NSF-PAR ID:
10302843
Journal Name:
2021 ASEE Virtual Annual Conference Content Access
Sponsoring Org:
National Science Foundation
More Like this
  1. Who and by what means do we ensure that engineering education evolves to meet the ever changing needs of our society? This and other papers presented by our research team at this conference offer our initial set of findings from an NSF sponsored collaborative study on engineering education reform. Organized around the notion of higher education governance and the practice of educational reform, our open-ended study is based on conducting semi-structured interviews at over three dozen universities and engineering professional societies and organizations, along with a handful of scholars engaged in engineering education research. Organized as a multi-site, multi-scale study, our goal is to document differences in perspectives and interest the exist across organizational levels and institutions, and to describe the coordination that occurs (or fails to occur) in engineering education given the distributed structure of the engineering profession. This paper offers for all engineering educators and administrators a qualitative and retrospective analysis of ABET EC 2000 and its implementation. The paper opens with a historical background on the Engineers Council for Professional Development (ECPD) and engineering accreditation; the rise of quantitative standards during the 1950s as a result of the push to implement an engineering science curriculum appropriate tomore »the Cold War era; EC 2000 and its call for greater emphasis on professional skill sets amidst concerns about US manufacturing productivity and national competitiveness; the development of outcomes assessment and its implementation; and the successive negotiations about assessment practice and the training of both of program evaluators and assessment coordinators for the degree programs undergoing evaluation. It was these negotiations and the evolving practice of assessment that resulted in the latest set of changes in ABET engineering accreditation criteria (“1-7” versus “a-k”). To provide an insight into the origins of EC 2000, the “Gang of Six,” consisting of a group of individuals loyal to ABET who used the pressure exerted by external organizations, along with a shared rhetoric of national competitiveness to forge a common vision organized around the expanded emphasis on professional skill sets. It was also significant that the Gang of Six was aware of the fact that the regional accreditation agencies were already contemplating a shift towards outcomes assessment; several also had a background in industrial engineering. However, this resulted in an assessment protocol for EC 2000 that remained ambiguous about whether the stated learning outcomes (Criterion 3) was something faculty had to demonstrate for all of their students, or whether EC 2000’s main emphasis was continuous improvement. When it proved difficult to demonstrate learning outcomes on the part of all students, ABET itself began to place greater emphasis on total quality management and continuous process improvement (TQM/CPI). This gave institutions an opening to begin using increasingly limited and proximate measures for the “a-k” student outcomes as evidence of effort and improvement. In what social scientific terms would be described as “tactical” resistance to perceived oppressive structures, this enabled ABET coordinators and the faculty in charge of degree programs, many of whom had their own internal improvement processes, to begin referring to the a-k criteria as “difficult to achieve” and “ambiguous,” which they sometimes were. Inconsistencies in evaluation outcomes enabled those most discontented with the a-k student outcomes to use ABET’s own organizational processes to drive the latest revisions to EAC accreditation criteria, although the organization’s own process for member and stakeholder input ultimately restored much of the professional skill sets found in the original EC 2000 criteria. Other refinements were also made to the standard, including a new emphasis on diversity. This said, many within our interview population believe that EC 2000 had already achieved much of the changes it set out to achieve, especially with regards to broader professional skills such as communication, teamwork, and design. Regular faculty review of curricula is now also a more routine part of the engineering education landscape. While programs vary in their engagement with ABET, there are many who are skeptical about whether the new criteria will produce further improvements to their programs, with many arguing that their own internal processes are now the primary drivers for change.« less
  2. Who and by what means do we ensure that engineering education evolves to meet the ever changing needs of our society? This and other papers presented by our research team at this conference offer our initial set of findings from an NSF sponsored collaborative study on engineering education reform. Organized around the notion of higher education governance and the practice of educational reform, our open-ended study is based on conducting semi-structured interviews at over three dozen universities and engineering professional societies and organizations, along with a handful of scholars engaged in engineering education research. Organized as a multi-site, multi-scale study, our goal is to document differences in perspectives and interest the exist across organizational levels and institutions, and to describe the coordination that occurs (or fails to occur) in engineering education given the distributed structure of the engineering profession. This paper offers for all engineering educators and administrators a qualitative and retrospective analysis of ABET EC 2000 and its implementation. The paper opens with a historical background on the Engineers Council for Professional Development (ECPD) and engineering accreditation; the rise of quantitative standards during the 1950s as a result of the push to implement an engineering science curriculum appropriate tomore »the Cold War era; EC 2000 and its call for greater emphasis on professional skill sets amidst concerns about US manufacturing productivity and national competitiveness; the development of outcomes assessment and its implementation; and the successive negotiations about assessment practice and the training of both of program evaluators and assessment coordinators for the degree programs undergoing evaluation. It was these negotiations and the evolving practice of assessment that resulted in the latest set of changes in ABET engineering accreditation criteria (“1-7” versus “a-k”).« less
  3. Unlike medicine, the engineering profession establishes new standards for engineering education through a distributed system of governance that mirrors the distributed structure of the profession. In this paper, we present our initial findings and data resulting from an NSF-sponsored study of this phenomenon. This qualitative study is multi-site and multi-scale in its design, and draws on interviews with faculty and administrators, of different rank, from at least two-dozen different colleges and universities as well as engineering professional organizations. We also complement our interview data with content analysis of archival documents and published studies, reports, and statements. The research questions that define our study consist of understanding and documenting the a) the basic structure of the engineering profession and U.S. higher education as it impacts engineering education reform initiatives; b) the historically evolving body of practice that has governed these reforms; c) the ways in which the epistemic habits of engineers, such as an emphasis on quantification and measurement, contributes to reform agendas and outcomes; d) the extent to which engineering educators are cognizant of the social and historical contexts within which they operate, and how their articulations of this context come to define dominant directions in reform; e) the processesmore »through which destabilization and closure occurs with regards to shared standards in engineering education; f) more specifically, the mechanisms through with engineering education reform agendas are coordinated across different institutions; f) and likewise, common mechanisms through which such coordination is frustrated, undermined, and sometimes reversed, especially as a consequence of competing agendas that arise out of institutional diversity and other identifiable causes. By the time of our annual meeting, we expect to be able to offer initial insights into each of our research questions. This paper will offer a preliminary presentation of our findings, including the presentation of illustrative evidence from our data set. The study is designed to provide all engineering educators with a deeper understanding of the context in which they operate, with the aim of producing more effective, inclusive, accommodating, and enduring solutions to the challenges of engineering education. (Note: A more speculative paper, exploring the theoretical and philosophical dimensions of governance in engineering education without a specific emphasis on our research questions and data set, has also been submitted separately to the TELPhE Division. The two papers will be different, presented by different lead authors, and complement one another.)« less
  4. Unlike medicine, the engineering profession establishes new standards for engineering education through a distributed system of governance that mirrors the distributed structure of the profession. In this paper, we present our initial findings and data resulting from an NSF-sponsored study of this phenomenon. This qualitative study is multi-site and multi-scale in its design, and draws on interviews with faculty and administrators, of different rank, from at least two-dozen different colleges and universities as well as engineering professional organizations. We also complement our interview data with content analysis of archival documents and published studies, reports, and statements. The research questions that define our study consist of understanding and documenting the a) the basic structure of the engineering profession and U.S. higher education as it impacts engineering education reform initiatives; b) the historically evolving body of practice that has governed these reforms; c) the ways in which the epistemic habits of engineers, such as an emphasis on quantification and measurement, contributes to reform agendas and outcomes; d) the extent to which engineering educators are cognizant of the social and historical contexts within which they operate, and how their articulations of this context come to define dominant directions in reform; e) the processesmore »through which destabilization and closure occurs with regards to shared standards in engineering education; f) more specifically, the mechanisms through with engineering education reform agendas are coordinated across different institutions; f) and likewise, common mechanisms through which such coordination is frustrated, undermined, and sometimes reversed, especially as a consequence of competing agendas that arise out of institutional diversity and other identifiable causes. By the time of our annual meeting, we expect to be able to offer initial insights into each of our research questions. This paper will offer a preliminary presentation of our findings, including the presentation of illustrative evidence from our data set. The study is designed to provide all engineering educators with a deeper understanding of the context in which they operate, with the aim of producing more effective, inclusive, accommodating, and enduring solutions to the challenges of engineering education. (Note: A more speculative paper, exploring the theoretical and philosophical dimensions of governance in engineering education without a specific emphasis on our research questions and data set, has also been submitted separately to the TELPhE Division. The two papers will be different, presented by different lead authors, and complement one another.)« less
  5. This work-in-progress paper presents preliminary findings on how the education of engineering ethics is justified by academic administrators and policymakers drawing from the data collected in a multi-institutional project called “The Distributed System of Governance in Engineering Education”. The project seeks to understand the practice of engineering education reform using ethnomethodological data collected from oral interviews at a variety of academic institutions and other organizations in engineering education. Investigations of effective strategies for ethical formation of engineering students have been continuously pursued in the engineering education community. Review of the literature on this topic results in not only identifying diverse approaches and conceptions of engineering ethics, but also a set of diverse rationales and contexts of justification for development and implementation of programs on engineering ethics. The students’ attitude towards ethical development is shaped by how the subject is delivered, e.g., use of “best practices” or conceptual clarity in the notion of ethics offered to them, as well as why it is taught. Institutions send a signal to students, even if they do not intend to, about the importance of ethics in the engineering profession by how and why they address this matter. The initial analysis of interview data frommore »over a hundred subjects from more than twenty universities demonstrates diverse ways of justifying ethics education such as satisfying ABET accreditation requirements or complying with recommendations of the disciplinary professional association (e.g., ASME or ASCE). Identifying a resistance to notions such as judgment, and in general, a disregard for engineering ethics in conversations on governance and educational decision-making are other preliminary findings of this work.« less