skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 5:00 PM ET until 11:00 PM ET on Friday, June 21 due to maintenance. We apologize for the inconvenience.


Title: Challenges and threats to quality in scholarly knowledge production: views of selected stakeholders from 17 countries
Abstract

This study applies Harvey and Green’s (1993) model of quality to scholarly knowledge production. Although studies of quality in higher education have been commonplace for decades, there is a gap in understanding quality in terms of research production from stakeholders’ perspectives. This study begins to fill that gap through a qualitative interview study of quality in the knowledge production process. Stakeholders in all parts of the scholarly knowledge production process, from 17 countries, are included in the data sample. Analysis of interview data extends Harvey and Green’s (1993) model into the realm of knowledge production. Definitions and challenges of quality in producing scholarly knowledge are discussed. The findings indicate a rift between the institutional view of quality and the individual perceptions of quality, suggesting the need for institutional policies that respond to stakeholders’ perceptions of quality in scholarly knowledge production and celebrate, rather than erase, epistemic diversity.

 
more » « less
Award ID(s):
1926348
NSF-PAR ID:
10368151
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
Springer Science + Business Media
Date Published:
Journal Name:
Discover Education
Volume:
1
Issue:
1
ISSN:
2731-5525
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Broadening participation in engineering is critical given the gap between the nation’s need for engineering graduates and its production of them. Efforts to spark interest in engineering among PreK-12 students have increased substantially in recent years as a result. However, past research has demonstrated that interest is not always sufficient to help students pursue engineering majors, particularly for rural students. In many rural communities, influential adults (family, friends, teachers) are often the primary influence on career choice, while factors such as community values, lack of social and cultural capital, limited course availability, and inadequate financial resources act as potential barriers. To account for these contextual factors, this project shifts the focus from individual students to the communities to understand how key stakeholders and organizations support engineering as a major choice and addresses the following questions: RQ1. What do current undergraduate engineering students who graduated from rural high schools describe as influences on their choice to attend college and pursue engineering as a post-secondary major? RQ2. How does the college choice process differ for rural students who enrolled in a 4-year university immediately after graduating from high school and those who transferred from a 2-year institution? RQ3. How do community members describe the resources that serve as key supports as well as the barriers that hinder support in their community? RQ4. What strategies do community members perceive their community should implement to enhance their ability to support engineering as a potential career choice? RQ5. How are these supports transferable or adaptable by other schools? What community-level factors support or inhibit transfer and adaptation? To answer the research questions, we employed a three-phase qualitative study. Phase 1 focused on understanding the experiences and perceptions of current [University Name] students from higher-producing rural schools. Analysis of focus group and interview data with 52 students highlighted the importance of interest and support from influential adults in students’ decision to major in engineering. One key finding from this phase was the importance of community college for many of our participants. Transfer students who attended community college before enrolling at [University Name] discussed the financial influences on their decision and the benefits of higher education much more frequently than their peers. In Phase 2, we used the findings from Phase 1 to conduct interviews within the participants’ home communities. This phase helped triangulate students’ perceptions with the perceptions and practices of others, and, equally importantly, allowed us to understand the goals, attitudes, and experiences of school personnel and local community members as they work with students. Participants from the students’ home communities indicated that there were few opportunities for students to learn more about engineering careers and provided suggestions for how colleges and universities could be more involved with students from their community. Phase 3, scheduled for Spring 2020, will bring the findings from Phases 1 and 2 back to rural communities via two participatory design workshops. These workshops, designed to share our findings and foster collaborative dialogue among the participants, will enable us to explore factors that support or hinder transfer of findings and to identify policies and strategies that would enhance each community’s ability to support engineering as a potential career choice. 
    more » « less
  2. Sudden population influxes in cities place unexpected demands on the urban housing system. During these influxes, decisions made to accommodate displaced persons are often controversial, potentially hindering the ability of organizations involved to respond. Understanding how individuals within those organizations legitimize and delegitimize actions taken to accommodate internationally displaced persons is thus crucial to make decisions that will lead to efficient institutional responses. Existing research relating to the adaptation of urban housing systems for international population influxes in developed countries primarily focus on the long-term response rather than on the short-term response. This study seeks to address this research gap by providing an overview of the perspectives of stakeholders involved in the provision of centralized accommodations for displaced persons during the refugee crisis in 2015 in Germany. A qualitative analysis of interview data was performed to obtain a holistic understanding of the studied institutional response. Twenty-five interviews with employees involved in different steps of the process for providing centralized accommodations for displaced persons were conducted in 2016. Interview content was analyzed to capture the way stakeholders legitimized (1) the provision of centralized accommodations for displaced persons, and (2) the provision of specific types of accommodations commonly used. Results show that interviewed individuals mainly legitimized the process for providing centralized accommodations to displaced persons based on their individual convictions and by using procedural, consequential, influence and exchange legitimacy. They mainly delegitimized this process based on self- interested calculations and by using exchange and influence legitimacy. Finally, results indicate that short-term accommodations, such as sport halls, were the least preferred option, while solutions such as modular housing and the renovation of unused buildings were the most preferred options. 
    more » « less
  3. Researchers, evaluators and designers from an array of academic disciplines and industry sectors are turning to participatory approaches as they seek to understand and address complex social problems. We refer to participatory approaches that collaboratively engage/ partner with stakeholders in knowledge creation/problem solving for action/social change outcomes as collaborative change research, evaluation and design (CCRED). We further frame CCRED practitioners by their desire to move beyond knowledge creation for its own sake to implementation of new knowledge as a tool for social change. In March and May of 2018, we conducted a literature search of multiple discipline-specific databases seeking collaborative, change-oriented scholarly publications. The search was limited to include peerreviewed journal articles, with English language abstracts available, published in the last five years. The search resulted in 526 citations, 236 of which met inclusion criteria. Though the search was limited to English abstracts, all major geographic regions (North America, Europe, Latin America/Caribbean, APAC, Africa and the Middle East) were represented within the results, although many articles did not state a specific region. Of those identified, most studies were located in North America, with the Middle East having only one identified study. We followed a qualitative thematic synthesis process to examine the abstracts of peer-reviewed articles to identify practices that transcend individual disciplines, sectors and contexts to achieve collaborative change. We surveyed the terminology used to describe CCRED, setting, content/topic of study, type of collaboration, and related benefits/outcomes in order to discern the words used to designate collaboration, the frameworks, tools and methods employed, and the presence of action, evaluation or outcomes. Forty-three percent of the reviewed articles fell broadly within the social sciences, followed by 26 percent in education and 25 percent in health/medicine. In terms of participants and/ or collaborators in the articles reviewed, the vast majority of the 236 articles (86%) described participants, that is, those who the research was about or from whom data was collected. In contrast to participants, partners/collaborators (n=32; 14%) were individuals or groups who participated in the design or implementation of the collaborative change effort described. In terms of the goal for collaboration and/or for doing the work, the most frequently used terminology related to some aspect of engagement and empowerment. Common descriptors for the work itself were ‘social change’ (n=74; 31%), ‘action’ (n=33; 14%), ‘collaborative or participatory research/practice’ (n=13; 6%), ‘transformation’ (n=13; 6%) and ‘community engagement’ (n=10; 4%). Of the 236 articles that mentioned a specific framework or approach, the three most common were some variation of Participatory Action Research (n=30; 50%), Action Research (n=40; 16.9%) or Community-Based Participatory Research (n=17; 7.2%). Approximately a third of the 236 articles did not mention a specific method or tool in the abstract. The most commonly cited method/tool (n=30; 12.7%) was some variation of an arts-based method followed by interviews (n=18; 7.6%), case study (n=16; 6.7%), or an ethnographic-related method (n=14; 5.9%). While some articles implied action or change, only 14 of the 236 articles (6%) stated a specific action or outcome. Most often, the changes described were: the creation or modification of a model, method, process, framework or protocol (n=9; 4%), quality improvement, policy change and social change (n=8; 3%), or modifications to education/training methods and materials (n=5; 2%). The infrequent use of collaboration as a descriptor of partner engagement, coupled with few reported findings of measurable change, raises questions about the nature of CCRED. It appears that conducting CCRED is as complex an undertaking as the problems that the work is attempting to address. 
    more » « less
  4. This paper reflects on the significance of ABET’s “maverick evaluators” and what it says about the limits of accreditation as a mode of governance in US engineering education. The US system of engineering education operates as a highly complex system, where the diversity of the system is an asset to robust knowledge production and the production of a varied workforce. ABET Inc., the principal accreditation agency for engineering degree programs in the US, attempts to uphold a set of professional standards for engineering education using a voluntary, peer-based system of evaluation. Key to their approach is a volunteer army of trained program evaluators (PEVs) assigned by the engineering professional societies, who serve as the frontline workers responsible for auditing the content, learning outcomes, and continuous improvement processes utilized by every engineering degree program accredited by ABET. We take a look specifically at those who become labeled “maverick evaluators” in order to better understand how this system functions, and to understand its limitations as a form of governance in maintaining educational quality and appropriate professional standards within engineering education. ABET was established in 1932 as the Engineers’ Council for Professional Development (ECPD). The Cold War consensus around the engineering sciences led to a more quantitative system of accreditation first implemented in 1956. However, the decline of the Cold War and rising concerns about national competitiveness prompted ABET to shift to a more neoliberal model of accountability built around outcomes assessment and modeled after total quality management / continuous process improvement (TQM/CPI) processes that nominally gave PEVs greater discretion in evaluating engineering degree programs. However, conflicts over how the PEVs exercised judgment points to conservative aspects in the structure of the ABET organization, and within the engineering profession at large. This paper and the phenomena we describe here is one part of a broader, interview-based study of higher education governance and engineering educational reform within the United States. We have conducted over 300 interviews at more than 40 different academic institutions and professional organizations, where ABET and institutional responses to the reforms associated with “EC 2000,” which brought outcomes assessment to engineering education, are extensively discussed. The phenomenon of so-called “maverick evaluators” reveal the divergent professional interests that remain embedded within ABET and the engineering profession at large. Those associated with Civil and Environmental Engineering, and to a lesser extent Mechanical Engineering continue to push for higher standards of accreditation grounded in a stronger vision for their professions. While the phenomenon is complex and more subtle than we can summarize in an abstract, “maverick evaluators” emerged as a label for PEVs who interpreted their role, including determinations about whether certain content “appropriate to the field of study,” utilizing professional standards that lay outside of the consensus position held by the majority of the member of the Engineering Accreditation Commission. This, conjoined with the engineers’ epistemic aversion to uncertainty and concerns about the legal liability of their decisions, resulted in a more narrow interpretation of key accreditation criteria. The organization then designed and used a “due-process” reviews process to discipline identified shortcomings in order to limit divergent interpretations. The net result is that the bureaucratic process ABET built to obtain uniformity in accreditation outcomes, simultaneously blunts the organization’s capacity to support varied interpretations of professional standards at the program level. The apparatus has also contributed to ABET’s reputation as an organization focused on minimum standards, as opposed to one that functions as an effective driver for further change in engineering education. 
    more » « less
  5. Abstract

    Essential for society to function, the production and consumption of food, energy, and water (FEW) are deeply intertwined, leading to calls for a nexus approach to understand and manage the complex tradeoffs and cascading effects. What research exists to date on this FEW nexus? How have scholars conceptualized these interactions at the urban scale? What are some promising approaches? Where are the research gaps? To answer these questions, we conducted a quantitative review of the academic literature on the FEW nexus (1399 publications) over more than four decades (1973–2017), followed by in-depth analysis of the most influential papers using an evaluation matrix that examined four components: 1) modeling approach; 2) scale; 3) nexus ‘trigger’; and 4) governance and policy. Scholars in the fields of environmental science predominated, while social science domains were under-represented. Most papers used quantitative rather than qualitative approaches, especially integrated assessment and systems dynamics modeling although spatial scale was generally recognized, explicit consideration of multi-scalar interactions was limited. Issues of institutional structure, governance, equity, resource access, and behavior were also underdeveloped. Bibliometric analysis of this literature revealed six distinct research communities, including a nascent urban FEW community. We replicated the analysis for this urban group, finding it to be just emerging (80% of papers have been published since 2010) and dominated by scholars in industrial ecology. These scholars focus on quantifying FEW flows of the urban metabolism in isolation rather than as a nexus, largely ignoring the political and socio-economic factors shaping these flows. We propose the urban FEW metabolism as a boundary object to draw in diverse scholarly and practitioner communities. This will advance research on complex FEW systems in four key areas: (1) integration of heterogeneous models and approaches; (2) scalar linkages between urban consumption and trans-boundary resource flows; (3) how actors and institutions shape resource access, distribution and use; and (4) co-production of knowledge with stakeholders.

     
    more » « less