This is the first of a series of studies that explore the relationship between disciplinary background and the weighting of various elements of a manuscript in peer reviewers’ determination of publication recommendations. Research questions include: (1) To what extent are tacit criteria for determining quality or value of EER manuscripts influenced by reviewers’ varied disciplinary backgrounds and levels of expertise? and (2) To what extent does mentored peer review professional development influence reviewers’ EER manuscript evaluations? Data were collected from 27 mentors and mentees in a peer review professional development program. Participants reviewed the same two manuscripts, using a form to identify strengths, weaknesses, and recommendations. Responses were coded by two researchers (70% IRR). Our findings suggest that disciplinary background influences reviewers’ evaluation of EER manuscripts. We also found evidence that professional development can improve reviewers’ understanding of EER disciplinary conventions. Deeper understanding of the epistemological basis for manuscript reviews may reveal ways to strengthen professional preparation in engineering education as well as other disciplines.
more »
« less
Methods for Conducting a Scoping Literature Review on Institutional Culture and Transformational Change in Engineering Education
There are a variety of urgent calls for institutional initiatives and actions to transform engineering education. For a transformational change to occur, the initiatives must alter the culture of the institutions (Eckel, Hill, and Green, 1998). In this work in progress, we detail the methods used to conduct a scoping literature review (ScR) concerning the current state of the literature surrounding institutional culture and transformational change in engineering education at institutions of higher learning in the United States. As institutional culture and transformational change are currently underexplored topics in the engineering education literature, we investigated the larger body of computer science and engineering literature in the United States. Once completed, this study aims to reveal the current trends, theories, and potential gaps in the literature regarding these topics.
Arksey and O’Malley’s methodology for conducting scoping reviews informed the development of our scoping review protocol, which similarly includes five stages: (1) identify the research questions, (2) identify relevant studies, (3) select relevant studies, (4) chart the data, and (5) collate, summarize, and report results (Arksey and O’Malley, 2005). University librarians who specialize in conducting systematic reviews aided in the refinement of this protocol. From the research question and aim of the study, three main inclusion criteria were created: (1) the literature must discuss both organizational culture and transformational change, (2) discussion of transformational change must describe the institution where the change happened, and (3) the literature must emphasize the agents of transformational change. Additional inclusion and exclusion criteria were created in collaboration with both the librarians and reviewers. These criteria guided the search for existing literature in the following online databases: Elsevier (Engineering Village – Compendex and Engineering Village – INSPEC), ProQuest (ERIC and Education Database), Scopus, and Web of Science. These six databases were selected as they often include publications relevant to the field of engineering education. After the search was conducted, the inclusion and exclusion criteria were turned into questions to inform a three-step screening process (title, abstract, and full text) used by reviewers to determine whether a publication was eligible for the study. Reviewers were assigned to review papers through Covidence, a cloud-based systematic literature review management platform. There are currently two primary reviewers and a third additional reviewer to resolve any conflicts or disagreements if they should arise. Before each review cycle, the inclusion and exclusion criteria are revisited, revised, and agreed upon by the three reviewers. This screening process is performed iteratively, allowing for critical reflection at each stage to drive the resulting findings by the reviewers in consultation with content matter experts.
We are currently conducting our first round of screening in the study selection (third stage) of the scoping review protocol. After the removal of duplicates, 999 publications were found by searching in the six selected databases. This number is expected to be further reduced with each step of the screening process. When this scoping review is complete, the resulting publication will contain an analysis of the literature and synthesis of our findings, and present the prominent themes, theories, and potential gaps in the literature. This publication is expected to unite disparate lines of research on institutional culture and transformational change, challenge the assumptions in the field, and change the way engineering education views transformational change.
more »
« less
- Award ID(s):
- 2106206
- PAR ID:
- 10436863
- Date Published:
- Journal Name:
- American Society for Engineering Education, 2023
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This is the first of a series of studies that explore the relationship between disciplinary background and the weighting of various elements of a manuscript in peer reviewers’ determination of publication recommendations. Research questions include: (1) To what extent are tacit criteria for determining quality or value of EER manuscripts influenced by reviewers’ varied disciplinary backgrounds and levels of expertise? and (2) To what extent does mentored peer review professional development influence reviewers’ EER manuscript evaluations? Data were collected from 27 mentors and mentees in a peer review professional development program. Participants reviewed the same two manuscripts, using a form to identify strengths, weaknesses, and recommendations. Responses were coded by two researchers (70% IRR). Our findings suggest that disciplinary background influences reviewers’ evaluation of EER manuscripts. We also found evidence that professional development can improve reviewers’ understanding of EER disciplinary conventions. Deeper understanding of the epistemological basis for manuscript reviews may reveal ways to strengthen professional preparation in engineering education as well as other disciplines.more » « less
-
more » « less
Abstract Computational thinking is of growing interest to the science, technology, engineering, and mathematics (STEM) education research community. Calls from national agencies look to increase computation in STEM education. This review identifies key areas for future study by reviewing recent empirical studies that investigate computational thinking in teaching and learning contexts within higher education. Using a systematic process, this review identified four different databases for peer‐reviewed research articles using keywords. Results were evaluated against inclusion and exclusion criteria. Studies were analyzed for types of methods, target population, the role of computational thinking, pedagogical designs used, and significant findings of the study. This process resulted in a final set of 13 studies. The results indicate that computational thinking research in higher education is growing, yet there are opportunities for more research. The findings of this study highlight the need for more concrete definitions and implementations of computational thinking within higher education spaces.
-
This full paper sets out a methodological protocol for conducting a scoping review of literature relating to teamwork effectiveness and equity. The goal of the study is to understand how academic discourse over the five-year period prior to the study being carried out has conceptualized teamwork success in educational and professional contexts, and to what extent equitable team practices are embedded within such conceptualizations. In line with ongoing initiatives to promote transparency in research, this protocol paper is intended for dissemination prior to the conduct of the study itself. The research context, questions, and rationale are set out, and a detailed methodology described, outlining procedures for data retrieval, screening, extraction, and analysis. The paper concludes with an outline of intended reporting methods for the study, including the reporting of deviations from the procedures set out herein. This paper contributes to the scoping review methodology, and especially its application in the field of engineering education research and education research more broadly.more » « less
-
This full paper sets out a methodological protocol for conducting a scoping review of literature relating to teamwork effectiveness and equity. The goal of the study is to understand how academic discourse over the five-year period prior to the study being carried out has conceptualized teamwork success in educational and professional contexts, and to what extent equitable team practices are embedded within such conceptualizations. In line with ongoing initiatives to promote transparency in research, this protocol paper is intended for dissemination prior to the conduct of the study itself. The research context, questions, and rationale are set out, and a detailed methodology described, outlining procedures for data retrieval, screening, extraction, and analysis. The paper concludes with an outline of intended reporting methods for the study, including the reporting of deviations from the procedures set out herein. This paper contributes to the scoping review methodology, and especially its application in the field of engineering education research and education research more broadly.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
