Search for: All records

This paper describes the Engineering Education Research (EER) Peer Review Training (PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences. Supported by the National Science Foundation, the overall programmatic goals of the PERT project are to establish and evaluate a mentored reviewer program for 1) EER journal manuscripts and 2) EER grant proposals. Concurrently, the project seeks to explore how EER scholars develop schema for evaluating EER scholarship, whether these schema are shared in the community, and how schema influence recommendations made to journal editors during the peer review process. To accomplish these goals, the PERT project leveraged the previously established Journal of Engineering Education (JEE) Mentored Reviewer Program, where two researchers with little reviewing experience are paired with an experienced mentor to complete three manuscript reviews collaboratively. In this paper we report on focus group and exit survey findings from the JEE Mentored Reviewer Program and discuss revisions to the program in response to those findings. 

This is the first of a series of studies that explore the relationship between disciplinary background and the weighting of various elements of a manuscript in peer reviewers’ determination of publication recommendations. Research questions include: (1) To what extent are tacit criteria for determining quality or value of EER manuscripts influenced by reviewers’ varied disciplinary backgrounds and levels of expertise? and (2) To what extent does mentored peer review professional development influence reviewers’ EER manuscript evaluations? Data were collected from 27 mentors and mentees in a peer review professional development program. Participants reviewed the same two manuscripts, using a form to identify strengths, weaknesses, and recommendations. Responses were coded by two researchers (70% IRR). Our findings suggest that disciplinary background influences reviewers’ evaluation of EER manuscripts. We also found evidence that professional development can improve reviewers’ understanding of EER disciplinary conventions. Deeper understanding of the epistemological basis for manuscript reviews may reveal ways to strengthen professional preparation in engineering education as well as other disciplines. 

This paper describes a peer reviewer mentoring program called the Engineering Education Research Peer Review Training (EER PERT) project and serves as a pilot study on longitudinal effects on researchers’ productivity and the impact of their work, differences in these factors for those who review journal manuscripts and those who review grant proposals, and what aspects of peer review training (knowledge, resources, collaborations, etc.) participants actually carry forward in their own research. Overall, the project seeks to investigate how engineering education research (EER) scholars develop skills and schema for reviewing scholarship, particularly in terms of developing constructive reviews that build expertise and advance knowledge. The Journal of Engineering Education (JEE) Mentored Reviewer Program constitutes the first phase of the EER PERT project. In this paper, we report on goals, structure and activities for the JEE Mentored Reviewer Program, pilot data from participants’ applications and exit surveys that will inform the EER PERT project in terms of participants’ developing skills and schema for reviewing and conducting EER, and provide initial suggestions from the training program that may benefit scholars new to EER.