This paper describes a peer reviewer mentoring program called the Engineering Education Research Peer Review Training (EER PERT) project and serves as a pilot study on longitudinal effects on researchers’ productivity and the impact of their work, differences in these factors for those who review journal manuscripts and those who review grant proposals, and what aspects of peer review training (knowledge, resources, collaborations, etc.) participants actually carry forward in their own research. Overall, the project seeks to investigate how engineering education research (EER) scholars develop skills and schema for reviewing scholarship, particularly in terms of developing constructive reviews that build expertise and advance knowledge. The Journal of Engineering Education (JEE) Mentored Reviewer Program constitutes the first phase of the EER PERT project. In this paper, we report on goals, structure and activities for the JEE Mentored Reviewer Program, pilot data from participants’ applications and exit surveys that will inform the EER PERT project in terms of participants’ developing skills and schema for reviewing and conducting EER, and provide initial suggestions from the training program that may benefit scholars new to EER.
more »
« less
This content will become publicly available on June 26, 2024
A Community-Driven Process for Developing NSF Review Panelists
Peer review of grant proposals is critical to the National Science Foundation (NSF) funding
process for STEM disciplinary and education research. Despite this, scholars receive little
training in effective and constructive review of proposals beyond definitions of review criteria
and an overview of strategies to avoid bias and communicate clearly. Senior researchers often
find that their reviewing skills improve and develop over time, but variations in reviewer starting
points can have a negative impact on the value of reviews for their intended audiences of
program officers, who make funding recommendations, and principal investigators, who drive
the research or want to improve their proposals. Building on the journal review component of the
Engineering Education Research Peer Review Training (EER PERT) project, which is designed
to develop EER scholars’ peer review skills through mentored reviewing experiences, this paper
describes a program designed to provide professional development for proposal reviewing and
provides initial evaluation results.
more »
« less
- Award ID(s):
- 2037807
- NSF-PAR ID:
- 10472890
- Publisher / Repository:
- ASEE
- Date Published:
- Journal Name:
- Proceedings ASEE annual conference
- ISSN:
- 0190-1052
- Format(s):
- Medium: X
- Location:
- Baltimore, MD
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)This paper describes a peer reviewer mentoring program called the Engineering Education Research Peer Review Training (EER PERT) project and serves as a pilot study on longitudinal effects on researchers’ productivity and the impact of their work, differences in these factors for those who review journal manuscripts and those who review grant proposals, and what aspects of peer review training (knowledge, resources, collaborations, etc.) participants actually carry forward in their own research. Overall, the project seeks to investigate how engineering education research (EER) scholars develop skills and schema for reviewing scholarship, particularly in terms of developing constructive reviews that build expertise and advance knowledge. The Journal of Engineering Education (JEE) Mentored Reviewer Program constitutes the first phase of the EER PERT project. In this paper, we report on goals, structure and activities for the JEE Mentored Reviewer Program, pilot data from participants’ applications and exit surveys that will inform the EER PERT project in terms of participants’ developing skills and schema for reviewing and conducting EER, and provide initial suggestions from the training program that may benefit scholars new to EER.more » « less
-
null (Ed.)This paper describes a peer reviewer mentoring program called the Engineering Education Research Peer Review Training (EER PERT) project and serves as a pilot study on longitudinal effects on researchers’ productivity and the impact of their work, differences in these factors for those who review journal manuscripts and those who review grant proposals, and what aspects of peer review training (knowledge, resources, collaborations, etc.) participants actually carry forward in their own research. Overall, the project seeks to investigate how engineering education research (EER) scholars develop skills and schema for reviewing scholarship, particularly in terms of developing constructive reviews that build expertise and advance knowledge. The Journal of Engineering Education (JEE) Mentored Reviewer Program constitutes the first phase of the EER PERT project. In this paper, we report on goals, structure and activities for the JEE Mentored Reviewer Program, pilot data from participants’ applications and exit surveys that will inform the EER PERT project in terms of participants’ developing skills and schema for reviewing and conducting EER, and provide initial suggestions from the training program that may benefit scholars new to EER.more » « less
-
This paper describes the Engineering Education Research (EER) Peer Review Training (PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences. Supported by the National Science Foundation, the overall programmatic goals of the PERT project are to establish and evaluate a mentored reviewer program for 1) EER journal manuscripts and 2) EER grant proposals. Concurrently, the project seeks to explore how EER scholars develop schema for evaluating EER scholarship, whether these schema are shared in the community, and how schema influence recommendations made to journal editors during the peer review process. To accomplish these goals, the PERT project leveraged the previously established Journal of Engineering Education (JEE) Mentored Reviewer Program, where two researchers with little reviewing experience are paired with an experienced mentor to complete three manuscript reviews collaboratively. In this paper we report on focus group and exit survey findings from the JEE Mentored Reviewer Program and discuss revisions to the program in response to those findings.more » « less
-
This paper describes the Engineering Education Research (EER) Peer Review Training (PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences. Supported by the National Science Foundation, the overall programmatic goals of the PERT project are to establish and evaluate a mentored reviewer program for 1) EER journal manuscripts and 2) EER grant proposals. Concurrently, the project seeks to explore how EER scholars develop schema for evaluating EER scholarship, whether these schema are shared in the community, and how schema influence recommendations made to journal editors during the peer review process. To accomplish these goals, the PERT project leveraged the previously established Journal of Engineering Education (JEE) Mentored Reviewer Program, where two researchers with little reviewing experience are paired with an experienced mentor to complete three manuscript reviews collaboratively. In this paper we report on focus group and exit survey findings from the JEE Mentored Reviewer Program and discuss revisions to the program in response to those findings.more » « less