An official website of the United States government
While there is evidence to support the existence of identity-based disparities, inequities, and biases in the academic journal peer-review process, little research supports the presence of this bias in the peer-review process for academic journals in science education. Through an analysis of six leading journals in science education, we aimed to investigate the extent to which diversity, equity, and inclusion (DEI), as well as the presence of bias in the peer-review process, are addressed by these journals. We analyzed trends in the gender/sex, geographical affiliation, race/ethnicity, and the presence of equity-centered research focus for members of these journals' editors and editorial boards. We found that although gender/sex is well-balanced in these journals' editors and editorial boards, they are typically North American centric, and White individuals are overwhelmingly represented. Four journals had a quarter or more of individuals who pursue equity-centered research. Only two journals provided detailed information on how manuscripts are reviewed in their author submission guidelines. All used a double-blind approach to peer-review. One of the journals includes an explicit position on DEI. Based on the analyses and reflections on our own experiences, we recommend science education journals consider ways to probe whether bias does exist in their peer-review process, diversify their board to be more inclusive of scholars from communities historically marginalized, and move to a triple-blind approach to their peer-review process as mechanisms to mitigate bias in the journal peer review.
more »
« less
“It is becoming increasingly difficult to find reviewers”—myths and facts about peer review
Abstract A frequent complaint of editors of scientific journals is that it has become increasingly difficult to find reviewers for evaluating submitted manuscripts. Such claims are, most commonly, based on anecdotal evidence. To gain more insight grounded on empirical evidence, editorial data of manuscripts submitted for publication to the Journal of Comparative Physiology A between 2014 and 2021 were analyzed. No evidence was found that more invitations were necessary over time to get manuscripts reviewed; that the reviewer’s response time after invitation increased; that the number of reviewers who completed their reports, relative to the number of reviewers who had agreed to review a manuscript, decreased; and that the recommendation behavior of reviewers changed. The only significant trend observed was among reviewers who completed their reports later than agreed. The average number of days that these reviewers submitted their evaluations roughly doubled over the period analyzed. By contrast, neither the proportion of late vs. early reviews, nor the time for completing the reviews among the punctual reviewers, changed. Comparison with editorial data from other journals suggests that journals that serve a smaller community of readers and authors, and whose editors themselves contact potential reviewers, perform better in terms of reviewer recruitment and performance than journals that receive large numbers of submissions and use editorial assistants for sending invitations to potential reviewers.
more »
« less
- Award ID(s):
- 1946910
- PAR ID:
- 10422821
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- Journal of Comparative Physiology A
- ISSN:
- 0340-7594
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The scientific publishing landscape is evolving rapidly. This evolution is driven by a confluence of internal and external forces, including the growth of metrics‐based evaluation of scientists; an increasing volume of manuscripts combined with expectations for rapid review and publication; an increasing number of journals, includingfor‐profitOpen Access publications; and the adoption of preprint servers across a growing range of disciplines. Many of these forces are contributing to personal anxiety and fatigue for authors, reviewers, and editors. Collectively, they are placing substantial stress on scientific publishing, which is a foundational pillar of the scientific enterprise. As editors of American Geophysical Union journals and books, we remain confident in the fundamental foundations of scientific publishing, but we are concerned about the impact of these increasing stressors. By affirming and investing in editorial values, respecting scientific integrity and credibility, and committing to accessibility, transparency, and accountability, we can fortify the foundations of the scientific enterprise during a time of rapid change.more » « less
-
Casadevall, Arturo (Ed.)ABSTRACT In this editorial, written by early-career scientists, we advocate for the invaluable role of society journals in our scientific community. By choosing to support these journals as authors, peer reviewers, and as editors, we can reinforce our academic growth and benefit from their re-investment back into the scientific ecosystem. Considering the numerous clear merits of this system for future generations of microbiologists and more broadly, society, we argue that early-career researchers should publish our high-quality research in society journals to shape the future of science and scientific publishing landscape.more » « less
-
This research paper study was situated within a peer review mentoring program in which novice reviewers were paired with mentors who are former National Science Foundation (NSF) program directors with experience running discipline-based education research (DBER) panels. Whether it be a manuscript or grant proposal, the outcome of peer review can greatly influence academic careers and the impact of research on a field. Yet the criteria upon which reviewers base their recommendations and the processes they follow as they review are poorly understood. Mentees reviewed three previously submitted proposals to the NSF and drafted pre-panel reviews regarding the proposals’ intellectual merit and broader impacts, strengths, and weaknesses relative to solicitation-specific criteria. After participation in one mock review panel, mentees could then revise their pre-review evaluations based on the panel discussion. Using a lens of transformative learning theory, this study sought to answer the following research questions: 1) What are the tacit criteria used to inform recommendations for grant proposal reviews among scholars new to the review process? 2) To what extent are there changes in these tacit criteria and subsequent recommendations for grant proposal reviews after participation in a mock panel review? Using a single case study approach to explore one mock review panel, we conducted document analyses of six mentees’ reviews completed before and after their participation in the mock review panel. Findings from this study suggest that reviewers primarily focus on the positive broader impacts proposed by a study and the level of detail within a submitted proposal. Although mentees made few changes to their reviews after the mock panel discussion, changes which were present illustrate that reviewers more deeply considered the broader impacts of the proposed studies. These results can inform review panel practices as well as approaches to training to support new reviewers in DBER fields.more » « less
-
This research paper study was situated within a peer review mentoring program in which novice reviewers were paired with mentors who are former National Science Foundation (NSF) program directors with experience running discipline-based education research (DBER) panels. Whether it be a manuscript or grant proposal, the outcome of peer review can greatly influence academic careers and the impact of research on a field. Yet the criteria upon which reviewers base their recommendations and the processes they follow as they review are poorly understood. Mentees reviewed three previously submitted proposals to the NSF and drafted pre-panel reviews regarding the proposals’ intellectual merit and broader impacts, strengths, and weaknesses relative to solicitation-specific criteria. After participation in one mock review panel, mentees could then revise their pre-review evaluations based on the panel discussion. Using a lens of transformative learning theory, this study sought to answer the following research questions: 1) What are the tacit criteria used to inform recommendations for grant proposal reviews among scholars new to the review process? 2) To what extent are there changes in these tacit criteria and subsequent recommendations for grant proposal reviews after participation in a mock panel review? Using a single case study approach to explore one mock review panel, we conducted document analyses of six mentees’ reviews completed before and after their participation in the mock review panel. Findings from this study suggest that reviewers primarily focus on the positive broader impacts proposed by a study and the level of detail within a submitted proposal. Although mentees made few changes to their reviews after the mock panel discussion, changes which were present illustrate that reviewers more deeply considered the broader impacts of the proposed studies. These results can inform review panel practices as well as approaches to training to support new reviewers in DBER fields.more » « less
