More Like this

1. A new approach to grant review assessments: score, then rank

   https://doi.org/10.1186/s41073-023-00131-7  

   Gallo, Stephen A. ; Pearce, Michael ; Lee, Carole J. ; Erosheva, Elena A. ( July 2023 , Research Integrity and Peer Review)

   In many grant review settings, proposals are selected for funding on the basis of summary statistics of review ratings. Challenges of this approach (including the presence of ties and unclear ordering of funding preference for proposals) could be mitigated if rankings such as top-k preferences or paired comparisons, which are local evaluations that enforce ordering across proposals, were also collected and incorporated in the analysis of review ratings. However, analyzing ratings and rankings simultaneously has not been done until recently. This paper describes a practical method for integrating rankings and scores and demonstrates its usefulness for making funding decisions in real-world applications.

   We first present the application of our existing joint model for rankings and ratings, the Mallows-Binomial, in obtaining an integrated score for each proposal and generating the induced preference ordering. We then apply this methodology to several theoretical “toy” examples of rating and ranking data, designed to demonstrate specific properties of the model. We then describe an innovative protocol for collecting rankings of the top-six proposals as an add-on to the typical peer review scoring procedures and provide a case study using actual peer review data to exemplify the output and how the model can appropriately resolve judges’ evaluations.

   For the theoretical examples, we show how the model can provide a preference order to equally rated proposals by incorporating rankings, to proposals using ratings and only partial rankings (and how they differ from a ratings-only approach) and to proposals where judges provide internally inconsistent ratings/rankings and outlier scoring. Finally, we discuss how, using real world panel data, this method can provide information about funding priority with a level of accuracy in a well-suited format for research funding decisions.

   A methodology is provided to collect and employ both rating and ranking data in peer review assessments of proposal submission quality, highlighting several advantages over methods relying on ratings alone. This method leverages information to most accurately distill reviewer opinion into a useful output to make an informed funding decision and is general enough to be applied to settings such as in the NIH panel review process.

    

   more » « less
2. A Community-Driven Process for Developing NSF Review Panelists

   Bates, R. A. ; Benson, L. ; Sims, R. ; Watts, K. ; Jensen, K. ; Ko, E. ; Lichtenstein, G. ( June 2023 , Proceedings ASEE annual conference)

   Peer review of grant proposals is critical to the National Science Foundation (NSF) funding process for STEM disciplinary and education research. Despite this, scholars receive little training in effective and constructive review of proposals beyond definitions of review criteria and an overview of strategies to avoid bias and communicate clearly. Senior researchers often find that their reviewing skills improve and develop over time, but variations in reviewer starting points can have a negative impact on the value of reviews for their intended audiences of program officers, who make funding recommendations, and principal investigators, who drive the research or want to improve their proposals. Building on the journal review component of the Engineering Education Research Peer Review Training (EER PERT) project, which is designed to develop EER scholars’ peer review skills through mentored reviewing experiences, this paper describes a program designed to provide professional development for proposal reviewing and provides initial evaluation results. 

   more » « less
3. Climate change and infectious disease: a review of evidence and research trends

   https://doi.org/10.1186/s40249-023-01102-2  

   Van de Vuurst, Paige ; Escobar, Luis E. ( May 2023 , Infectious Diseases of Poverty)

   Climate change presents an imminent threat to almost all biological systems across the globe. In recent years there have been a series of studies showing how changes in climate can impact infectious disease transmission. Many of these publications focus on simulations based on in silico data, shadowing empirical research based on field and laboratory data. A synthesis work of empirical climate change and infectious disease research is still lacking.

   We conducted a systemic review of research from 2015 to 2020 period on climate change and infectious diseases to identify major trends and current gaps of research. Literature was sourced from Web of Science and PubMed literary repositories using a key word search, and was reviewed using a delineated inclusion criteria by a team of reviewers.

   Our review revealed that both taxonomic and geographic biases are present in climate and infectious disease research, specifically with regard to types of disease transmission and localities studied. Empirical investigations on vector-borne diseases associated with mosquitoes comprised the majority of research on the climate change and infectious disease literature. Furthermore, demographic trends in the institutions and individuals published revealed research bias towards research conducted across temperate, high-income countries. We also identified key trends in funding sources for most resent literature and a discrepancy in the gender identities of publishing authors which may reflect current systemic inequities in the scientific field.

   Future research lines on climate change and infectious diseases should considered diseases of direct transmission (non-vector-borne) and more research effort in the tropics. Inclusion of local research in low- and middle-income countries was generally neglected. Research on climate change and infectious disease has failed to be socially inclusive, geographically balanced, and broad in terms of the disease systems studied, limiting our capacities to better understand the actual effects of climate change on health.

    

   more » « less
4. Understanding and Addressing Gender‐Based Inequities in STEM: Research Synthesis and Recommendations for U.S. K‐12 Education

   https://doi.org/10.1111/sipr.12087  

   Kuchynka, Sophie L. ; Eaton, Asia ; Rivera, Luis M. ( January 2022 , Social Issues and Policy Review)

   We draw from ecological systems and social psychological theories to elucidate macrosystem‐ and microsystem‐level variables that promote and maintain gender inequities in science, technology, engineering, and math (STEM). Because gender‐STEM stereotypes undermine girls’ (and women's), but boosts boys’ (and men's), STEM interest and success, we review how they operate in STEM learning environments to differentially socialize girls and boys and undermine gender integroup relations. We propose seven practice recommendations to improve STEM K‐12 education: (1) design relational classrooms, (2) teach the history of gender inequality and bias, (3) foster collaborative and cooperative classrooms, (4) promote active learning and growth mindset strategies, (5) reframing STEM as inclusive, (6) create near‐peer mentorship programs, and (7) re‐imagine evaluation metrics. To support these practice recommendations, three policy recommendations are posited: (1) increase teacher autonomy, training, and representation, (2) re‐evaluate standardized testing, and (3) reallocate and increase government funding for public schools.

    

   more » « less
5. Legal Feasibility and Implementation of Federal Strategies for a National Retail‐Based Fruit and Vegetable Subsidy Program in the United States

   https://doi.org/10.1111/1468-0009.12461  

   POMERANZ, JENNIFER L. ; HUANG, YUE ; MOZAFFARIAN, DARIUSH ; MICHA, RENATA ( July 2020 , The Milbank Quarterly)

   Suboptimal intake of fruit and vegetables is associated with increased risk of diet‐related diseases. A national retail‐based fruit and vegetable subsidy program could broadly benefit the health of the entire population.

   Existing fruit and vegetable subsidy programs can inform potential implementation mechanisms; Congress's powers to tax, spend, and regulate interstate commerce can be leveraged to create a federal program.

   Legal and administrative feasibility considerations support a conditional funding program or a federal‐state cooperative program combining regulation, licensing, and state or local options for flexible implementation strategies. Strategies to engage key stakeholders would enable the program to utilize lessons learned from existing programs.

   Suboptimal intake of fruit and vegetables (F&Vs) is associated with increased risk of diet‐related diseases. Yet, there are no US government programs to support increased F&V consumption nationally for the whole population, most of whom purchase food at retail establishments. To inform policy discussion and implementation, we identified mechanisms to effectuate a national retail‐based F&V subsidy program.

   We conducted legal and policy research using LexisNexis, the UConn Rudd Center Legislation Database, the Centers for Disease Control and Prevention Chronic Disease State Policy Tracking System, the US Department of Agriculture's website, Congress.gov, gray literature, and government reports. First, we identified existing federal, state, local, and nongovernmental organization (NGO) policies and programs that subsidize F&Vs. Second, we evaluated Congress's power to implement a national retail‐based F&V subsidy program.

   We found five federal programs, three federal bills, four state laws, and 17 state (including the District of Columbia [DC]) bills to appropriate money to supplement federal food assistance programs with F&Vs; 74 programs (six multistate, 22 state [including DC], and 46 local) administered by state and local governments and NGOs that incentivize the purchase of F&Vs for various subpopulations; and two state laws and 11 state bills to provide tax exemptions for F&Vs. To create a national F&V subsidy program, Congress could use its Commerce Clause powers or its powers to tax or spend, through direct regulation, licensing, taxation, tax incentives, and conditional funding. Legal and administrative feasibility considerations support a voluntary conditional funding program or, as a second option, a mandatory federal‐state cooperative program combining regulation and licensing.

   Multiple existing programs provide an important foundation to inform potential implementation mechanisms for a national F&V subsidy program. Results also highlight the value of state and local participation to leverage existing networks and stakeholder knowledge.

    

   more » « less