skip to main content


Title: Assessing Broader Impacts
ABSTRACT The National Alliance for Broader Impacts (NABI) seeks to foster a community of practice that increases individual and institutional capacity for, and engagement in, broader impact (BI) activities and scholarship. NABI currently has 537 individual members representing more than 210 institutions and organizations who are part of the growing network of professionals. The National Science Foundation (NSF) evaluates all proposals on their intellectual merit and their broader impacts. Many investigators grapple with how to articulate and effectively engage broad audiences in materials science and STEM. Here, we describe the effort of NABI to address BI challenges, present the NABI document Broader Impacts, Guiding Principles and Questions for National Science Foundation Principal Investigators and Proposal Reviewers; highlight the impacts of NABI as a catalyst for building BI capacity; and provide an example of assessing an innovative program’s BI.  more » « less
Award ID(s):
1663296
NSF-PAR ID:
10028113
Author(s) / Creator(s):
;
Date Published:
Journal Name:
MRS Advances
Volume:
2
Issue:
31-32
ISSN:
2059-8521
Page Range / eLocation ID:
1681 to 1686
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This theory paper describes the development and use of a framework for supporting early career faculty development, especially in competitive National Science Foundation (NSF) CAREER proposals. Engineering Education Research (EER) has developed into a field of expertise and a career pathway over the past three decades. In response to numerous reports in the 1990s and early 2000s, multiple EER graduate programs were established in the mid-2000s and a growing number continue to emerge to educate and train the next generation of EER faculty and policy makers. Historically, many came to EER as individuals trained in other disciplines, but with an interest in improving teaching and learning. This approach created an interdisciplinary space where many could learn the norms, practices, and language of EER, as they became scholars. This history combined with the emergence of EER as a discipline with academic recognition; specific knowledge, frameworks, methodologies, and ways of conducting research; and particular emphasis and goals, creates a tension for building capacity to continue to develop EER and also include engineering education researchers who have not completed PhDs in an engineering education program. If EER is to continue to develop and emerge as a strong and robust discipline with high quality engineering education research, support mechanisms must be developed to both recognize outstanding EER scholars and develop the next generation of researchers in the field. The Five I’s framework comes from a larger project on supporting early career EER faculty in developing NSF CAREER proposals. Arguably, a NSF CAREER award is significant external recognition of EER that signals central membership in the community. The Five I’s were developed using collaborative inquiry, a tool and process to inform practice, with 19 EER CAREER awardees during a retreat in March 2019. The Five I’s include: Ideas, Integration, Impact, Identity, and Infrastructure. Ideas is researchers’ innovative and potentially transformative ideas that can make a significant contribution to EER. All NSF proposals are evaluated using the criteria of intellectual merit and broader impacts, and ideas aligned with these goals are essential for funding success. The integration of research and education is a specific additional consideration of CAREER proposals. Both education and research must inform one another in the proposal process. Demonstrating the impact of research is essential to convey why research should be funded. This impact is essential to address as it directly relates to the NSF criteria of broader impacts as well as why an individual is positioned to carry out that impact. This positioning is tied to identity or the particular research expertise from which a faculty member will be a leader in the field. Finally, infrastructure includes the people and physical resources from which a faculty member must draw to be successful. This framework has proven useful in helping early career faculty evaluate their readiness to apply for an NSF CAREER award or highlight the particular areas of their development that could be improved for future success. 
    more » « less
  2. Abstract

    Broader impacts (BI) policies generate debate on the purpose of science, measuring the impact of research, and is an important topic for the science policy community. However, BI policies often fail to determine if R&D funding helps marginalized communities. This paper introduces a new framework, the Inclusion-Immediacy Criterion, that assesses who benefits from research impacts as divided into three groups: (1) advantaged groups; (2) the general population; and (3) marginalized groups. The study analyzes National Science Foundation (NSF) project outcome reports and finds that advantaged groups are the most likely to benefit from NSF-funded research. The study also shows that certain areas of NSF research, such as Social, Behavioral, and Economic Sciences, more efficiently generate impacts for marginalized groups compared to other directorates. This paper further argues that persistent inequalities in BIs limit the potential of R&D to increase prosperity and well-being, two of NSF’s mandated goals.

     
    more » « less
  3. With the help of the National Science Foundation (NSF), many Principal Investigators (PIs) have been able to mentor undergraduates through Research Experience for Undergraduates (REU) site awards. These REU sites are critical to the development of future graduate students, but can be challenging to run due to several required skills outside the scope of most faculty members' expertise, e.g., recruiting applicants, navigating the logistics of housing visiting undergraduate students, and tracking student outcomes after their REU experiences. In recent years, REU PIs in NSF's Computer & Information Science & Engineering (CISE) Directorate have come together through PI meetings to share best practices for running a successful REU site. While PIs inevitably take different approaches to running their sites based on their research projects, there is still a need to provide new PIs with guidance on the different aspects of an REU site such as identifying resources that can assist in recruiting women and underrepresented minority applicants, providing training for graduate students acting as mentors, and strategies for keeping a mentoring connection to undergraduate researchers after they return to their home institutions. Currently, REU site preparation and orientation for new PIs is a face-to-face process that requires careful planning and significant travel costs. The REU PI Guide, a set of web-based resources at https://www.vrac.iastate.edu/cise-reu-pi-resources/, was developed to share best practices of experienced PIs and build capacity within the REU PI community in a more scalable and cost-effective way. The REU PI Guide allows PIs to look up advice and guidance when needed and share their own best practices. This paper describes our approach to designing the REU PI Guide. The Guide is a database of documents, examples, and overviews of the different aspects of running an REU site. The Guide was developed by assessing new PIs' needs at an NSF workshop for new PIs, gathering existing resources from experienced PIs, creating and refining a website, and evaluation with new PIs. The website’s content and design will be refined through on-going feedback from PIs and other REU site stakeholders. This site has the potential broader impact to share best practices with REU PIs outside the CISE directorate and significantly ease the process of engaging future scientists via REU sites. 
    more » « less
  4. Abstract

    It is important for funding agencies to evaluate if scientists accomplish their research goals. By comparing a representative sample of National Science Foundation abstracts and project outcome reports (PORs) from 2014 to 2017, this article investigates whether scientists attain the broader impacts they propose. We find that the number of broader impacts proposed in the abstracts is significantly higher than the number of broader impacts reported in the PORs. The trend is common across directorates and type of impact, except when impacts serve advantaged groups. Only the number of broader impacts for advantaged groups increases from the abstract to the POR. Despite the difference between proposed impact and reported impact, our study does not conclude that scientists are delinquent or disingenuous when they propose their research. Rather, we question the capacity of current frameworks to capture the quality of impacts and to weigh the relative importance of impacts that serve marginalized groups versus those that sustain the status quo.

     
    more » « less
  5. The HSI (Hispanic Serving Institution) ATE (Advanced Technological Education) Hub 2 is a three-year collaborative research project funded by the National Science Foundation (NSF) that continues the partnership between two successful programs and involves a third partner in piloting professional development that draws upon findings from the initial program. The goal of HSI ATE Hub 2 is to improve outcomes for Latinx students in technician education programs through design, development, pilot delivery, and dissemination of a 3-tier professional development (PD) model for culturally responsive technician education at 2-year Hispanic Serving Institutions (HSIs). The project seeks to do this by developing the awareness and ability of faculty to appreciate, engage, and affirm the unique cultural identities of the students in their classes and use this connection to deepen students’ belonging and emerging identities as STEM learners and future STEM technicians. This paper shares the research foundations shaping this approach and the methods by which faculty professional development is being provided to develop this important and sensitive instructional capability in participating faculty. The tiered PD model features a scaffolded series of reflective and activity-oriented modules to incrementally enrich the instructional practices and mindset of HSI STEM educators and strengthen their repertoire of strategies for engaging culturally diverse students. Scaffolding that translates culturally responsive theory to practice spans each of the four distinct topic modules in each tier. Each topic module in a tier then scaffolds to a more advanced topic module in the next tier. Tier 1, Bienvenidos, welcomes HSI STEM educators who recognize the need to better serve their Latinx students, and want guidance for small practical activities to try with their students. Tier 2, Transformation through Action, immerses HSI STEM educators in additional activities that bring culturally responsive practices into their technician training while building capacity to collect evidence about impacts and outcomes for students. Tier 3, Engaging Community, strengthens leadership as HSI STEM educators disseminate results from activities completed in Tiers 1 and 2 at conferences that attract technician educators. Sharing the evidence-based practices and their outcomes contributes to achieving broader impacts in the Advanced Technological Education or ATE Community of NSF grantees. Westchester Community College (WCC), the first 2-year HSI in the State University of New York (SUNY) 64 campus system, is piloting the 3-tier PD model using virtual learning methods mastered through previous NSF ATE work and the COVID-19 context. During the pilot, over 20 WCC technician educators in three cohorts will develop leadership skills and practice culturally responsive methods. The pilot will build capacity within WCC STEM technician programs to better support the diversity of students, industry demand for a diverse workforce, and WCC’s capacity for future development of technician education programs. This first paper in a three part series describes the program goals and objectives, the 3-Tier PD model, and reports initial results for Cohort A’s engagement in the first three modules of Tier 1. 
    more » « less