skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


This content will become publicly available on November 13, 2025

Title: The Reliability and Validity of the ARIS Broader Impacts Rubric
More than 27 years have passed since the National Science Board identified Broader Impacts as one of two merit criteria for National Science Foundation proposals. Yet many researchers remain less certain of how to develop, implement, and assess a broader impact plan. This multi-method study of a Broader Impacts (BI) rubric analyzed expert panels that included BI professionals and researchers for both content validity and reliability. Focus groups with researchers explicate the challenges researchers face regarding BI plans and the potential value of the rubric as a tool for use. It revealed the challenges researchers have in weighing proven strategies versus innovative strategies, a bias documented by other scholars. Researchers stated concern with how to weigh the different facets of the rubric to arrive at a single score. Moreover, researchers reported that their disciplinary field influenced how they interpreted the audiences whose needs and interests may be met through BI plans. These distinctions represent a range of different types of community engaged scholarship (e.g., public information network, community-campus partnership, K-12 schools’ partnerships). Finally, researchers found the BI rubric useful in evaluating and developing their own BI plans as well as their role in panels to ultimately strengthen the field of funded BI work.  more » « less
Award ID(s):
2140950
PAR ID:
10644187
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
University of Alabama Division of Community Affairs
Date Published:
Journal Name:
Journal of Community Engagement and Scholarship
Volume:
17
Issue:
2
ISSN:
1944-1207
Page Range / eLocation ID:
3
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Journal of Community Engagement and Scholarship)
    The Broader Impacts (BI) activities required of researchers funded by the National Science Foundation (NSF) often involve public engagement, including K-12 outreach, informal science education, public exhibits and performances, advocacy and policy change, and business and entrepreneurship. The ARIS Broader Impacts Toolkit (Advancing Research Impacts in Society, 2024) is an online resource designed to help researchers develop BI plans for their NSF proposals. Several elements of the Toolkit address critical aspects of public engagement, making the Toolkit a valuable resource for researchers new to BI or public engagement, especially when integrated into wider BI communities of practice. We discuss how the national-level BI Community of Practice (BI-CoP) developed and sustained by the NSF-funded Center for Advancing Research Impacts in Society (ARIS) contributed to the development and continuing evolution of an institutional-level BI CoP at a large land-grant public university. The personal narratives of members of the institutional-level BI-CoP reveal how the ARIS BI-CoP has supported their learning, fostered collaboration around BI at their institution, supported the development of an institutional BI-CoP, and increased their capacity to assist researchers with developing and implementing BI plans. The experiences of consultants and researchers demonstrate that supportive and well-resourced BI-CoPs at the national and institutional level are essential for making effective use of the Toolkit and developing BI plans that are innovative, inclusive, and impactful. 
    more » « less
  2. ; (, MRS Advances)
    ABSTRACT The National Alliance for Broader Impacts (NABI) seeks to foster a community of practice that increases individual and institutional capacity for, and engagement in, broader impact (BI) activities and scholarship. NABI currently has 537 individual members representing more than 210 institutions and organizations who are part of the growing network of professionals. The National Science Foundation (NSF) evaluates all proposals on their intellectual merit and their broader impacts. Many investigators grapple with how to articulate and effectively engage broad audiences in materials science and STEM. Here, we describe the effort of NABI to address BI challenges, present the NABI document Broader Impacts, Guiding Principles and Questions for National Science Foundation Principal Investigators and Proposal Reviewers; highlight the impacts of NABI as a catalyst for building BI capacity; and provide an example of assessing an innovative program’s BI. 
    more » « less
  3. ; ; ; ; (, Journal of Community Engagement and Scholarship)
    The Center for Advancing Research Impacts in Society (ARIS) Broader Impacts (BI) Toolkit is a collection of online, interactive tools focused on the generation of broader impacts activity plans that satisfy the criteria of proposals submitted to the National Science Foundation (NSF). The BI Toolkit includes a set of ARIS Guiding Principles, a Planning Checklist, the BI Wizard, and the BI Rubric. Over a three-year period, the ARIS BI Toolkit was pilot-tested with BI professionals located within ten institutions geographically distributed across the United States. The participating institutions represented a range of R1 and R2 institutions, Land-Grant Institutions, and Minority Serving Institutions with BI professionals serving in centralized and decentralized capacities within the institutions. This paper discusses the development of the ARIS BI Toolkit, its evolution through usability testing, a description of the BI Toolkit as it currently exists, and goals for future expansion. 
    more » « less
  4. ; (, Journal of Community Engagement and Scholarship)
    The ARIS Broader Impacts (BI) Toolkit Consultation Protocol (BITCoP) was designed to help BI professionals deliver foundational knowledge and resources to support higher education staff and faculty in developing BI plans for their NSF proposals. Beyond the scope of the National Science Foundation’s Proposal and Award Policies and Procedures Guide, BITCoP was designed to address key terminology, valuable resources, and the usefulness of the ARIS BI Toolkit. Goals of the BITCoP include to give BI professionals nationwide a mechanism for delivering consultations that result in robust BI plans and gain insight into the evolving needs of the research community. 
    more » « less
  5. ; ; ; (, Journal of Community Engagement and Scholarship)
    This article offers community partner perspectives on the National Science Foundation–funded Broader Impacts (BI) Toolkit, a new digital tool researchers can use to improve the societal relevance of their research. Partners suggest three ways the toolkit could be improved moving forward, underscoring the importance of relationship building to any BI activity. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email