skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Research analytics capabilities (RAC) survey: development, validation and revision using the Rasch model
PurposeThe research enterprise within higher education is becoming more competitive as funding agencies require more collaborative research projects, higher-level of accountability and competition for limited resources. As a result, research analytics has emerged as a field, like many other areas within higher education to act as a data-informed unit to better understand how research institutions can effectively grow their research strategy. This is a new and emerging field within higher education. Design/methodology/approachAs businesses and other industries are embracing recent advances in data technologies such as cloud computing and big data analytic tools to inform decision making, research administration in higher education is seeing a potential in incorporating advanced data analytics to improve day-to-day operations and strategic advancement in institutional research. This paper documents the development of a survey measuring research administrators’ perspectives on how higher education and other research institutions perceive the use of data and analytics within the research administration functions. The survey development process started with composing a literature review on recent developments in data analytics within the research administration in the higher education domain, from which major components of data analytics in research administration were conceptualized and identified. This was followed by an item matrix mapping the evidence from literature with corresponding, newly drafted survey items. After revising the initial survey based on suggestions from a panel of subject matter experts to review, a pilot study was conducted using the revised survey instrument and validated by employing the Rasch measurement analysis. FindingsAfter revising the survey based on suggestions from the subject matter experts, a pilot study was conducted using the revised survey instrument. The resultant survey instrument consists of six dimensions and 36 survey items with an establishment of reasonable item fit, item separation and reliability. This survey protocol is useful for higher educational institutions to gauge research administrators’ perceptions of the culture of data analytics use in the workplace. Suggestions for future revisions and potential use of the survey were made. Originality/valueVery limited scholarly work has been published on this topic. The use of data-informed and data-driven approaches with in research strategy within higher education is an emerging field of study and practice.  more » « less
Award ID(s):
2215223
PAR ID:
10626348
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Emerald
Date Published:
Journal Name:
Journal of Applied Research in Higher Education
ISSN:
2050-7003
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, International Journal of Mentoring and Coaching in Education)
    PurposeThe purpose of this study was to examine the experiences of multiple campus teams as they engaged in the assessment of their science, technology, engineering and mathematics (STEM) mentoring ecosystems within a peer assessment dialogue exercise. Design/methodology/approachThis project utilized a qualitative multicase study method involving six campus teams, drawing upon completed inventory and visual mapping artefacts, session observations and debriefing interviews. The campuses included research universities, small colleges and minority-serving institutions (MSIs) across the United States of America. The authors analysed which features of the peer assessment dialogue exercise scaffolded participants' learning about ecosystem synergies and threats. FindingsThe results illustrated the benefit of instructor modelling, intra-team process time and multiple rounds of peer assessment. Participants gained new insights into their own campuses and an increased sense of possibility by dialoguing with peer campuses. Research limitations/implicationsThis project involved teams from a small set of institutions, relying on observational and self-reported debriefing data. Future research could centre perspectives of institutional leaders. Practical implicationsThe authors recommend dedicating time to the institutional assessment of mentoring ecosystems. Investing in a campus-wide mentoring infrastructure could align with campus equity goals. Originality/valueIn contrast to studies that have focussed solely on programmatic outcomes of mentoring, this study explored strategies to strengthen institutional mentoring ecosystems in higher education, with a focus on peer assessment, dialogue and learning exercises. 
    more » « less
  2. ; ; ; ; ; ; (, Studies in Graduate and Postdoctoral Education)
    PurposeThis study aims to present the evaluation of a competency-based online professional development training program, PhD Progression, tied to a digital badge system, created to support PhD students across fields. Design/methodology/approachThis study took place at Boston University, a large, nonprofit, Carnegie Classified R1 research-intensive institution located in the northeastern region of the USA. Through internal campus collaborations, the authors developed a PhD core capacities framework. Building from this framework, the authors designed the first learning level of the program and ran a pilot study with PhD students from various fields and at different stages of their PhD. Using surveys and focus groups, the authors collected both quantitative and qualitative data to evaluate this program. FindingsThe quantitative and qualitative data show that the majority of the PhD student participants found the contents of the competency-based training program useful, appropriate for building skills and knowledge and therefore relevant for both their degree progress and their future job. Gaining digital badges significantly increased their motivation to complete training modules. Practical implicationsThis type of resource is scalable to other institutions that wish to provide self-paced professional development support to their PhD students while rewarding them for investing time in building professional skills and enabling them to showcase these skills to potential employers. Originality/valueThis study demonstrates, for the first time, that tying a digital badging system to a competency-based professional development program significantly motivates PhD students to set professional development goals and invest time in building skills. 
    more » « less
  3. ; (, SSRN Electronic Journal)
    As data-informed decision-making continues to evolve across multiple disciplines in higher education institutions, and as the role of research administration continues to expand from proposal submissions, compliance, and managing research and development expenditures to a profession of an active partnership with investigators to support institutional research, it becomes necessary to have a strategy or model to inform best practices in both strategic planning and business operations within research administration. As numerous research evaluation frameworks and assessment models were proposed to improve the administrative processes of institutional research, this project proposes a data-analytics approach to enhance the services provided by research administrators to minimize risks, maximize efficiencies, and better support researchers and other research and development (R&D) personnel in higher education. This article documents the process of conducting a systematic literature review on leveraging research administration data-analytics in higher education to provide evidence-based practices to inform decision-making regarding strategic advancement in institutional research. The literature review examines current works on how data can be effectively used in process operations, control, and provides new perspectives on how research administration process systems might evolve into a new paradigm of data enhanced operations and control. 
    more » « less
  4. ; ; ; ; ; (, Journal of Applied Research in Higher Education)
    PurposeThe National Science Foundation (NSF) Research Experience for Undergraduates (REU) programs are traditionally delivered in-person and full-time (40 h per week) for 10 weeks during the summer. However, this type of format has the potential to limit broader student participation. This study aims to compare learning assessment data between a traditional NSF REU (10 weeks of summer, full-time, in-person) to an alternative NSF REU delivered virtually, part-time and over 10 months as a result of the coronavirus disease 2019 (COVID-19) pandemic. Design/methodology/approachA retrospective pre-then-post survey was completed to assess perceived learning gains for each REU program. Three learning gains categories were assessed: entrepreneurial competencies, career goals and research skill development.T-tests were used to evaluate a difference in means between pre and post. FindingsFindings show the greatest quantity of learning gains within the alternative program delivery. Moreover, a larger quantity of learning gains was perceived within the first semester of the alternative program delivery compared to the second semester. Practical implicationsThe authors propose the NSF should be intentional about trying new approaches to REU programs delivery, including duration and format, as a way to broaden participation in engineering. Originality/valueThis study is original in that it is the first of its kind to assess an alternative REU program delivery (allowed only because of the COVID-19 pandemic) in comparison to traditional REU program delivery. 
    more » « less
  5. ; ; ; (, Information and Learning Sciences)
    PurposeThis study is part of a participatory design research project and aims to develop and study pedagogical frameworks and tools for integrating computational thinking (CT) concepts and data science practices into elementary school classrooms. Design/methodology/approachThis paper describes a pedagogical approach that uses a data science framework the research team developed to assist teachers in providing data science instruction to elementary-aged students. Using phenomenological case study methodology, the authors use classroom observations, student focus groups, video recordings and artifacts to detail ways learners engage in data science practices and understand how they perceive their engagement during activities and learning. FindingsFindings suggest student engagement in data science is enhanced when data problems are contextualized and connected to students’ lived experiences; data analysis and data-based decision-making is practiced in multiple ways; and students are given choices to communicate patterns, interpret graphs and tell data stories. The authors note challenges students experienced with data practices including conflict between inconsistencies in data patterns and lived experiences and focusing on data visualization appearances versus relationships between variables. Originality/valueData science instruction in elementary schools is an understudied, emerging and important area of data science education. Most elementary schools offer limited data science instruction; few elementary schools offer data science curriculum with embedded CT practices integrated across disciplines. This research assists elementary educators in fostering children's data science engagement and agency while developing their ability to reason, visualize and make decisions with data. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email