skip to main content

Title: Toward a Measurement of Co-Curricular Support: Insights from an Exploratory Factor Analysis
The purpose of this work-in-progress paper is to share insights from current efforts to develop and test the validity of an instrument to measure undergraduate students’ perceived support in science, technology, engineering, and mathematics (STEM). The development and refinement of our survey instrument ultimately functions to extend, operationalize, and empirically test the Model of Co-curricular Support (MCCS). The MCCS is a conceptual framework of student support that demonstrates the breadth of assistance currently used to support undergraduate students in STEM, particularly those from underrepresented groups. We are currently gathering validity evidence for an instrument that evaluates the extent to which colleges of engineering and science offer supportive environments. To date, exploratory factor analysis and correlation for construct validity have helped us develop 14 constructs for student support in STEM. Future work will focus on modeling relationships between these constructs and student outcomes, providing the explanatory power needed to explain empirically how co-curricular supports contribute to different forms of student success in STEM. We hope that operationalizing the MCCS through this survey will shift how we conceptualize and offer student support, enabling college administrators and student support practitioners to evaluate their portfolio of student support efforts.
Authors:
; ; ; ;
Award ID(s):
1704350
Publication Date:
NSF-PAR ID:
10092335
Journal Name:
Collaborative Network for Engineering and Computing Diversity (CoNECD) Conference
Sponsoring Org:
National Science Foundation
More Like this
  1. Student-retention theories traditionally focus on institutional retention, even though efforts to support students in science, technology, engineering, and mathematics (STEM) occur at the college level. This study bridges this gap between research and practice by extending and empirically testing the Model of Co-Curricular Support (MCCS), which specifically focuses on supporting and retaining underrepresented groups in STEM. The MCCS is a student-retention model that demonstrates the breadth of assistance currently used to support undergraduate students in STEM, particularly those from underrepresented groups. The aim of this exploratory research is to develop and validate a survey instrument grounded in the MCCS thatmore »can be used by college administrators and student-support practitioners to assess the magnitude of institutional support received by undergraduate students in STEM. To date, such an instrument does not exist. Our poster will present a summary of the instrument development process that has occurred to date. We are developing the survey following best practices outlined in the literature. We are clearly defining the construct of interest and target population; reviewing related tests; developing the prototype of the survey instrument; evaluating the prototype for face and content validity from students and experts; revising and testing based on suggestion; and collecting data to determine test validity and reliability across four institutional contexts. Our institutional sample sites were purposefully selected because of their large size and diversity with respect to undergraduates in STEM. The results from our study will help prioritize the elements of institutional support that should appear somewhere in a college’s suite of support efforts. Our study will provide scientific evidence that STEM researchers, educators, administrators, and policy makers need to make informed decisions to improve STEM learning environments and design effective programs, activities, and services.« less
  2. This work-in-progress research paper stems from a larger project where we are developing and gathering validity evidence for an instrument to measure undergraduate students' perceptions of support in science, technology, engineering, and mathematics (STEM). The refinement of our instrument functions to extend, operationalize, and empirically test the model of co-curricular support (MCCS). The MCCS is a conceptual framework of student support that explains how a student's interactions with the professional, academic and social systems within a college could influence their success more broadly in an undergraduate STEM degree program. Our goal is to create an instrument that functions diagnostically tomore »help colleges effectively allocate resources for the various financial, physical, and human capital support provided to undergraduate students in STEM. While testing the validity of our newly developed instrument, an analysis of the data revealed differences in perceived support among College of Engineering (COE) and College of Science (COS) students. In this work-in-progress paper, we examine these differences at one institution using descriptive statistics and Welch's t-tests to identify trends and patterns of support among different student groups.« less
  3. The purpose of the project is to identify how to measure various types of institutional support as it pertains to underrepresented and underserved populations in colleges of engineering and science. We are grounding this investigation in the Model of Co-Curricular Support, a conceptual framework that emphasizes the breadth of assistance currently used to support undergraduate students in engineering and science. The results from our study will help prioritize the elements of institutional support that should appear somewhere in a college’s suite of support efforts to improve engineering and science learning environments and design effective programs, activities, and services. Our postermore »will present: 1) an overview of the instrument development process; 2) evaluation of the prototype for face and content validity from students and experts; and 3) instrument revision and data collection to determine test validity and reliability across varied institutional contexts. In evaluating the initial survey, we included multiple rounds of feedback from students and experts, receiving feedback from 46 participants (38 students, 8 administrators). We intentionally sampled for representation across engineering and science colleges; gender identity; race/ethnicity; international student status; and transfer student status. The instrument was deployed for the first time in Spring 2018 to the institutional project partners at three universities. It was completed by 722 students: 598 from University 1, 51 from University 2, and 123 from University 3. We tested the construct validity of these responses using a minimum residuals exploratory factor analysis and correlation. A preliminary data analysis shows evidence of differences in perception on types of support college of engineering and college of science students experience. The findings of this preliminary analysis were used to revise the instrument further prior to the next round of testing. Our target sample for the next instrument deployment is 2,000 students, so we will survey ~13,000 students based on a 15% anticipated response rate. Following data collection, we will use confirmatory factor analysis to continue establishing construct validity and report on the stability of constructs emerging from our piloting on a new student sample(s). We will also investigate differences across these constructs by subpopulations of students.« less
  4. This work-in-progress paper presents emerging results from a research study aiming to develop and gather validity evidence for an instrument that can be used by college administrators and student-support practitioners to assess the magnitude of undergraduate students’ perceived institutional support received in science, technology, engineering, and mathematics (STEM). Our goal is to provide stakeholders with a validated tool to diagnose areas of strength and opportunities to better support students, particularly those from underserved populations. Over the past year, we have engaged in a systematic process of instrument development. We began by developing a prototype based on the newly developed Modelmore »of Co-Curricular Support (MCCS). We refined it by reviewing existing literature and instruments germane to student support, and soliciting stakeholder feedback. During the spring of 2018, we distributed the instrument to STEM undergraduate students at three U.S. institutions. In this paper, we report our process of instrument development and preliminary results. These results will inform the next revision of our instrument, ultimately providing the STEM education community with novel and theory-based ways to measure students’ perceptions of support in STEM.« less
  5. The purpose of this study is to develop an instrument to measure student perceptions about the learning experiences in their online undergraduate engineering courses. Online education continues to grow broadly in higher education, but the movement toward acceptance and comprehensive utilization of online learning has generally been slower in engineering. Recently, however, there have been indicators that this could be changing. For example, ABET has accredited online undergraduate engineering degrees at Stony Brook University and Arizona State University (ASU), and an increasing number of other undergraduate engineering programs also offer online courses. During this period of transition in engineering education,more »further investigation about the online modality in the context of engineering education is needed, and survey instrumentation can support such investigations. The instrument presented in this paper is grounded in a Model for Online Course-level Persistence in Engineering (MOCPE), which was developed by our research team by combining two motivational frameworks used to study student persistence: the Expectancy x Value Theory of Achievement Motivation (EVT), and the ARCS model of motivational design. The initial MOCPE instrument contained 79 items related to students’ perceptions about the characteristics of their courses (i.e., the online learning management system, instructor practices, and peer support), expectancies of course success, course task values, perceived course difficulties, and intention to persist in the course. Evidence of validity and reliability was collected using a three-step process. First, we tested face and content validity of the instrument with experts in online engineering education and online undergraduate engineering students. Next, the survey was administered to the online undergraduate engineering student population at a large, Southwestern public university, and an exploratory factor analysis (EFA) was conducted on the responses. Lastly, evidence of reliability was obtained by computing the internal consistency of each resulting scale. The final instrument has seven scales with 67 items across 10 factors. The Cronbach alpha values for these scales range from 0.85 to 0.97. The full paper will provide complete details about the development and psychometric evaluation of the instrument, including evidence of and reliability. The instrument described in this paper will ultimately be used as part of a larger, National Science Foundation-funded project investigating the factors influencing online undergraduate engineering student persistence. It is currently being used in the context of this project to conduct a longitudinal study intended to understand the relationships between the experiences of online undergraduate engineering students in their courses and their intentions to persist in the course. We anticipate that the instrument will be of interest and use to other engineering education researchers who are also interested in studying the population of online students.« less