Effects of High Impact Educational Practices on Engineering and Computer Science Student Participation, Persistence, and Success at Land Grant Universities: Award# RIEF-1927218 – Year 2 Abstract Funded by the National Science Foundation (NSF), this project aims to investigate and identify associations (if any) that exist between student participation in High Impact Educational Practices (HIP) and their educational outcomes in undergraduate engineering and computer science (E/CS) programs. To understand the effects of HIP participation among E/CS students from groups historically underrepresented and underserved in E/CS, this study takes place within the rural, public university context at two western land grant institutions (one of which is an Hispanic-serving institution). Conceptualizing diversity broadly, this study considers gender, race and ethnicity, and first-generation, transfer, and nontraditional student status to be facets of identity that contribute to the diversity of academic programs and the technical workforce. This sequential, explanatory, mixed-methods study is guided by the following research questions: 1. To what extent do E/CS students participate in HIP? 2. What relationships (if any) exist between E/CS student participation in HIP and their educational outcomes (i.e., persistence in major, academic performance, and graduation)? 3. How do contextual factors (e.g., institutional, programmatic, personal, social, financial, etc.) affect E/CS student awareness of, interest in, and participation in HIP? During Project Year 1, a survey driven quantitative study was conducted. A survey informed by results of the National Survey of Student Engagement (NSSE) from each institution was developed and deployed. Survey respondents (N = 531) were students enrolled in undergraduate E/CS programs at either institution. Frequency distribution analyses were conducted to assess the respondents’ level of participation in extracurricular HIPs (i.e., global learning and study aboard, internships, learning communities, service and community-based learning, and undergraduate research) that have been shown in the literature to positively impact undergraduate student success. Further statistical analysis was conducted to understand the effects of HIP participation, coursework enjoyability, and confidence at completing a degree on the academic success of underrepresented and nontraditional E/CS students. Exploratory factor analysis was used to derive an "academic success" variable from five items that sought to measure how students persevere to attain academic goals. Results showed that a linear relationship in the target population exists and that the resultant multiple regression model is a good fit for the data. During the Project Year 2, survey results were used to develop focus group interview protocols and guide the purposive selection of focus group participants. Focus group interviews were conducted with a total of 27 undergraduates (12 males, 15 females, 16 engineering students, 11 computer science students) across both institutions via video conferencing (i.e., ZOOM) during the spring and fall 2021 semesters. Currently, verified focus group transcripts are being systematically analyzed and coded by a team of four trained coders to identify themes and answer the research questions. This paper will provide an overview of the preliminary themes so far identified. Future project activities during Project Year 3 will focus on refining themes identified during the focus group transcript analysis. Survey and focus group data will then be combined to develop deeper understandings of why and how E/CS students participate in the HIP at their university, taking into account the institutional and programmatic contexts at each institution. Ultimately, the project will develop and disseminate recommendations for improving diverse E/CS student awareness of, interest in, and participation in HIP, at similar land grant institutions nationally.
more »
« less
Using Participant Voices to Inform Validity Evidence in the Survey Development
This study presents qualitative findings from a larger instrument validation study. Undergraduates and subject matter experts (SMEs) were pivotal in early-stage development of a survey focusing on the four domains of Validation Theory (academic-in-class, academic-out-of-class, interpersonal-in-class, interpersonal-out-of-class). An iterative approach allowed for a more rigorously constructed survey refined through multiple phases. The research team met regularly to determine how feedback from undergraduates and SMEs could improve items and if certain populations were potentially being excluded. To date, the research team has expanded on the original 47 items up to 51 to address feedback provided by SMEs and undergraduate participants. Numerous item wording revisions have been made. Support for content, response process, and consequential validity evidence is strong.
more »
« less
- Award ID(s):
- 1922202
- NSF-PAR ID:
- 10327842
- Date Published:
- Journal Name:
- Annual meeting program American Educational Research Association
- ISSN:
- 0163-9676
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Identity, or how people choose to define themselves, is emerging as an attractive explanation for who persists in engineering. Many studies of engineering identity build off of prior work in math and science identity, emphasizing the academic aspects of engineering. However, affect towards professional practice is also central to engineering identity development. This paper describes the methods used to create a new survey measure of individuals’ affect toward elements of engineering practice. We followed the item generation, refinement, and instrument validation steps required for psychometric validation of a new survey measure. We generated items deductively using the literature on engineering professional skills and practice and inductively based on interviews with practicing engineers, engineering graduate students, and engineering undergraduate students. We blended the inductively and deductively derived item lists to create a list of initial items for the measure. We circulated this list of items to a set of engineering and professional identity experts to establish face validity and made modifications based on their feedback. The final list included 34 items. These 34 items were administered in a questionnaire survey in the fall of 2016 to 1465 engineering undergraduates in three majors at two institutions. We conducted an exploratory factor analysis (EFA) and established internal consistency using Cronbach’s alpha on a subset of the analytical sample data (n=384). The resulting factors fit our a priori assumption of the factors theorized to characterize affect towards engineering professional practice. Using the remaining data (n=904), we conducted a confirmatory factor analysis on the reduced set of items resulting from EFA. The results indicate an emergent factor structure for affect towards elements of engineering practice.more » « less
-
Abstract. The Circumplex Team Scan (CTS) assesses the degree to which a team’s interaction/communication norms reflect each segment (16th) of the interpersonal circle/circumplex. We developed and evaluated an abbreviated 16-item CTS-16 that uses one CTS item to measure each segment. Undergraduates ( n = 446) completing engineering course projects in 139 teams completed the CTS-16. CTS-16 items showed a good fit to confirmatory structural models (e.g., that expect greater positive covariation between items theoretically closer to the circumplex). Individuals’ ratings sufficiently reflected team-level norms to justify averaging team members’ ratings. However, individual items’ marginal reliabilities suggest using the CTS-16 to assess general circumplex-wide patterns rather than specific segments. CTS-16 ratings correlated with respondents’ and their teammates’ ratings of team climate (inclusion, justice, psychological safety). Teams with more extraverted (introverted) members were perceived as having more confident/engaged (timid/hesitant) cultures. Members predisposed to social alienation perceived their team’s culture as relatively disrespectful/unengaged, but their teammates did not corroborate those perceptions. The results overall support the validity and utility of the CTS-16 and of an interpersonal circumplex model of team culture more generally.more » « less
-
The development of inclusive leaders is essential for the success of future engineering and our nation. Equipping students with vital leadership-enabling competencies is necessary to develop a workforce that is prepared to act ethically, and responsibly, and tackle unforeseen challenges in the future. Inclusive leaders, or leaders that are self-aware, empathetic, and prioritize diversity, equity, and inclusion in their decision-making, are essential for the forward progress of engineering. A growing body of literature highlights the numerous ways in which students may develop leadership skills outside of the classroom through involvement in out-of-class activities (e.g., internships, clubs, sports, and research experiences). Research Experiences for Undergraduates (REUs) may provide students with a unique opportunity to develop leadership-enabling competencies that will prepare them for leadership in graduate school, the engineering industry, or academia. The goal of this research was to identify how students’ engagement in an engineering education virtual REU site contributed to their development of essential leadership-enabling competencies. The research question guiding this study was ‘What inclusive leadership-enabling competencies and skills did engineering students learn and develop during an engineering education Summer REU program?’ Qualitative data was collected via weekly open-ended surveys from 9 students (7 women, 2 men) participating in an REU over 9 weeks. Participants in this study consisted of students from underrepresented groups in engineering (e.g., Black, Latinx, women, students from low SES backgrounds, or first-generation students), attending large public research universities across the United States. This study implemented mixed methods to understand what leadership competencies were occurring most frequently and how students were learning and developing these competencies. A combination of text mining for frequency (quantitative analysis) and deductive and inductive coding (qualitative analysis) was used to analyze the data. A codebook was developed based on the leadership-coupled professional competencies that engineering industry leaders identified as essential for engineers entering the workforce. Researchers also allowed for other competencies and leadership-enabling skills to emerge from the data. Findings from this work indicate that students were developing a vast amount of inclusive leadership knowledge and skills from participating in the virtual REU site. This paper highlights, through the use of word clouds and text mining software, the many leadership-enabling competencies that participants developed throughout the summer research experience (e.g., learning, communication, adaptability, self-awareness, balance, networking, etc.). Further, students were able to develop digital literacy, increased communication skills, knowledge of career pathways, intrapersonal growth, and interpersonal relations. This work offers a novel contribution to the literature in understanding how students can develop technical engineering and research skills as well as professional and leadership skills in the same space. Findings from this work help to illuminate the benefits of this virtual REU site focused on engineering education research resulting in terms of developing inclusive leadership skills. Implications for future REU programs, students interested in developing leadership skills, engineering graduate programs, academia, and industry employers are outlined.more » « less
-
Over the years, researchers have found that student engagement facilitates desired academic success outcomes for college undergraduate students. Much research on student engagement has focused on academic tasks and classroom context. High impact engagement practices (HIEP) have been shown to be effective for undergraduate student academic success. However, less is known about the effects of HIEP specifically on engineering and computer science (E/CS) student outcomes. Given the high attrition rates for E/CS students, student involvement in HIEP could be effective in improving student outcomes for E/CS students, including those from various underrepresented groups. More generally, student participation in specific HIEP activities has been shown to shape their everyday experiences in school, both academically and socially. Hence, the primary goal of this study is to examine the factors that predict academic success in E/CS using multiple regression analysis. Specifically, this study seeks to understand the effects of high impact engagement practices (HIEP), coursework enjoyability, confidence at completing a degree on academic success of the underrepresented and nontraditional E/CS students. We used exploratory factor analyses to derive “academic success” variable from five items that sought to measure how students persevere to attain academic goals. A secondary goal of the present study is to address the gap in research literature concerning how participation in HIEP affects student persistence and success in E/CS degree programs. Our research team developed and administered an online survey to investigate and identify factors that affect participation in HIEP among underrepresented and nontraditional E/CS students. Respondents (N = 531) were students enrolled in two land grant universities in the Western U.S. Multiple regression analyses were conducted to examine the proportion of the variation in the dependent variable (academic success) explained by the independent variables (i.e., high impact engagement practice (HIEP), coursework enjoyability, and confidence at completing a degree). We hypothesized that (1) high impact engagement practices will predict academic success; (2) coursework enjoyability will predict academic success; and (3) confidence at completing a degree will predict academic success. Results showed that the multiple regression model statistically predicted academic success , F(3, 270) = 33.064, p = .001, adjusted R2 = .27. This results indicate that there is a linear relationship in the population and the multiple regression model is a good fit for the data. Further, findings show that confidence at completing a degree is significantly predictive of academic success. In addition, coursework enjoyability is a strong predictor of academic success. Specifically, the result shows that an increase in high impact engagement activity is associated with an increase in students’ academic success. In sum, these findings suggest that student participation in High Impact Engagement Practices might improve academic success and course retention. Theoretical and practical implications are discussed.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
