skip to main content


Title: Quantitative and Qualitative Assessment of Large-Scale Interventions in a First-Year Experience Program
In this Work-in-Progress paper, we report on the challenges and successes of a large-scale First- Year Engineering and Computer Science Program at an urban comprehensive university, using quantitative and qualitative assessment results. Large-scale intervention programs are especially relevant to comprehensive minority serving institutions (MSIs) that serve a high percentage of first-generation college students who often face academic and socioeconomic barriers. Our program was piloted in 2015 with 30 engineering students, currently enrolls 60 engineering and computer science students, and is expected to grow to over 200 students by Fall 2020. The firstyear program interventions include: (i) block schedules for each cohort in the first year; (ii) redesigned project-based introduction to engineering and introduction to computer science courses; (iii) an introduction to mechanics course, which provides students with the foundation needed to succeed in the traditional physics sequence; and (iv) peer-led supplemental instruction (SI) workshops for Calculus, Physics and Chemistry. A faculty mentorship program was implemented to provide additional support to students, but was phased out after the first year. Challenges encountered in the process of expanding the program include administrative, such as scheduling and training faculty and SI leaders; barriers to improvement of math and science instruction; and more holistic concerns such as creating a sense of community and identity for the program. Quantitative data on academic performance includes metrics such as STEM GPA and persistence, along with the Force Concept Inventory (FCI) for physics. Qualitative assessments of the program have used student and instructor surveys, focus groups, and individual interviews to measure relationships among factors associated with college student support and to extract student perspectives on what works best for them. Four years of data tell a mixed story, in which the qualitative effect of the interventions on student confidence and identity is strong, while academic performance is not yet significantly different than that of comparison groups. One of the most significant results of the program is the development of a FYrE Professional Learning Community which includes faculty (both tenure-track and adjunct), department chairs, staff, and administrators from across the campus.  more » « less
Award ID(s):
1727054
NSF-PAR ID:
10100253
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ASEE Annual Conference proceedings
ISSN:
1524-4644
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to these needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family to attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees. 
    more » « less
  2. The College of Engineering, Computer Science, and Technology (ECST) at California State University, Los Angles, an Hispanic Serving Institution (HSI) with over 60% Hispanic students, is committed to improving graduation rates through the Grad initiative 2025 (the California State University’s initiative to increase graduation rates for all CSU students while eliminating achievement gaps). The majority of our students are under-represented minorities, low-income, Pell-eligible and first generation. Currently, one quarter of the students leaving the major before the second year. Many that “survive” the first two years of math and science do not develop the knowledge and the skills that are needed to succeed in upper division engineering courses, leading to more students unable to finish their engineering majors. Three years ago, we launched a pilot program for the First-Year Experience at ECST (FYrE@ECST) for incoming freshmen. The program focuses on providing academic support for math and physics courses while introducing students to the college community, and comprises a summer bridge program, a hands-on introductory course, cohorted math and science sections, and staff and faculty mentoring. Academic support is provided through peer-led supplemental instruction (SI) workshops. The workshops have led to a significant improvement in student performance in Math, but have had no significant impact in the student performance in physics. Our hypothesis is that students, in addition to having limited understanding of calculus, struggle to understand the fundamental principles of physics and thus cannot apply their knowledge of math to theories in physics to solve problems. This work-in-progress paper describes an inquiry-based hands-on pre-physics course for first-year students as part of the FYrE@ECST program. The course is intended to prepare students for the calculus-based mechanics course in physics and covers about half of the competencies of a classical mechanics course, with focuses on the fundamental concepts of mechanics (i.e. Newton’s Laws, Types of forces, vectors, free-body diagrams, position, velocity and acceleration). Equations are only introduced in the second half of the semester, while the first half is directed to help students develop a deep understanding of these fundamental concepts. During classes, students run simple experiments, watch segments of movies and cartoons and are asked questions (written and orally) which can guide them to think intuitively and critically. A think-pair-share mode of instruction is implemented to promote inquiry and discussion. Students work in groups of five to discuss and solve problems, carry out experiments to better understand processes and systems, and share what they learned with the whole class. The paper presents preliminary results on student achievement. 
    more » « less
  3. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  4. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  5. With support from NSF Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM), the Culturally Adaptive Pathway to Success (CAPS) program aims to build an inclusive pathway to accelerate the graduation for academically talented, low-income students in Engineering and Computer Science majors at [University Name], which traditionally serves the underrepresented and educationally disadvantaged minority students in the [City Name area]. CAPS focuses on progressively developing social and career competence in our students via three integrated interventions: (1) Mentor+, a relationally informed advising strategy that encourages students to see their academic work in relation to their families and communities; (2) peer cohorts, providing social support structure for students and enhancing their sense of belonging in engineering and computer science classrooms and beyond; and (3) professional development from faculty who have been trained in difference-education theory, so that they can support students with varying levels of understanding of the antecedents of college success. To ensure success of these interventions, the CAPS program places great emphasis on developing culturally responsive advisement methods and training faculty mentors to facilitate creating a culture of culturally adaptive advising. This paper presents the CAPS progress in the past two project years. In particular, we will share several changes that we have made after the first project year to improve several key components of the program - recruitment, cohort building, and mentor training. The program strengthened the recruitment by actively involving scholars and faculties in reaching out to students and successfully recruited more scholars for the second cohort (16 scholars) than the first cohort (12 scholars). Also, the program has initiated new activities for peer-mentoring and cohort gathering within each major. As continuous development of the mentor training, the program has added a training session focusing on various aspects of intersectionality as it relates to individual’s social identities, and how mentors can use these knowledge to better interact with mentees. In addition to these changes, we will also report findings on how the program impacted on scholars’ academic growth and mentors’ understanding about the culturally adaptive advisement to answer the CAPS research questions (a) how these interventions affect the development of social belonging and engineering identity of CAPS scholars, and (b) the impact of Mentor+ on academic resilience and progress to degree. The program conducted qualitative data collection and analysis via focus group meetings and interviews as well as quantitative data collection and analysis using academic records and surveys. Our findings will help enhance the CAPS program and establish a sustainable Scholars Support Program at the university, which can be implemented with scholarships funded by other sources, and which can be transferred to similar culturally diverse institutions to increase success for students who have socio-economic challenges. 
    more » « less