skip to main content


Title: Quantitative and Qualitative Assessment of Large-Scale Interventions in a First-Year Experience Program
In this Work-in-Progress paper, we report on the challenges and successes of a large-scale First- Year Engineering and Computer Science Program at an urban comprehensive university, using quantitative and qualitative assessment results. Large-scale intervention programs are especially relevant to comprehensive minority serving institutions (MSIs) that serve a high percentage of first-generation college students who often face academic and socioeconomic barriers. Our program was piloted in 2015 with 30 engineering students, currently enrolls 60 engineering and computer science students, and is expected to grow to over 200 students by Fall 2020. The firstyear program interventions include: (i) block schedules for each cohort in the first year; (ii) redesigned project-based introduction to engineering and introduction to computer science courses; (iii) an introduction to mechanics course, which provides students with the foundation needed to succeed in the traditional physics sequence; and (iv) peer-led supplemental instruction (SI) workshops for Calculus, Physics and Chemistry. A faculty mentorship program was implemented to provide additional support to students, but was phased out after the first year. Challenges encountered in the process of expanding the program include administrative, such as scheduling and training faculty and SI leaders; barriers to improvement of math and science instruction; and more holistic concerns such as creating a sense of community and identity for the program. Quantitative data on academic performance includes metrics such as STEM GPA and persistence, along with the Force Concept Inventory (FCI) for physics. Qualitative assessments of the program have used student and instructor surveys, focus groups, and individual interviews to measure relationships among factors associated with college student support and to extract student perspectives on what works best for them. Four years of data tell a mixed story, in which the qualitative effect of the interventions on student confidence and identity is strong, while academic performance is not yet significantly different than that of comparison groups. One of the most significant results of the program is the development of a FYrE Professional Learning Community which includes faculty (both tenure-track and adjunct), department chairs, staff, and administrators from across the campus.  more » « less
Award ID(s):
1727054
NSF-PAR ID:
10100253
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ASEE Annual Conference proceedings
ISSN:
1524-4644
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to these needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family to attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees. 
    more » « less
  2. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  3. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  4. The College of Engineering, Computer Science, and Technology (ECST) at California State University, Los Angles, an Hispanic Serving Institution (HSI) with over 60% Hispanic students, is committed to improving graduation rates through the Grad initiative 2025 (the California State University’s initiative to increase graduation rates for all CSU students while eliminating achievement gaps). The majority of our students are under-represented minorities, low-income, Pell-eligible and first generation. Currently, one quarter of the students leaving the major before the second year. Many that “survive” the first two years of math and science do not develop the knowledge and the skills that are needed to succeed in upper division engineering courses, leading to more students unable to finish their engineering majors. Three years ago, we launched a pilot program for the First-Year Experience at ECST (FYrE@ECST) for incoming freshmen. The program focuses on providing academic support for math and physics courses while introducing students to the college community, and comprises a summer bridge program, a hands-on introductory course, cohorted math and science sections, and staff and faculty mentoring. Academic support is provided through peer-led supplemental instruction (SI) workshops. The workshops have led to a significant improvement in student performance in Math, but have had no significant impact in the student performance in physics. Our hypothesis is that students, in addition to having limited understanding of calculus, struggle to understand the fundamental principles of physics and thus cannot apply their knowledge of math to theories in physics to solve problems. This work-in-progress paper describes an inquiry-based hands-on pre-physics course for first-year students as part of the FYrE@ECST program. The course is intended to prepare students for the calculus-based mechanics course in physics and covers about half of the competencies of a classical mechanics course, with focuses on the fundamental concepts of mechanics (i.e. Newton’s Laws, Types of forces, vectors, free-body diagrams, position, velocity and acceleration). Equations are only introduced in the second half of the semester, while the first half is directed to help students develop a deep understanding of these fundamental concepts. During classes, students run simple experiments, watch segments of movies and cartoons and are asked questions (written and orally) which can guide them to think intuitively and critically. A think-pair-share mode of instruction is implemented to promote inquiry and discussion. Students work in groups of five to discuss and solve problems, carry out experiments to better understand processes and systems, and share what they learned with the whole class. The paper presents preliminary results on student achievement. 
    more » « less
  5. Achieving Change in our Communities for Equity and Student Success (ACCESS) in STEM at the University of Washington Tacoma started as a Track 1 S-STEM program in 2018 and has supported 69 students to date. This year we received Track 2 funding and welcomed our fifth cohort to campus, with funding to support ~32 additional students through 2026. University of Washington Tacoma is an Asian American and Native American Pacific Islander-serving institution (AANAPISI), and we serve a high proportion of racial minority and first generation college students. Our ACCESS scholars are pursuing bachelor’s degrees in Mathematics, Environmental Science, Biomedical Sciences, Information Technology, Computer Science and Systems, Computer Engineering and Systems, Electrical Engineering, Mechanical Engineering, and Civil Engineering, with Computer Science and Engineering representing over 60% of ACCESS scholars to date. First-time college students and first-year transfer students receive full scholarships for their first two years, and partial scholarships for their third and fourth years. The project includes an optional Early Fall Math course to enhance entry into STEM majors, and participants are able to engage in a Research Experience or project-based Introduction to Engineering course in their first year. Coupled with individual faculty mentoring and an on-campus STEM living learning community, the quarterly Success in STEM seminar course helps scholars form a cohesive community through group mentoring, as well as develop a sense of belonging, identity, and empowerment to transform the culture of STEM. This program is distinguished by its focus on pre-STEM majors in their first and second years on campus, and includes mentor training for ~30-40 faculty in teaching and mentoring diverse student populations, thus impacting all students in our majors. Our goal was to evaluate the effectiveness of a program that focuses on the first two years of college and provides financial support, courses to introduce students to research and project-based engineering, and intensive mentoring in increasing retention and academic success for Computer Science and Engineering (CS+E) students, and whether this program helps to close equity gaps for CS+E students who are low socioeconomic status (SES), underrepresented minorities (URMs), female, and/or first generation in college (First Gen) students. We compared our student scholars to a comparison group of students who met eligibility requirements but did not participate in the program. Program scholars had higher first and second year retention, and had significantly higher GPAs. The pandemic resulted in significant social, emotional, and economic stresses for our program scholars, which may have heightened the impact of the ACCESS in STEM program. 
    more » « less