skip to main content


Title: A learning analytics case study: On class sizes in undergraduate writing courses

With the collection and availability of data on student academic performance and academic background, higher education institutions have recently stepped up initiatives in and infrastructure for learning analytics, leveraging this deluge of data to inform student success. With definitions of student success varying from analyses of what predicts levels of specific career readiness competencies to degree completion, the environment is a fertile ground for statistical practice and collaboration among a statistically savvy yet diverse clientele of instructors, programme advisors and administrators. In this paper, we discuss our experiences to this end through a consulting project evaluating the impact of writing course class size on students achieving a graduation writing requirement. In detailing the workflow for and challenges in this project, we share aspects of statistical communication and reporting, applications of innovative statistical methodology developed by our research group for handling confounding factors and correlated inputs and training through an interdisciplinary applied institutional research professional development programme. This paper illustrates how instilling an appreciation for statistical inference through each of these components is invaluable for capturing institutional buy‐in for data‐informed decision‐making in general statistical practice.

 
more » « less
Award ID(s):
1633130
NSF-PAR ID:
10395486
Author(s) / Creator(s):
 ;  ;  ;  ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Stat
Volume:
12
Issue:
1
ISSN:
2049-1573
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. A 2019 report from the National Academies on Minority Serving Institutions (MSIs) concluded that MSIs need to change their culture to successfully serve students with marginalized racial and/or ethnic identities. The report recommends institutional responsiveness to meet students “where they are,” metaphorically, creating supportive campus environments and providing tailored academic and social support structures. In recent years, the faculty, staff, and administrators at California State University, Los Angeles have made significant efforts to enhance student success through multiple initiatives including a summer bridge program, first-year in engineering program, etc. However, it has become clear that more profound changes are needed to create a culture that meets students “where they are.” In 2020, we were awarded NSF support for Eco-STEM, an initiative designed to change a system that demands "college-ready" students into one that is "student-ready." Aimed at shifting the deficit mindset prevailing in engineering education, the Eco-STEM project embraces an asset-based ecosystem model that thinks of education as cultivation, and ideas as seeds we are planting, rather than a system of standards and quality checks. This significant paradigm and culture transformation is accomplished through: 1) The Eco-STEM Faculty Fellows’ Community of Practice (CoP), which employs critically reflective dialogue[ ][ ] to enhance the learning environment using asset-based learner-centered instructional approaches; 2) A Leadership CoP with department chairs and program directors that guides cultural change at the department/program level; 3) A Facilitators’ CoP that prepares facilitators to lead, sustain, update, and expand the Faculty and Leadership CoPs; 4) Reform of the teaching evaluation system to sustain the cultural changes. This paper presents the progress and preliminary findings of the Eco-STEM project. During the first project year, the project team formulated the curriculum for the Faculty CoP with a focus on inclusive pedagogy, community cultural wealth, and community building, developed a classroom peer observation tool to provide formative data for teaching reflection, and designed research inquiry tools. The latter investigates the following research questions: 1) To what extent do the Eco-STEM CoPs effectively shift the mental models of participants from a factory-like model to an ecosystem model of education? 2) To what extent does this shift support an emphasis on the assets of our students, faculty, and staff members and, in turn, allow for enhanced motivation, excellence and success? 3) To what extent do new faculty assessment tools designed to provide feedback that reflects ecosystem-centric principles and values allow for individuals within the system to thrive? In Fall 2021, the first cohort of Eco-STEM Faculty Fellows were recruited, and rich conversations and in-depth reflections in our CoP meetings indicated Fellows’ positive responses to both the CoP curriculum and facilitation practices. This paper offers a work-in-progress introduction to the Eco-STEM project, including the Faculty CoP, the classroom peer observation tool, and the proposed research instruments. We hope this work will cultivate broader conversations within the engineering education research community about cultural change in engineering education and methods towards its implementation. 
    more » « less
  2. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  3. The STEM Excellence through Engagement in Collaboration, Research, and Scholarship (SEECRS) project at Whatcom Community College is a five-year program aiming to support academically talented students with demonstrated financial need in biology, chemistry, geology, computer science, engineering, and physics. This project is funded by an NSF S-STEM (Scholarships in Science, Technology, Engineering, and Mathematics) grant awarded in January 2017. Through an inclusive and long-range effort, the college identified a strong need for financial and comprehensive supports for STEM students. This project will offer financial, academic, and professional support to three two-year cohorts of students. The SEECRS project aims to utilize a STEM-specific guided pathways approach to strengthen recruitment, retention, and matriculation of STEM students at the community college level. Scholarship recipients will be supported through participation in the SEECRS Scholars Academy, a multi-pronged approach to student support combining elements of community building, faculty mentorship, targeted advising activities, authentic science practice, and social activities. Students are introduced to disciplines of interest through opportunities to engage in course-based undergraduate research experiences (CUREs) in Biology, Chemistry and Engineering courses, funded summer research opportunities, and seminars presented by STEM professionals. Communities of practice will be nurtured through the introduction of cohort building and faculty mentorship. Cohort development starts with a required two-credit course for all scholars that emphasizes STEM identity development, specifically focusing on identifying and coping with the ways non-dominant individuals (racial/ethnic minorities, non-male gender, lower socioeconomic status, first-generation, 2-year community college vs. 4-year institutions) are made to feel as outsiders in STEM. Each SEECRS scholar is paired with a faculty mentor who engages in ongoing mentor training. The project evaluation will determine the efficacy of the project activities in achieving their intended outcomes. Specifically, we will collect data to answer the research question: To what extent can a guided pathways approach provide a coordinated and supported STEM experience at Whatcom Community College that: (1) increases student success, and (2) positively shifts students’ STEM self-identity? The evaluation will employ a quasi-experimental research design, specifically a pretest-posttest design with a matched comparison group. Our first cohort of 14 students was selected over two application rounds (winter and summer 2017). We awarded ten full scholarships and four half-scholarships based on financial need data. Cohort demographics of note compared to institutional percentages are: females (64% vs. 57%), Hispanic (14% vs. 17%), African American (7% vs. 2%), white (79% vs. 66%), first generation college bound (43% vs. 37%). The cohort is comprised of six students interested in engineering, six in biology, and one each in geology and environmental sciences. With increased communication between the project team, our Financial Aid office, Entry and Advising, high school outreach, and the Title III grant-funded Achieve, Inspire, Motivate (AIM) Program, as well as a longer advertising time, we anticipate significantly enhancing our applicant pool for the next cohort. The results and lessons learned from our first year of implementation will be presented. 
    more » « less
  4. Effects of High Impact Educational Practices on Engineering and Computer Science Student Participation, Persistence, and Success at Land Grant Universities: Award# RIEF-1927218 – Year 2 Abstract Funded by the National Science Foundation (NSF), this project aims to investigate and identify associations (if any) that exist between student participation in High Impact Educational Practices (HIP) and their educational outcomes in undergraduate engineering and computer science (E/CS) programs. To understand the effects of HIP participation among E/CS students from groups historically underrepresented and underserved in E/CS, this study takes place within the rural, public university context at two western land grant institutions (one of which is an Hispanic-serving institution). Conceptualizing diversity broadly, this study considers gender, race and ethnicity, and first-generation, transfer, and nontraditional student status to be facets of identity that contribute to the diversity of academic programs and the technical workforce. This sequential, explanatory, mixed-methods study is guided by the following research questions: 1. To what extent do E/CS students participate in HIP? 2. What relationships (if any) exist between E/CS student participation in HIP and their educational outcomes (i.e., persistence in major, academic performance, and graduation)? 3. How do contextual factors (e.g., institutional, programmatic, personal, social, financial, etc.) affect E/CS student awareness of, interest in, and participation in HIP? During Project Year 1, a survey driven quantitative study was conducted. A survey informed by results of the National Survey of Student Engagement (NSSE) from each institution was developed and deployed. Survey respondents (N = 531) were students enrolled in undergraduate E/CS programs at either institution. Frequency distribution analyses were conducted to assess the respondents’ level of participation in extracurricular HIPs (i.e., global learning and study aboard, internships, learning communities, service and community-based learning, and undergraduate research) that have been shown in the literature to positively impact undergraduate student success. Further statistical analysis was conducted to understand the effects of HIP participation, coursework enjoyability, and confidence at completing a degree on the academic success of underrepresented and nontraditional E/CS students. Exploratory factor analysis was used to derive an "academic success" variable from five items that sought to measure how students persevere to attain academic goals. Results showed that a linear relationship in the target population exists and that the resultant multiple regression model is a good fit for the data. During the Project Year 2, survey results were used to develop focus group interview protocols and guide the purposive selection of focus group participants. Focus group interviews were conducted with a total of 27 undergraduates (12 males, 15 females, 16 engineering students, 11 computer science students) across both institutions via video conferencing (i.e., ZOOM) during the spring and fall 2021 semesters. Currently, verified focus group transcripts are being systematically analyzed and coded by a team of four trained coders to identify themes and answer the research questions. This paper will provide an overview of the preliminary themes so far identified. Future project activities during Project Year 3 will focus on refining themes identified during the focus group transcript analysis. Survey and focus group data will then be combined to develop deeper understandings of why and how E/CS students participate in the HIP at their university, taking into account the institutional and programmatic contexts at each institution. Ultimately, the project will develop and disseminate recommendations for improving diverse E/CS student awareness of, interest in, and participation in HIP, at similar land grant institutions nationally. 
    more » « less
  5. This NSF Advanced Technological Education (ATE) research and development project aims to design and test a Backtracking Technique for understanding the pathways students take through college and into careers in science, technology, engineering, and mathematics (STEM) and career and technical education (CTE), with the focus of this project on information technology (IT). The project gathers data about current and former students who started in the same cohort, includes institutional research data (e.g., grades, demographics, course-taking) and merges these data with employment data from surveys and lived experiences obtained from interviews. These data are analyzed to identify potential pathways and critical junctions that may lead to student success or other outcomes. The research team is led by a doctoral granting institution and a community college, and includes four additional community colleges that collectively serve rural and urban student populations. In this paper we share the potential of the Backtracking Technique to generate contextualized career pathway data for institutions and create visualizations that can aid in institutional decision-making through a study pilot. The pilot is an initial effort to test the project’s aims of integrating institutional data with phenomenological data to model student progression through post-secondary STEM programs. The analysis will identify and verify influencers that support or hinder student success. Quantitative data analyses will consist of descriptive and comparative methods, which will be verified and informed by open coding and thematic analysis of the qualitative data. We share how the systematic investigation of institutional and phenomenological data used in the Backtracking Technique has the potential to: (1) generate practical knowledge about academic/career pathways in information technology for use by stakeholders; (2) identify and examine relationships among these pathways, students experiences, and psychosocial factors; and (3) add to the analytical methods available to institutional research professionals to document, investigate, and visualize student pathway information using data dashboards. This ATE project has great potential to transform the technician preparation for the advanced technology fields that drive the nation's economy. 
    more » « less