skip to main content

Title: “What Will I Experience in My College STEM Courses?” An Investigation of Student Predictions about Instructional Practices in Introductory Courses
The instructional practices used in introductory college courses often differ dramatically from those used in high school courses, and dissatisfaction with these practices is cited by students as a prominent reason for leaving science, technology, engineering, and mathematics (STEM) majors. To better characterize the transition to college course work, we investigated the extent to which incoming expectations of course activities differ based on student demographic characteristics, as well as how these expectations align with what students will experience. We surveyed more than 1500 undergraduate students in large introductory STEM courses at three research-intensive institutions during the first week of classes about their expectations regarding how class time would be spent in their courses. We found that first-generation and first-semester students predict less lecture than their peers and that class size had the largest effect on student predictions. We also collected classroom observation data from the courses and found that students generally underpredicted the amount of lecture observed in class. This misalignment between student predictions and experiences, especially for first-generation and first-semester college students and students enrolled in large- and medium-size classes, has implications for instructors and universities as they design curricula for introductory STEM courses with explicit retention goals.  more » « less
Award ID(s):
1712060 1712074
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
CBE—Life Sciences Education
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Background

    Large introductory lecture courses are frequently post-secondary students’ first formal interaction with science, technology, engineering, and mathematics (STEM) disciplines. Grade outcomes in these courses are often disparate across student populations, which, in turn, has implications for student retention. This study positions such disparities as a manifestation of systemic inequities along the dimensions of sex, race/ethnicity, income, and first-generation status and investigates the extent to which they are similar across peer institutions.


    We examined grade outcomes in a selected set of early STEM courses across six large, public, research-intensive universities in the United States over ten years. In this sample of more than 200,000 STEM course enrollments, we find that course grade benefits increase significantly with the number of systemic advantages students possess at all six institutions. The observed trends in academic outcomes versus advantage are strikingly similar across universities despite the fact that we did not control for differences in grading practices, contexts, and instructor and student populations. The findings are concerning given that these courses are often students’ first post-secondary STEM experiences.


    STEM course grades are typically lower than those in other disciplines; students taking them often pay grade penalties. The systemic advantages some student groups experience are correlated with significant reductions in these grade penalties at all six institutions. The consistency of these findings across institutions and courses supports the claim that inequities in STEM education are a systemic problem, driven by factors that go beyond specific courses or individual institutions. Our work provides a basis for the exploration of contexts where inequities are exacerbated or reduced and can be used to advocate for structural change within STEM education. To cultivate more equitable learning environments, we must reckon with how pervasive structural barriers in STEM courses negatively shape the experiences of marginalized students.

    more » « less
  2. Abstract

    Introductory STEM courses represent entry points into a major, and student experiences in these courses can affect both their persistence and success in STEM disciplines. Identifying course-based student concerns may help instructors detect negative perceptions, areas of struggle, and potential barriers to success. Using an open-response survey question, we identified 13 common concerns expressed by students in introductory STEM courses. We converted these student-generated concerns into closed-ended items that were administered at the beginning and middle of the semester to students in 22 introductory STEM course sections across three different institutions. Students were asked to reflect on each item on a scale from very concerned to not concerned. A subset of these concerns was used to create a summary score of course-based concern for each student. Overall levels of student concern decreased from the first week to the middle of the semester; however, this pattern varied across different demographic groups. In particular, when controlling for initial concern and course grades, female students held higher levels of concern than their peers. Since student perceptions can impact their experiences, addressing concerns through communication and instructional practices may improve students’ overall experiences and facilitate their success.

    more » « less
  3. Abstract

    Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.Practitioner notesWhat is already known about this topic

    Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies.

    Prediction models based on demographic data can perpetuate systemic biases.

    Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks.

    What this paper adds

    Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication.

    The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters.

    Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling.

    Implications for practice and/or policy

    High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable.

    Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors.

    Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.

    more » « less
  4. The drive to encourage young people to pursue degrees and careers in engineering has led to an increase in student populations in engineering programs. For some institutions, such as large public research institutions, this has led to large class sizes for courses that are commonly taken across multiple programs. While this decision is reasonable from an operational and resource management perspective, research on large classes have shown that students suffer decreased engagement, motivation and achievement. Instructors, on the other hand, report having difficulty establishing rapport with their students and a growing inability to monitor students’ learning gains and provide quality individualized feedback. To address these issues, our project draws from Lattuca and Stark’s Academic Plan model, which incorporates a thorough consideration of factors influencing curricular activities that can be applied at the course, program, and institutional levels, and assumes that instructors are key actors in curriculum development and revision. We aim to revitalize feedback loops to help instructors and departments continuously improve. Recognizing that we must understand both individual and systems level perspectives, we prioritize regular engagement between faculty and institutional support structures to collaboratively identify problems and systematically establish continuous improvement. In the first phase of this NSF IUSE Institutional Transformation project, we focus on specifically prompting and studying the experiences of 8 instructors of foundational engineering courses usually taught in large class sizes across 4 different departments at a large public research institution. We collected qualitative data (semi-structured interviews, reflective journals, course-related documents) and quantitative data (student surveys and institution-provided transcript data) to answer research questions (e.g., what data do faculty teaching large foundational undergraduate engineering courses identify as being useful so that they may enhance students’ experiences and outcomes within the classes that they teach and across students’ multiple large classes?) at the intersection of learning analytics and faculty change. The data was used as a baseline to further refine data collection protocols, identify data that faculty consider meaningful and useful for managing large foundational engineering courses, and consider ways of productively leveraging institutional data to improve the learning experience in these courses. Data collection for the first phase is ongoing and will continue through the Spring 2018 semester. Findings for this paper will include high-level insights from Fall interviews with instructors as well as data visualizations created from the population-level data characterizing student performance in the foundational courses within the context of pre-college characteristics (e.g., SAT scores) and/or other academic outcomes (e.g., major switching within or out of engineer, degree attainment). 
    more » « less
  5. There is a critical need for more students with engineering and computer science majors to enter into, persist in, and graduate from four-year postsecondary institutions. Increasing the diversity of the workforce by inclusive practices in engineering and science is also a profound identified need. According to national statistics, the largest groups of underrepresented minority students in engineering and science attend U.S. public higher education institutions. Most often, a large proportion of these students come to colleges and universities with unique challenges and needs, and are more likely to be first in their family to attend college. In response to these needs, engineering education researchers and practitioners have developed, implemented and assessed interventions to provide support and help students succeed in college, particularly in their first year. These interventions typically target relatively small cohorts of students and can be managed by a small number of faculty and staff. In this paper, we report on “work in progress” research in a large-scale, first-year engineering and computer science intervention program at a public, comprehensive university using multivariate comparative statistical approaches. Large-scale intervention programs are especially relevant to minority serving institutions that prepare growing numbers of students who are first in their family to attend college and who are also under-resourced, financially. These students most often encounter academic difficulties and come to higher education with challenging experiences and backgrounds. Our studied first-year intervention program, first piloted in 2015, is now in its 5th year of implementation. Its intervention components include: (a) first-year block schedules, (b) project-based introductory engineering and computer science courses, (c) an introduction to mechanics course, which provides students with the foundation needed to succeed in a traditional physics sequence, and (d) peer-led supplemental instruction workshops for calculus, physics and chemistry courses. This intervention study responds to three research questions: (1) What role does the first-year intervention’s components play in students’ persistence in engineering and computer science majors across undergraduate program years? (2) What role do particular pedagogical and cocurricular support structures play in students’ successes? And (3) What role do various student socio-demographic and experiential factors play in the effectiveness of first-year interventions? To address these research questions and therefore determine the formative impact of the firstyear engineering and computer science program on which we are conducting research, we have collected diverse student data including grade point averages, concept inventory scores, and data from a multi-dimensional questionnaire that measures students’ use of support practices across their four to five years in their degree program, and diverse background information necessary to determine the impact of such factors on students’ persistence to degree. Background data includes students’ experiences prior to enrolling in college, their socio-demographic characteristics, and their college social capital throughout their higher education experience. For this research, we compared students who were enrolled in the first-year intervention program to those who were not enrolled in the first-year intervention. We have engaged in cross-sectional 2 data collection from students’ freshman through senior years and employed multivariate statistical analytical techniques on the collected student data. Results of these analyses were interesting and diverse. Generally, in terms of backgrounds, our research indicates that students’ parental education is positively related to their success in engineering and computer science across program years. Likewise, longitudinally (across program years), students’ college social capital predicted their academic success and persistence to degree. With regard to the study’s comparative research of the first-year intervention, our results indicate that students who were enrolled in the first-year intervention program as freshmen continued to use more support practices to assist them in academic success across their degree matriculation compared to students who were not in the first-year program. This suggests that the students continued to recognize the value of such supports as a consequence of having supports required as first-year students. In terms of students’ understanding of scientific or engineering-focused concepts, we found significant impact resulting from student support practices that were academically focused. We also found that enrolling in the first-year intervention was a significant predictor of the time that students spent preparing for classes and ultimately their grade point average, especially in STEM subjects across students’ years in college. In summary, we found that the studied first-year intervention program has longitudinal, positive impacts on students’ success as they navigate through their undergraduate experiences toward engineering and computer science degrees. 
    more » « less