skip to main content


Title: Crowdsourcing Classroom Observations to Identify Misconceptions in Data Science
Web-browsing histories, online newspapers, streaming music, and stock prices all show that we live in an age of data. Extracting meaning from data is necessary in many fields to comprehend the information flow. This need has fueled rapid growth in data science education aiming to serve the next generation of policy makers, data science researchers, and global citizens. Initially, teaching practices have been drawn from data science's parent disciplines (e.g., computer science and mathematics). This project addresses the early stages of developing a concept inventory of student difficulty within the newly emerging field of data science. In particular this project will address three primary research objectives: (1) identify student misconceptions in data science courses; (2) document students’ prior knowledge and identify courses that teach early data science concepts; and (3) confirm expert identification of data science concepts, and their importance for introductory-level data science curricula. During the first year of this grant, we have collected approximately 200 responses for a survey to confirm concepts from an existing body of knowledge presented by the Edison Project. Survey respondents are comprised of faculty and industry practitioners within data science and closely related fields. Preliminary analysis of these results will be presented with respect to our third research objective. In addition, we developed and launched a pilot assessment for identifying student difficulties within data science courses. The protocol includes regular responses to reflective questions by faculty, teaching assistants, and students from selected data science courses offered at the three participating institutions. Preliminary analyses will be presented along with implications for future data collection in year two of the project. In addition to the anticipated results, we expect that the data collection and analysis methodologies will be of interest to many scholars who have or will engage in discipline-based educational research.  more » « less
Award ID(s):
1839270 1839357
NSF-PAR ID:
10191570
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2020 ASEE Virtual Annual Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Need/Motivation (e.g., goals, gaps in knowledge) The ESTEEM implemented a STEM building capacity project through students’ early access to a sustainable and innovative STEM Stepping Stones, called Micro-Internships (MI). The goal is to reap key benefits of a full-length internship and undergraduate research experiences in an abbreviated format, including access, success, degree completion, transfer, and recruiting and retaining more Latinx and underrepresented students into the STEM workforce. The MIs are designed with the goals to provide opportunities for students at a community college and HSI, with authentic STEM research and applied learning experiences (ALE), support for appropriate STEM pathway/career, preparation and confidence to succeed in STEM and engage in summer long REUs, and with improved outcomes. The MI projects are accessible early to more students and build momentum to better overcome critical obstacles to success. The MIs are shorter, flexibly scheduled throughout the year, easily accessible, and participation in multiple MI is encouraged. ESTEEM also establishes a sustainable and collaborative model, working with partners from BSCS Science Education, for MI’s mentor, training, compliance, and building capacity, with shared values and practices to maximize the improvement of student outcomes. New Knowledge (e.g., hypothesis, research questions) Research indicates that REU/internship experiences can be particularly powerful for students from Latinx and underrepresented groups in STEM. However, those experiences are difficult to access for many HSI-community college students (85% of our students hold off-campus jobs), and lack of confidence is a barrier for a majority of our students. The gap between those who can and those who cannot is the “internship access gap.” This project is at a central California Community College (CCC) and HSI, the only affordable post-secondary option in a region serving a historically underrepresented population in STEM, including 75% Hispanic, and 87% have not completed college. MI is designed to reduce inequalities inherent in the internship paradigm by providing access to professional and research skills for those underserved students. The MI has been designed to reduce barriers by offering: shorter duration (25 contact hours); flexible timing (one week to once a week over many weeks); open access/large group; and proximal location (on-campus). MI mentors participate in week-long summer workshops and ongoing monthly community of practice with the goal of co-constructing a shared vision, engaging in conversations about pedagogy and learning, and sustaining the MI program going forward. Approach (e.g., objectives/specific aims, research methodologies, and analysis) Research Question and Methodology: We want to know: How does participation in a micro-internship affect students’ interest and confidence to pursue STEM? We used a mixed-methods design triangulating quantitative Likert-style survey data with interpretive coding of open-responses to reveal themes in students’ motivations, attitudes toward STEM, and confidence. Participants: The study sampled students enrolled either part-time or full-time at the community college. Although each MI was classified within STEM, they were open to any interested student in any major. Demographically, participants self-identified as 70% Hispanic/Latinx, 13% Mixed-Race, and 42 female. Instrument: Student surveys were developed from two previously validated instruments that examine the impact of the MI intervention on student interest in STEM careers and pursuing internships/REUs. Also, the pre- and post (every e months to assess longitudinal outcomes) -surveys included relevant open response prompts. The surveys collected students’ demographics; interest, confidence, and motivation in pursuing a career in STEM; perceived obstacles; and past experiences with internships and MIs. 171 students responded to the pre-survey at the time of submission. Outcomes (e.g., preliminary findings, accomplishments to date) Because we just finished year 1, we lack at this time longitudinal data to reveal if student confidence is maintained over time and whether or not students are more likely to (i) enroll in more internships, (ii) transfer to a four-year university, or (iii) shorten the time it takes for degree attainment. For short term outcomes, students significantly Increased their confidence to continue pursuing opportunities to develop within the STEM pipeline, including full-length internships, completing STEM degrees, and applying for jobs in STEM. For example, using a 2-tailed t-test we compared means before and after the MI experience. 15 out of 16 questions that showed improvement in scores were related to student confidence to pursue STEM or perceived enjoyment of a STEM career. Finding from the free-response questions, showed that the majority of students reported enrolling in the MI to gain knowledge and experience. After the MI, 66% of students reported having gained valuable knowledge and experience, and 35% of students spoke about gaining confidence and/or momentum to pursue STEM as a career. Broader Impacts (e.g., the participation of underrepresented minorities in STEM; development of a diverse STEM workforce, enhanced infrastructure for research and education) The ESTEEM project has the potential for a transformational impact on STEM undergraduate education’s access and success for underrepresented and Latinx community college students, as well as for STEM capacity building at Hartnell College, a CCC and HSI, for students, faculty, professionals, and processes that foster research in STEM and education. Through sharing and transfer abilities of the ESTEEM model to similar institutions, the project has the potential to change the way students are served at an early and critical stage of their higher education experience at CCC, where one in every five community college student in the nation attends a CCC, over 67% of CCC students identify themselves with ethnic backgrounds that are not White, and 40 to 50% of University of California and California State University graduates in STEM started at a CCC, thus making it a key leverage point for recruiting and retaining a more diverse STEM workforce. 
    more » « less
  2. Our transformative mixed-methods project, funded by the Division of Engineering Education and Centers, responds to calls for more cross-institutional qualitative and longitudinal studies of minorities in engineering education. We seek to identify the factors that promote persistence and graduation as well as attrition for Black students in Electrical Engineering (EE), Computer Engineering (CpE), and Mechanical Engineering (ME). Our work combines quantitative exploration and qualitative interviews to better understand the nuanced and complex nature of retention and attrition in these fields. We are investigating the following overarching research questions: 1. Why do Black men and women choose and persist in, or leave, EE, CpE, and ME? 2. What are the academic trajectories of Black men and women in EE, CpE, and ME? 3. In what ways do these pathways vary by gender or institution? 4. What institutional policies and practices promote greater retention of Black engineering students? In this paper, we report on the results from 79 in-depth interviews with students at Predominantly White Institutions (PWIs) and a Historically Black University (HBCU [or HBU]). We describe emergent findings during Year 3 of our project, with a focus on four papers-in-progress: • Paper # 1: Our project utilized several innovative strategies for collecting narratives from our 79 interviewees. In particular, we developed a card-sorting activity to learn more about students’ reasons for choosing their engineering major. We have explored a variety of ways to analyze the data that illustrate the value of this type of data collection strategy and which will be of value to other researchers interested in decision making where there is a potentially complex set of factors, such as those found in deciding on a major. • Paper # 2: We summarized student responses to a pre-interview climate survey about three domains – Teaching and Learning, Faculty and Peer Interactions, and Belonging and Commitment. We investigated two questions: Are there differences between persisters and switchers? And, are there differences by study major? Results indicate substantial differences between persisters and switchers and some differences between ME and ECE students. • Paper # 3: Preliminary analysis of interviews of 10 HBCU Black students and 10 PWI Black students revealed that students enact several different types of community cultural wealth, particularly family, navigational, aspirational, social and resistant capital. Early results suggest that the HBCU students enacted a different form of family capital that resided in their “HBCU family” and the opportunities that their college-based networks afforded them to succeed in the major. PWI students described various forms of navigational capital and assets that were enacted in order to succeed at their study institutions. Our paper concludes with implications for university policies and practices aimed toward underrepresented students. 
    more » « less
  3. This paper presents the initial work of a recently funded NSF project on ethical and responsible research and practices in science and engineering. The objective of this research is to improve instructor training, interventions, and student outcomes in high schools and universities to improve awareness and commitment to ethical practices in STEM coursework. The project will generate a robust snapshot of the ethical knowledge, reasoning skills, attitudes, and practices of several thousand undergraduate engineering students. This snapshot will inform the development of a three-week enrichment opportunity for high school STEM teachers. Working with university faculty and graduate students, these teachers will develop learning modules on ethical issues related to their courses. The snapshot will also identify gaps and guide the creation of targeted interventions that will be used in second-, third-, and fourth-year engineering courses. This data-driven project uses a mixed-methods approach to generate a better understanding of the impact of ethics interventions at various points in a student's academic development by developing and using a set of instruments to measure cognitive, affective, and behavioral aspects of ethical competency and self-efficacy. To that end, a second snapshot will be taken by testing and surveying engineering students in their capstone courses to provide a broad overview of the competence and self-confidence that engineering students have in dealing with ethical STEM issues, to determine the efficacy of various interventions, and to improve future interventions. Utilizing repeated measures and possessing a longitudinal dimension, the project will generate extensive data about the development of ethical competency, ethical self-efficacy, and their relationship. The interventions designed for secondary and tertiary classrooms will build on best practices for micro-insertion of ethics content that are practical and help students understand how technical competencies fit within broader social, economic, and environmental contexts. The capstone snapshot will also provide some measure of the impact of other experiences (e.g., undergraduate research, internships, service learning) and courses (e.g., humanities, social science, and business courses) on development of ethical practices. This report marks the start of a five-year project; therefore, the results presented in this paper represent findings from the engineering ethics literature and baseline results from survey of engineering freshmen at Texas A&M University. The findings from the survey are being utilized in developing intervention modules that will be integrated in upper-level engineering courses and training materials for high school teachers. 
    more » « less
  4. Eastern Mennonite University received a 5-year S-STEM award for their STEM Scholars Engaging in Local Problems (SSELP) program. The goal of this place-based, interdisciplinary scholarship program is to increase the number of academically talented, low-income students who graduate in STEM fields and either pursue immediate employment in STEM careers or STEM-related service or continue their STEM education in graduate school. In 2018 and 2019, two cohorts of seven students were recruited to major in biology, chemistry, engineering, computer science, mathematics, or environmental science. A key part of recruitment involved on-campus interviews, during a February Scholarship Day, between STEM faculty and potential scholars. As the yield rate for the event is high (54-66%), the university has continued this practice, funding additional STEM scholarships. In order to retain and graduate the scholars in STEM fields, the SSELP faculty designed and carried out various projects and activities to support the students. The SSELP Scholars participated in a first-year STEM Career Practicum class, a one-credit course that connected students with regional STEM practitioners across a variety of fields. The scholars were supported by peer tutors embedded in STEM classes, and now many are tutors themselves. They participated in collaborative projects where the cohorts worked to identify and solve a problem or need in their community. The SSELP scholars were supported by both faculty and peer mentors. Each scholarship recipient was matched with a faculty mentor in addition to an academic advisor. A faculty mentor was in a related STEM field but typically not teaching the student. Each scholar was matched with a peer mentor (junior or senior) in their intended major of study. In addition, community building activities were implemented to provide a significant framework for interaction within the cohort. To evaluate the progress of the SSELP program, multiple surveys were conducted. HERI/CIRP Freshman Survey was used in the fall of 2018 for the first cohort and 2019 for the second cohort. The survey indicated an upward shift in students’ perception of science and in making collaborative effort towards positive change. Preliminary data on the Science Motivation Questionnaire showed that the SSELP scholars began their university studies with lower averages than their non-SSELP STEM peers in almost every area of science motivation. After over three years of implementation of the NSF-funded STEM Scholars Engaging in Local Problems program, the recruitment effort has grown significantly in STEM fields in the university. Within the two cohorts, the most common majors were environmental science and engineering. While 100% of Cohorts 1 and 2 students were retained into the Fall semester of the second year, two students from Cohort 1 left the program between the third and fourth semesters of their studies. While one student from Cohort 2 had a leave of absence, they have returned to continue their studies. The support system formed among the SSELP scholars and between the scholars and faculty has benefited the students in both their academic achievement as well as their personal growth. 
    more » « less
  5. null (Ed.)
    As technology advances, data-driven work is becoming increasingly important across all disciplines. Data science is an emerging field that encompasses a large array of topics including data collection, data preprocessing, data visualization, and data analysis using statistical and machine learning methods. As undergraduates enter the workforce in the future, they will need to “benefit from a fundamental awareness of and competence in data science”[9]. This project has formed a research-practice partnership that brings together STEM+C instructors and researchers from three universities and education research and consulting groups. We aim to use high-frequency monitoring data collected from real-world systems to develop and implement an interdisciplinary approach to enable undergraduate students to develop an understanding of data science concepts through individual STEM disciplines that include engineering, computer science, environmental science, and biology. In this paper, we perform an initial exploratory analysis on how data science topics are introduced into the different courses, with the ultimate goal of understanding how instructional modules and accompanying assessments can be developed for multidisciplinary use. We analyze information collected from instructor interviews and surveys, student surveys, and assessments from five undergraduate courses (243 students) at the three universities to understand aspects of data science curricula that are common across disciplines. Using a qualitative approach, we find commonalities in data science instruction and assessment components across the disciplines. This includes topical content, data sources, pedagogical approaches, and assessment design. Preliminary analyses of instructor interviews also suggest factors that affect the content taught and the assessment material across the five courses. These factors include class size, students’ year of study, students’ reasons for taking class, and students’ background expertise and knowledge. These findings indicate the challenges in developing data modules for multidisciplinary use. We hope that the analysis and reflections on our initial offerings have improved our understanding of these challenges, and how we may address them when designing future data science teaching modules. These are the first steps in a design-based approach to developing data science modules that may be offered across multiple courses. 
    more » « less