Data-driven work has the potential to advance research in medicine, psychology, sociology, economics, education, and beyond. Ensuring that students who are interested in these fields are part of an environment that prepares them for the research process also means encouraging positive exposure to statistics and data analysis work. It has been claimed that students' attitudes are the second most important outcome for an introductory statistics course, after increasing conceptual understanding (Ramirez et al., 2012). Students who have negative feelings about their experience studying statistics are less likely to gravitate toward work in fields that require statistical literacy, and less likely to make use of statistical information to support evidence-based practice (Petocz & Reid, 2005;Zanakis & Valenzi, 1997). Therefore, crafting an introductory statistics course that leaves students eager to engage in statistical thinking beyond the course, may be an important indicator of course success.
Previous researchers have examined how flipped classrooms (Carlson & Winquist, 2011;Heringer et al., 2019;Kahn & Watson, 2018;Nielson et al., 2018;Wilson, 2013;Winquist & Carlson, 2014) and inquiry-based projects (Bailey et al., 2013) can be used to improve student attitudes and academic performance in statistics. Teaching statistics through inquiry-based learning can also encourage higherlevel development of statistical concepts (DaSilva & Pinto, 2014) that inspire students to engage in statistical inquiry beyond the course (Fagundes et al., 1999). Research on these pedagogical approaches contributes to the rationale behind the development of Passion-Driven Statistics, a multidisciplinary, project-based introductory statistics course (Dierker et al., 2012).
Passion-Driven Statistics was introduced into the curriculum in the fall of 2009 at a selective liberal arts college. The course aims to engage students by allowing them to experience statistics through the lens of a research question of their choosing in a flipped classroom format. The project-based course follows each of the recommendations outlined in the Guidelines for Assessment and Instruction in Statistics Education (GAISE) Report (Aliaga et al., 2005;American Statistical Association [ASA], 2014; Carver et al., 2016). For example, the course teaches statistical thinking by asking students to choose their own research project, giving them time to think critically about statistical issues (Wild & Pfannkuch, 1999;Chance, 2002;DaSilva & Pinto, 2014). This in turn shows students how data can be used to answer questions of interest to them and to society (Neumann et al., 2013). The course requires students to tackle complicated real-world questions that involve more than one or two variables (De Veaux, 2015) and emphasizes practical problem-solving skills that are necessary to answer statistical questions (Garfield et al., 2012). Projects are presented at the end of the semester at a research poster session in which students have the opportunity to describe their process of inquiry, including the different decisions made along the way, their premises, conclusions and any barriers that they faced.
One of the goals of the course is to increase the number of students with exposure to statistics, generally and, more specifically, among under-represented minority (URM) students and for students from diverse academic backgrounds. Previous work has demonstrated that the project-based course enrolled higher numbers of URM students compared to a traditional introductory statistics course (Dierker et al., 2015). Data analysis programming is another focus of the project-based course, and higher rates of female and URM enroll in the project-based course compared to both a general introductory programming course and an introductory course representing a gateway to the computer science major (Cooper & Dierker, 2017). Overall, when compared to both traditional introductory statistics and introductory programming courses, the project-based course attracted students from a much wider range of mathematical aptitude as measured by their Math SAT scores (Cooper & Dierker, 2017;Dierker et al., 2015).
Another aim for the course is to inspire further study in statistics and quantitative methods. The project-based course has shown increased reported interest from students in pursuing advanced coursework in statistics (Dierker et al., 2018). Specifically, students in the project-based course were more likely to report wanting to take courses in advanced statistical techniques, data visualization, and programming. Given these promising findings, we sought to evaluate the possible impact of the projectbased course on actual future enrollment, not merely interest or intent. The purpose of this study was to compare future data-oriented course enrollment of students completing the project-based course against three alternative introductory courses. We hypothesized that since students enrolled in the multidisciplinary project-based course have been found to have more positive experiences, it would also lead to differences in future course decision making.
Administrative data were examined for individual student enrollment in four introductory statistics courses: 1) a multidisciplinary, project-based course; 2) an activity-based psychology course; and introductory statistics courses offered through 3) the economics, or 4) mathematics departments between fall semester 2009 and spring semester 2018. Each course was a full semester, meeting 2 to 3 times a week. Only students for whom one of these introductory courses was their first college-level statistics course and who were not concurrently enrolled in any other data analysis course were considered in the present analyses (n = 3711). Students who had taken a course focused on statistical concepts, applied data analysis, and/or use of statistical software before enrolling in one of the four introductory statistics courses were excluded from the analyses.
The multidisciplinary, project based introductory statistics course was offered through the Quantitative Analysis Center, a collaborative effort of academic and administrative departments that supports quantitative analysis across the curriculum and provides an institutional framework for collaboration across departments and disciplines in the area of statistics and data analysis. Wesleyan University (2018) described "Applied Data Analysis" as a project-based course, [in which] you will have the opportunity to answer questions that you feel passionately about through independent research based on existing data. Students will have the opportunity to develop skills in generating testable hypotheses, conducting a literature review, preparing data for analysis, conducting descriptive and inferential statistical analyses, and presenting research findings. The course offers unlimited one-on-one support, ample opportunities to work with other students, and training in the skills required to complete a project of your own design. These skills will prepare you to work in many different research labs across the University that collect empirical data. It is also an opportunity to fulfill an important requirement in several different majors. The course format was listed as "laboratory" and one of several code-based statistical software platforms was used in each section (i.e., SAS, R, Stata, SPSS). The assessments were common between instructors and included exams, project components, a research paper, and a final project and presentation.
Wesleyan University (2018) described the activity-based introductory statistics course offered though the Psychology Department, "Statistics: An Activity-Based Approach", as introducing [students] to the concepts and methods used in the analysis of quantitative data in the behavioral and life sciences. The approach will emphasize activity-based learning. Lectures will be used for the initial presentation and wrap-up of topics, but most class time will be devoted to activities in which students perform analyses. The topics covered will include descriptive statistics, sampling distributions, estimation, hypothesis testing, analysis of variance, and regression. The course format was listed as "laboratory" and SPSS was used as the statistical software platform in each section. The assessments varied by instructor and included labs, quizzes, problem sets, and a final project.
Wesleyan University (2018) described the introductory statistics course offered through the Economics Department, "Quantitative Methods in Economics", as "an introduction to quantitative techniques widely used by economists. Topics include various methods of applied statistics that facilitate the understanding of economic literature and the pursuit of empirical research; elements of probability, correlation, multiple regression, and hypothesis testing." Largely a lecture course, workshops on computer skills, use of statistical packages (Stata and E-views), and data access were required approximately 3-4 times during the semester in addition to regularly scheduled class times. The course format was listed as "lecture". The assessments were common between instructors and included homework, midterm exams, a research paper, and a cumulative final exam.
Wesleyan University (2018) described the introductory statistics course offered through the Mathematics Department, "Elementary Statistics", as "covering the topics of organizing data, central measures, measures of variation, distributions, sampling, estimation, conditional probability (Bayes' theorem), hypothesis testing, simple regression and correlation, and analysis of variation." The course format was listed as "lecture" and statistical software was not generally used. The assessments varied by instructor but commonly included exams, homework, and labs.
The multidisciplinary project-based course, the psychology course, and the mathematics course were open to all students and there were no prerequisites for enrollment. Each could be used as one option to fulfill a major requirement for Biology, Neuroscience & Behavior, and Psychology. In addition, each could be applied to the natural sciences and mathematics general education recommendation. As of 2019 these courses could be used toward a certificate in education and a dataanalysis minor. In addition, the multidisciplinary course was an option for fulfilling a major requirement in Government and Sociology and the mathematics course could be used as a requirement for the Molecular Biology & Biochemistry major. Importantly, none of the three courses was specifically required of any student, nor did any represent a single option for fulfilling requirements for any major within the university.
In comparison, enrollment in the economics course required completion of two semesters of college level calculus and an introductory course in economic theory. Students must have completed, or been enrolled in, the Quantitative Methods in Economics course to be accepted into the economics major. The course could also be applied to the social and behavioral sciences general education recommendation.
Typical enrollments per class section included the following: multidisciplinary sections ~25 students, psychology sections ~19 students, economics sections and mathematics sections ~35 students.
Administrative data drawn from students' application to the university, class enrollment each semester, and their chosen major, were supplied by the institutional research office. These data included age of enrollment in the university, gender, international student status, and the following variables:
• Class year. Class year was dichotomized into first-and second-year students vs. third-and fourth-year students. • First Generation: Students self-reported whether or not they were the first generation of their family to attend college.
Bivariate analyses were conducted to examine associations between student background characteristics and first introductory statistics course enrollment (multidisciplinary, psychology, economics, and mathematics). Chi-square Tests of Independence and ANOVA were used for categorical and quantitative variables, respectively. For categorical variables with low cell counts (< 5), Fisher's exact p-values were estimated through Monte Carlo simulation. Significant tests were followed by planned post hoc paired comparisons between students enrolled in the multidisciplinary statistics course and each of the comparison courses. The Duncan test was used to evaluate post hoc paired comparisons for ANOVA and a Bonferroni adjustment of 0.0007 was used to evaluate post hoc paired comparisons for Chi Square Tests of Independence and Fisher's test.
Because of the quasi-experimental design of our study, we used a causal inference technique to achieve matched comparisons for each of our course enrollment outcomes (i.e. enrollment in additional courses measured both as a count and as a binary outcome). A case-control matched analysis is often used in observational studies to reduce selection bias and approximate a randomized trial (Rosenbaum, 2010). A propensity score is the predicted probability of an outcome. It has been shown that a sample matched on propensity score will be similar for all the covariates included in the compution of the propensity score. Thus, matching on the propensity score can reduce the selection bias in an observational study (Rosenbaum, 2010). In these analyses, students completing the multidisciplinary, project-based course were considered the treatment group compared to control groups of students completing one of the comparison courses. Propensity scores were calculated based on student background characteristics. Students were matched on the propensity for enrollment in the multidisciplinary course using the OneToManyMTCH SAS Macro (Parsons, 2004) specifying a 1:1 nearest neighbor match. Matching was evaluated to check whether balance was achieved; that is, whether there were no longer significant differences between treatment (multidisciplinary) and control conditions (psychology, economics and math) on each background variable. Logistic and multiple regression were then used on the matched samples to estimate differences in a student's likelihood of taking one or more additional courses (binary) and mean number of additional courses (count) based on enrollment in an introductory statistics course.
The associations between first introductory statistics course and student background characteristics are presented in Tables 1 and2. These tables allow us to see that there are many student background characteristics that vary between courses. Post hoc paired comparisons between the multidisciplinary statistics course and each of the three other introductory statistics courses showed that the multidisciplinary course enrolled lower rates of first-and second-year students compared to each of the comparison courses. Students taking the multidisciplinary course were also more likely to be Black compared to those enrolling in both the economics and mathematics courses. Students in the multidisciplinary course were slightly older when entering college compared to students completing the psychology and mathematics courses. The multidisciplinary course was found to enroll lower rates of female students and higher rates of international students compared to the psychology course and higher rates of female students and lower rates of international students compared to the economics course. Students taking the multidisciplinary course also differed from students in the economics course in that they were more likely to have a Pell grant and to be a first generation college student, were less likely to be Asian and among those with verified SAT scores, had somewhat higher scores on the SAT writing test and somewhat lower scores on both the SAT math and SAT critical thinking tests. While there were no differences between students taking the multidisciplinary and mathematics courses in terms of SAT scores, those in the multidisciplinary course had somewhat higher scores than psychology statistics students on the SAT critical thinking and SAT math tests. Students in the multidisciplinary course were more likely to have declared a major at the time of enrollment than students in the economics and mathematics courses.
Of the 3711 students taking one of the four introductory courses as their first college level statistics course, 1218 (32.8%) took one or more additional courses with an emphasis on statistical concepts, applied data analysis, and/or use of statistical software. Among those who took at least one additional course, 662 (54.4%) took just one, 272 (22.3%) took two, 129 (10.6%) took three, and 155 (12.7%) took four or more.
When examining specific courses taken by students completing at least one additional course, the multidisciplinary introductory statistics course was the most common single second course among those students originally taking introductory statistics in psychology, economics and mathematics (17.3%, 12.9% and 21.8% of students, respectively, taking at least one follow-up course). Similarly, for students originally enrolling in the mathematics course, the economics introductory statistics course was a common second course (17.8%).
Econometrics was a common second course for students originally taking the introductory statistics course through economics (16.2%), while course credit for peer tutoring of data analysis courses was a common second experience for students originally enrolled in the multidisciplinary course (15.7%). Introduction to Statistical Consulting was often selected as a second course particularly for students originally enrolling in the multidisciplinary and psychology courses (12.5% and 11.4% respectively, compared to 6.7% of students originally taking the introductory economics course and 3.1% of students originally taking the mathematics course).
Taken together, the "working with" courses (i.e., Working with Excel and VBA, Mathematica, Python, R, SAS, SQL Databases, and Stata) were common second courses for students from each of the introductory statistics courses (18.6% of students from the multidisciplinary course, 28.2% from psychology, 22.2% from economics, and 17.8% of students from the mathematics course).
Differences in background characteristics of students enrolled in each of the four courses are summarized in Table 1. Variables retained in the model for calculation of propensity scores and show the pre-matched differences between groups are presented in Table 3.
Using the OneToManyMTCH SAS Macro (Parsons, 2004) and specifying a 1:1 nearest neighbor match, separate data sets were created in which each student taking the multidisciplinary course was matched to a student in each of the comparison courses, yielding three different matched sample pairs. Generating each of the three matched samples successfully eliminated differences on all background variables between students taking the multidisciplinary statistics course and students in each of the comparison courses (i.e., all differences after matching resulted in p > 0.05). Successful matching was achieved for 389 students taking the psychology statistics course, 335 taking the statistics course the economics and 711 students taking the statistics course through the mathematics department.
Based on regression models using each of matched samples, students taking the multidisciplinary statistics course were found to be more likely to take at least one additional course focused on statistical concepts, applied data analysis, and/or use of statistical software compared to students taking either the statistics course offered through the psychology department (25.7% vs. 12.3%, OR = 2.5, 95% CI [1.68-3.59]) or the math department (34.0% vs 25.5%, OR = 1.5, 95% CI [1.20-1.90]). Course counts were also found to differ. That is, students in the multidisciplinary statistics course took a greater number of additional courses compared to students in the statistics course offered through the psychology department (M = 1.4, SD = 0.94 vs. M = 1.2, SD = 0.56, β=0.27, p < 0.0001) or through the mathematics department (M = 1.7, SD = 1.23 vs. M = 1.4, SD = 0.93, β = 0.23, p < 0.0001).
In contrast, students in the multidisciplinary introductory statistics course were less likely to have taken at least one additional course compared to students taking the statistics course offered through the economics department (38.5% vs. 49.6%, OR = 0.64, 95% CI [0.47-0.87]). Students from the multidisciplinary course were also found to have taken a smaller number of additional courses compared to students in the statistics course offered through the economics department (M = 1.8, SD =. 1.37 vs. M = 2.1, SD = 1.60, β = -0.31, p < 0.007). Because two semesters of college calculus were required for entry into the statistics course offered through economics, but not the multidisciplinary course, a second matched sample was created that included a variable for completion of calculus in the matching procedure. In this model, balance between the treatment and control groups was achieved on every variable except Black ethnicity, suggesting the model was unable to generate a fully matched sample.
There are a variety of student outcomes that can be examined when attempting to determine the success of a course in introductory statistics. Previous publications of this multidisciplinary course show it attracts a more academically and demographically diverse audience (Dierker et al., 2015). Additionally, it has been found to be associated with increases in confidence and interest in pursuing additional coursework in data analysis and statistics (Dierker et al., 2018).
The present study is the first to evaluate the success and potential impact of this multidisciplinary, project-based course on academic decision-making in the form of additional courses in applied statistics and data analysis. These finding suggest that the multidisciplinary course may contribute to the decision for students to enroll in future courses in statistics and data analysis, when compared to the psychology and mathematics department courses. The findings also show that students in the economics course were more likely to take future courses in statistics and data analysis compared to the multidisciplinary course. Since the economics course requires two semesters of calculus, however, it seems plausible that the students who enter the economics course may have a higher pre-existing interest in mathematics and statistics compared to students in the other three courses. Therefore, since balance was not achieved when including completion of these prerequisites, further investigation would be necessary to compare the two courses.
Identifying appropriate comparison groups in this type of research be challenging given that many educational settings have created structured requirements that give students little choice or flexibility that might uncover preferences for particular curriculum. Though quasi experimental, a strength of the present study is that it was conducted at a university with an open curriculum (i.e., no requirements other than completing 8 semesters and completing a major) in which the multidisciplinary project-based statistics course can provide an alternative for fulfilling major requirements across diverse disciplines. Therefore, many students at the university where these introductory courses were held had the flexibility of taking courses based on intellectual curiosity alone, which may not be possible at schools with more rigid university requirements.
Overall, juniors and seniors were more likely than freshmen and sophomores to enroll in the multidisciplinary course compared to the statistics courses offered through the psychology, economics, and mathematics departments. Given that participation by upperclassmen in introductory statistics courses provides less opportunity to influence students' academic trajectories, more needs to be done to encourage enrollment of students in high quality statistics courses as early as possible in their academic careers (Seymour et al., 2004;ASA, 2014).
The differences found among the different introductory approaches may have been influenced by one or more of the many differences between courses (e.g. project-based approach, flipped classroom, use of statistical software, etc.). Therefore, our findings do not suggest that any method on its own will increase future course interest in statistics.
In addition, in assessing these course differences there is the potential that the differences found between the courses could be partially the result of instructor differences. One limitation of the available data is there is no information on instructors to address the multilevel nature present in the data. Future work would be needed to address the extent that the course itself effects future course decisions controlling for these instructor-level differences.
Since the number of course offerings increased over the time period, the estimates found for both (1) likelihood of enrolling in a future course, and (2) the average number of future courses is likely to be underestimated across all four introductory courses studied. As the number of upper-level courses in statistics and data analysis continues to grow, it would be interesting to examine how the trends vary over time between the four introductory courses.
The methodology in Section 3.3 only attempts to achieve balance between the observed covariates listed in Table 3. There are potentially other confounding variables that are not accounted for in this approach and our attempt to estimate the causal effect of the course may be limited by these unobserved covariates (e.g., participation in AP statistics or prior interest in statistics and data analysis).
Given that no single course will adequately prepare students for either the amount or complexity of data they will encounter as professionals and as citizens (Collins & Halverson, 2010;Cobb, 2007;Horton, 2015), modern courses need to focus on imparting a deep interest among students and a desire to continue learning statistics. Encouraging further study of statistics beyond the introductory level will help solidify and broaden statistical literacy (Chance, 2002). Since the progression of many fields requires some degree of quantitative research, the goal is to give students more tools to aid them in the scientific process within their chosen field.
While more work is needed to evaluate the potential impact of the multidisciplinary, project-based course in terms of other academic decisions and career paths, the present findings suggest that a course in which students have the opportunity to answer questions that they feel passionately about through independent research, may represent an effective option that motivates students to pursue further course work. We believe that this multidisciplinary, project-based model can benefit other schools and are currently disseminating it within high schools, state universities and community colleges. We are happy to share our course materials with others and encourage instructors to consider using a multidisciplinary project-based approach in their own classrooms (http://passiondrivenstatistics.com/).