We consider user retention analytics for online freemium role-playing games (RPGs). RPGs constitute a very popular genre of computer-based games that, along with a player’s gaming actions, focus on the development of the player’s in-game virtual character through a persistent exploration of the gaming environment. Most RPGs follow the freemium business model in which the gamers can play for free but they are charged for premium add-on amenities. As with other freemium products, RPGs suffer from the curse of high dropout rates. This makes retention analysis extremely important for successful operation and survival of their gaming portals. Here, we develop a disciplined statistical framework for retention analysis by modelling multiple in-game player characteristics along with the dropout probabilities. We capture players’ motivations through engagement times, collaboration and achievement score at each level of the game, and jointly model them using a generalized linear mixed model (glmm) framework that further includes a time-to-event variable corresponding to churn. We capture the interdependencies in a player’s level-wise engagement, collaboration, achievement with dropout through a shared parameter model. We illustrate interesting changes in player behaviours as the gaming level progresses. The parameters in our joint model were estimated by a Hamiltonian Monte Carlo algorithm which incorporated a divide-and-recombine approach for increased scalability in glmm estimation that was needed to accommodate our large longitudinal gaming data-set. By incorporating the level-wise changes in a player’s motivations and using them for dropout rate prediction, our method greatly improves on state-of-the-art retention models. Based on data from a popular action based RPG, we demonstrate the competitive optimality of our proposed joint modelling approach by exhibiting its improved predictive performance over competitors. In particular, we outperform aggregate statistics based methods that ignore level-wise progressions as well as progression tracking non-joint model such as the Cox proportional hazards model. We also display improved predictions of popular marketing retention statistics and discuss how they can be used in managerial decision making.
Survival Analysis based Framework for Early Prediction of Student Dropouts
Retention of students at colleges and universities has been a concern among educators for many decades. The consequences of student attrition are significant for students, academic staffs and the universities. Thus, increasing student retention is a long term goal of any academic institution. The most vulnerable students are the freshman, who are at the highest risk of dropping out at the beginning of their study. Therefore, the early identification of "at-risk'' students is a crucial task that needs to be effectively addressed. In this paper, we develop a survival analysis framework for early prediction of student dropout using Cox proportional hazards regression model (Cox). We also applied time-dependent Cox (TD-Cox), which captures time-varying factors and can leverage those information to provide more accurate prediction of student dropout. For this prediction task, our model utilizes different groups of variables such as demographic, family background, financial, high school information, college enrollment and semester-wise credits. The proposed framework has the ability to address the challenge of predicting dropout students as well as the semester that the dropout will occur. This study enables us to perform proactive interventions in a prioritized manner where limited academic resources are available. This is critical in the student retention problem because not only correctly classifying whether a student is going to dropout is important but also when this is going to happen is crucial for a focused intervention. We evaluate our method on real student data collected at Wayne State University. Results show that the proposed Cox-based framework can predict the student dropouts and semester of dropout with high accuracy and precision compared to the other state-of-the-art methods.
more »
« less
- Award ID(s):
- 1527827
- NSF-PAR ID:
- 10021820
- Date Published:
- Journal Name:
- Proceedings of the 25th ACM International on Conference on Information and Knowledge Management
- Page Range / eLocation ID:
- 903 to 912
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
This project will contribute to the national need for well-educated scientists, mathematicians, engineers, and technicians by supporting the retention and graduation of high-achieving, low-income students with demonstrated financial need at Minnesota State University, Mankato. Over its six year duration, this project will fund scholarships to 120 unique full-time students who are pursuing Bachelor of Science degrees in engineering. First semester junior, primarily transfer, students at Iron Range Engineering will receive scholarships for one semester. The Iron Range Engineering (IRE) STEM Scholars Program provides a financially sustainable pathway for students across the nation to graduate with an engineering degree and up to two years of industry experience. Students typically complete their first two years of engineering coursework at community colleges across the country. Students then join IRE and spend one transitional semester gaining training and experience to equip them with the technical, design, and professional skills needed to succeed in the engineering workforce. During the last two years of their education, IRE students work in industry, earning an engineering intern salary, while being supported in their technical and professional development by professors, learning facilitators, and their own peers. The IRE STEM Scholars project will provide access to a financially responsible engineering degree for low-income students by financially supporting them during the transitional semester, which has two financial challenges: university tuition costs are higher than their previous community college costs, and the semester occurs before they are able to earn an engineering co-op income. In addition, the project will provide personalized mentorship throughout students’ pathway to graduation, such as weekly conversations with a mentor. By providing these supports, the IRE STEM Scholars project aims to prepare students to be competitive applicants for the engineering workforce with career development and engineering co-op experience. Because community colleges draw relatively representative proportions of students from a variety of backgrounds, this project has the potential to learn how transfer pathways and co-op education can support financially sustainable pathways to engineering degrees for a more diverse group of students and contribute to the development of a diverse, competitive engineering workforce. The overall goal of this project is to increase STEM degree completion of low-income, high-achieving undergraduates with demonstrated financial need. As part of the scope of this project, a concurrent mixed-methods research study will be done on engineering students’ thriving, specifically their identity, belonging, motivation, and overall wellbeing (or mental and physical health). Student outcomes have previously been measured primarily through academic markers such as graduation rates and GPA. In addition to these outcomes, this project explores ways to better support overall student thriving. This study will address the following research questions: How do undergraduate students’ engineering identity and belongingness develop over time in a co-op-based engineering program? How do undergraduate students’ motivation and identity connect to overall wellbeing in a co-op-based engineering program? In the first year of the IRE STEM Scholars Project, initial interview data describe scholars’ sense of belonging in engineering, prior to their first co-op experiences and survey data describe IRE students’ experiences in co-op and overall sense of belonging. Future work will utilize these values to identify ways to better support the IRE STEM scholars’ identity development as they move into their first co-op experiences. This project is funded by NSF’s Scholarships in Science, Technology, Engineering, and Mathematics program, which seeks to increase the number of low-income academically talented students with demonstrated financial need who earn degrees in STEM fields. It also aims to improve the education of future STEM workers, and to generate knowledge about academic success, retention, transfer, graduation, and academic/career pathways of low-income students.more » « less
-
Additive manufacturing (AM) is prevalent in academic, industrial, and layperson use for the design and creation of objects via joining materials together in a layer upon layer fashion. However, few universities have an undergraduate course dedicated to it. Thus, using NSF IUSE support [grant number redacted for review] from the Exploration and Design Tier of the Engaged Student Learning Track, this project has created and implemented such a course at three large universities: Texas Tech (a Carnegie high research productivity and Hispanic Serving Institution), Kansas State (a Carnegie high research productivity and land grant university) and California State, Northridge (the largest of all the California State campuses and highly ranked in serving underprivileged students). Our research team includes engineering professors and a sociologist trained in assessment and K-12 outreach to determine the effects of the course on the undergraduate and high school students. We are currently in year two of the three years of NSF support. The course focuses on the fundamentals of the three families of prevailing AM processes: extrusion-based, powder-based, and liquid-based, as well as learning about practical solutions to additive manufacturing of common engineering materials including polymers, metals and alloys, ceramics, and composites. It has a lecture plus lab format, in that students learn the fundamentals in a classroom, but then apply and broaden their knowledge in lab projects and independent studies. Additionally, as outreach, we host field trips from local high schools during which the undergraduates give presentations about discrete AM skills, then lead the high school students through a lab project focused on those skills. This creates a pipeline of knowledge about AM for younger students as well as an opportunity for undergraduates to develop leadership and speaking skills while solidifying their knowledge. We are also in the process of uploading videos and lab projects to an online Google Classroom so that those with access to 3D printers in other areas can learn online for free. We are also self-publishing an accompanying textbook and lab manual. Beyond the course itself, one of the innovations of our project is the assessment strategy. For both undergraduates and high school students, we have been able to collect content area knowledge both before and after the class, as well as information about their attitudes towards engineering and self-efficacy beliefs. This has been particularly illuminating in regards to subgroups like women and students of color. Our research questions include: i) what is the knowledge growth about AM during this course? ii) does this differ by university? iii) does this differ by gender or race? iv) what are the best ways to make this course portable to other universities? Preliminary results indicate a statistically significant improvement in knowledge for all students. This was particularly true for women, which may indicate the promise of AM courses in decreasing the female dropout rate in engineering. Attitudes towards engineering and self-efficacy perceptions also differed after the class, but in varying ways by demographic subgroups and university. This will be explored more in the paper.more » « less
-
Background & Program Description: The link between student engagement and retention is well-established in the education literature. As a result, many colleges have developed first-year experience programs to engage students in early technical work and to promote community-building. However, many of these student success programs require participation in extracurricular activities, which require time outside of class. Yet time for extracurricular activities is a luxury that many students of low socioeconomic status (SES) cannot afford due to family or work obligations. The Scholarships in STEM (S-STEM) program, funded by the National Science Foundation, provides crucial financial support to high-achieving low-SES STEM students. The S-STEM scholarships give students the option to work less or not at all. The intended result is that students regain the time afforded to their more privileged peers, thereby also giving them the opportunity to more effectively engage with their institution, studies, and peers. The Endeavour Program is a two-year program that incorporates the S-STEM financial support into a multi-faceted and multi-college program in STEM designed to increase the level of student engagement in school. The scholars, who are recruited from three colleges, take classes together, work on hands-on team projects, attend professional and personal development events, participate in outreach events, and conduct research with faculty mentors. Over the course of the two-year program, four dimensions of student engagement (academic, behavioral, cognitive, and affective) are tracked to determine the appropriateness of using these engagement levels as predictors of success. Results: Two cohorts of 20 students were recruited in the fall of 2017 and in the fall of 2018. The first cohort completed the two-year program in the spring of 2020, and the second cohort began the second year of the program in the fall of 2020. No third cohort was recruited in 2020 due the Covid19 pandemic. The third and fourth cohorts will now enter the program in the fall of 2021 and the fall of 2022 respectively. Overall, the results of the Endeavour Program have been positive. The final retention outcome for the first cohort (the only cohort to complete the program thus far) was 85% (17/20). Retention for the second cohort is currently at 100% (20/20). Initial results show that the S-STEM scholars are performing academically as well as their peers who do not share the same risk factors. In addition, the number of completed hours is also on par with their peers. However, the most significant gains were observed in the qualitative data. Students expressed fears and anxieties about the high school to college transition and reported that the guidance provided and the community formed through the Endeavour Program alleviated many of those negative emotions. The full paper shows student engagement data obtained over time for the first and second cohorts as well as lessons learned and directions for future work. Also, examples of advising charts created in an engagement data dashboard show how the quantitative engagement data has been compiled and organized to show early warning signs for current and future cohorts.more » « less
-
null (Ed.)The Academy of Engineering Success (AcES) program, established in 2012 and supported by NSF S-STEM award number 1644119 throughout 2016-2021, employs literature-based, best practices to support and retain underprepared and underrepresented students in engineering through graduation with the ultimate goal of diversifying the engineering workforce. A total of 71 students, including 21 students supported by S-STEM scholarships, participated in the AcES program between 2016-2019 at a large R1 institution in the mid-Atlantic region. All AcES students participate in a common program during their first year, comprised of: a one-week summer bridge experience, a common fall professional development course and spring “Engineering in History” course, and a common academic advisor. These students also have opportunities for: (1) faculty-student, student-student, and industry mentor-student interaction, (2) academic support and student success education, and (3) major and career exploration – all designed to help students develop feelings of institutional inclusion, engineering self-efficacy and identity, and academic and professional success skills. They also participate in the GRIT, Longitudinal Assessment of Engineering Self-Efficacy (LAESE), and the Motivated Strategies for Learning Questionnaire (MSLQ) surveys plus individual and focus group interviews at the start, midpoint, and end of each fall semester and at the end of the spring semester. The surveys provide a measure of students’ GRIT, their beliefs related to the intrinsic value of engineering and learning, their feelings of inclusion and test anxiety, and their self-efficacy related to engineering, math, and coping skills. The interviews provide information related to the student experience, feelings of inclusion, and program impact. Institutional data, combined with the survey and interview responses, are used to examine four research questions designed to examine the relationship of the elements of the AcES program to participants’ academic success and retention in engineering. Early analyses of the student retention data and survey responses from the 2017 and 2018 cohorts indicated students who ultimately left engineering before the start of their second year initially scored higher in areas of engineering self-efficacy and test anxiety, than those who stayed in engineering, while those who retained to the second year began their engineering education with lower self-efficacy scores, but higher scores related to the belief in the intrinsic value of engineering, learning strategy use, and coping self-efficacy. These results suggest that students who start with unrealistically high expectations of their performance leave engineering at higher rates than students who start with lower personal performance expectations, but have stronger value of the field and strategies for meeting challenges. These data appear to support the Kruger-Dunning effect in which students with limited knowledge of a specific field overestimate their abilities to perform in that area or underestimate the level of effort success may require. This paper will add an analysis of the academic success and retention data from 2019 cohort to this research, discuss the impact of COVID-19 to this program and research, as well as illuminate the quantitative results with the qualitative data from individual and focus group interviews regarding the aspects of the AcES program that impact student success, their expectations and methods for overcoming academic challenges, and their feelings of motivation and inclusion.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/2983323.2983351
