skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Towards Generalizable Detection of Urgency of Discussion Forum Posts
Students who take an online course, such as a MOOC, use the course's discussion forum to ask questions or reach out to instructors when encountering an issue. However, reading and responding to students' questions is difficult to scale because of the time needed to consider each message. As a result, critical issues may be left unresolved, and students may lose the motivation to continue in the course. To help address this problem, we build predictive models that automatically determine the urgency of each forum post, so that these posts can be brought to instructors' attention. This paper goes beyond previous work by predicting not just a binary decision cut-off but a post's level of urgency on a 7-point scale. First, we train and cross-validate several models on an original data set of 3,503 posts from MOOCs at University of Pennsylvania. Second, to determine the generalizability of our models, we test their performance on a separate, previously published data set of 29,604 posts from MOOCs at Stanford University. While the previous work on post urgency used only one data set, we evaluated the prediction across different data sets and courses. The best-performing model was a support vector regressor trained on the Universal Sentence Encoder embeddings of the posts, achieving an RMSE of 1.1 on the training set and 1.4 on the test set. Understanding the urgency of forum posts enables instructors to focus their time more effectively and, as a result, better support student learning.  more » « less
Award ID(s):
1931419
PAR ID:
10448493
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Proceedings of the 16th International Conference on Educational Data Mining
Page Range / eLocation ID:
302–309
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; (, Proceedings of the 15th International Conference on Educational Data Mining)
    Mitrovic, A.; Bosch, N. (Ed.)
    Emoji are commonly used in social media to convey attitudes and emotions. While popular, their use in educational contexts has been sparsely studied. This paper reports on the students’ use of emoji in an online course forum in which students annotate and discuss course material in the margins of the online textbook. For this study, instructors created 11 custom emoji-hashtag pairs that enabled students to quickly communicate affects and reactions in the forum that they experienced while interacting with the course material. Example reporting includes, inviting discussion about a topic, declaring a topic as interesting, or requesting assistance about a topic. We analyze emoji usage by over 1,800 students enrolled in multiple offerings of the same course across multiple academic terms. The data show that some emoji frequently appear together in posts associated with the same paragraphs, suggesting that students use the emoji in this way to communicating complex affective states. We explore the use of computational models for predicting emoji at the post level, even when posts are lacking emoji. This capability can allow instructors to infer information about students’ affective states during their ”at home” interactions with course readings. Finally, we show that partitioning the emoji into distinct groups, rather than trying to predict individual emoji, can be both of pedagogical value to instructors and improve the predictive performance of our approach using the BERT language model. Our procedure can be generalized to other courses and for the benefit of other instructors. 
    more » « less
  2. ; ; ; ; ; ; ; (, LAK21: 11th International Learning Analytics and Knowledge Conference)
    null (Ed.)
    Online forums are an integral part of modern day courses, but motivating students to participate in educationally beneficial discussions can be challenging. Our proposed solution is to initialize (or “seed”) a new course forum with comments from past instances of the same course that are intended to trigger discussion that is beneficial to learning. In this work, we develop methods for selecting high-quality seeds and evaluate their impact over one course instance of a 186-student biology class. We designed a scale for measuring the “seeding suitability” score of a given thread (an opening comment and its ensuing discussion). We then constructed a supervised machine learning (ML) model for predicting the seeding suitability score of a given thread. This model was evaluated in two ways: first, by comparing its performance to the expert opinion of the course instructors on test/holdout data; and second, by embedding it in a live course, where it was actively used to facilitate seeding by the course instructors. For each reading assignment in the course, we presented a ranked list of seeding recommendations to the course instructors, who could review the list and filter out seeds with inconsistent or malformed content. We then ran a randomized controlled study, in which one group of students was shown seeds that were recommended by the ML model, and another group was shown seeds that were recommended by an alternative model that ranked seeds purely by the length of discussion that was generated in previous course instances. We found that the group of students that received posts from either seeding model generated more discussion than a control group in the course that did not get seeded posts. Furthermore, students who received seeds selected by the ML-based model showed higher levels of engagement, as well as greater learning gains, than those who received seeds ranked by length of discussion. 
    more » « less
  3. ; ; ; (, International Conference on Educational Data Mining)
    Lynch, Collin F.; Merceron, Agathe; Desmarais, Michel; Nkambou, Roger (Ed.)
    Discussion forums are the primary channel for social interaction and knowledge sharing in Massive Open Online Courses (MOOCs). Many researchers have analyzed social connections on MOOC discussion forums. However, to the best of our knowledge, there is little research that distinguishes between the types of connections students make based upon the content of their forum posts. We analyze this effect by distinguishing on- and off-topic posts and comparing their respective social networks. We then analyze how these types of posts and their social connections can be used to predict the students’ final course performance. Pursuant to this work we developed a binary classifier to identify on- and off- topic posts and applied our analysis with the hand-coded and predicted labels. We conclude that the post type does affect the relationship between the students and their closest neighbors or community members clustered communities and their closest neighbor to their learning outcomes. 
    more » « less
  4. ; ; ; ; (, Proceedings of REES AAEE 2021)
    ABSTRACT CONTEXT This paper examines an engineering dynamics course at Purdue University that was specifically designed to create an active, blended, and collaborative environment. In addition to in-person classes and support, students have access to blended content such as solution videos, mechanics visualizations, a course discussion forum, and interactive simulations. PURPOSE Many studies have shown that students’ engagement in an online discussion forum enhances their learning performance (Davies & Graff, 2005; Hrastinski, 2008). However, our previous research showed that students’ engagement in the online forum of our dynamics course differed significantly across students’ demographics. We showed that women, white, or Asian American students were more likely to be involved in online discussions than men, international, or Hispanic students (Duan et al., 2018). In this paper, we take the previous analysis further by examining whether the observed differences in online student engagement mediate or moderate student performance. APPROACH To answer our research question, we will first investigate the mediation effect by creating two models. A first model with race/international status as the mediating variable and gender identity as a control variable, and a second model with gender identity as the mediating variable and race/international status as a control. Second, we will investigate the moderation effect of demographic factors by creating a regression model including interaction terms to show the relationship of each demographic’s discussion forum engagement to overall performance. The goal of investigating these interaction terms is to determine if a moderating relationship exists where demographic factors impact online engagement, which in turn impact course performance. CONCLUSIONS We find that gender identity is the only significant demographic factor that moderates the effect of a student’s engagement on their performance. Based on the findings of our previous work, students of various racial and ethnic identities do engage differently in the discussion forum. However, this analysis was unable to detect any significant difference in student engagement based on demographics. Our paper contributes to understanding the mechanisms through which students’ engagement can translate into academic performance by focusing on their demographic background. The moderating role of students’ demographic background calls for a more targeted design of instructional tools in blended and collaborative environments to better support students from various demographic backgrounds. 
    more » « less
  5. ; (, ACM)
    The recent public releases of AI tools such as ChatGPT have forced computer science educators to reconsider how they teach. These tools have demonstrated considerable ability to generate code and answer conceptual questions, rendering them incredibly useful for completing CS coursework. While overreliance on AI tools could hinder students’ learning, we believe they have the potential to be a helpful resource for both students and instructors alike. We propose a novel system for instructor-mediated GPT interaction in a class discussion board. By automatically generating draft responses to student forum posts, GPT can help Teaching Assistants (TAs) respond to student questions in a more timely manner, giving students an avenue to receive fast, quality feedback on their solutions without turning to ChatGPT directly. Additionally, since they are involved in the process, instructors can ensure that the information students receive is accurate, and can provide students with incremental hints that encourage them to engage critically with the material, rather than just copying an AI-generated snippet of code. We utilize Piazza—a popular educational forum where TAs help students via text exchanges—as a venue for GPT-assisted TA responses to student questions. These student questions are sent to GPT-4 alongside assignment instructions and a customizable prompt, both of which are stored in editable instructor-only Piazza posts. We demonstrate an initial implementation of this system, and provide examples of student questions that highlight its benefits. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email