More Like this

1. Seeding Course Forums using the Teacher-in-the-Loop

   https://doi.org/10.1145/3448139.3448142  

   Shusterman, Einat ; Kim, Hyunsoo Gloria ; Facciotti, Marc ; Igo, Michele ; Sripathi, Kamali ; Karger, David ; Segal, Avi ; Gal, Kobi ( April 2021 , LAK21: 11th International Learning Analytics and Knowledge Conference)

   null (Ed.)

   Online forums are an integral part of modern day courses, but motivating students to participate in educationally beneficial discussions can be challenging. Our proposed solution is to initialize (or “seed”) a new course forum with comments from past instances of the same course that are intended to trigger discussion that is beneficial to learning. In this work, we develop methods for selecting high-quality seeds and evaluate their impact over one course instance of a 186-student biology class. We designed a scale for measuring the “seeding suitability” score of a given thread (an opening comment and its ensuing discussion). We then constructed a supervised machine learning (ML) model for predicting the seeding suitability score of a given thread. This model was evaluated in two ways: first, by comparing its performance to the expert opinion of the course instructors on test/holdout data; and second, by embedding it in a live course, where it was actively used to facilitate seeding by the course instructors. For each reading assignment in the course, we presented a ranked list of seeding recommendations to the course instructors, who could review the list and filter out seeds with inconsistent or malformed content. We then ran a randomized controlled study, in which one group of students was shown seeds that were recommended by the ML model, and another group was shown seeds that were recommended by an alternative model that ranked seeds purely by the length of discussion that was generated in previous course instances. We found that the group of students that received posts from either seeding model generated more discussion than a control group in the course that did not get seeded posts. Furthermore, students who received seeds selected by the ML-based model showed higher levels of engagement, as well as greater learning gains, than those who received seeds ranked by length of discussion. 

   more » « less
2. Toward Benchmarking Student Progress in Mechanics: Assessing Learning Cycles through Mastery Learning and Concept Questions

   Papadopoulos, C. ; Santiago-Román, A. I. ; Hillman, E. F. ; Figueroa, G. L. ; Vega Morales, I. ( July 2021 , ASEE Virtual Annual Conference)

   null (Ed.)

   In mechanics, the standard 3-credit, 45-hour course is sufficient to deliver standard lectures with prepared examples and questions. Moreover, it is not only feasible, but preferable, to employ any of a variety of active learning and teaching techniques. Nevertheless, even when active learning is strategically used, students and instructors alike experience pressure to accomplish their respective learning and teaching goals under the constraints of the academic calendar, raising questions as to whether the allocated time is sufficient to enable authentic learning. One way to assess learning progress is to examine the learning cycles through which students attempt, re-think, and re-attempt their work. This article provides data to benchmark the time required to learn key Statics concepts based on results of instruction of approximately 50 students in a Statics class at a public research university during the Fall 2020 semester. Two parallel techniques are employed to foster and understand student learning cycles. • Through a Mastery Based Learning model, 15 weekly pass/fail “Mastery Tests” are given. Students who do not pass may re-test with a different but similar test on the same topic each week until the semester’s conclusion. The tests are highly structured in that they are well posed and highly focused. For example, some tests focus only on drawing Free Body Diagrams, with no equations or calculations. Other tests focus on writing equilibrium equations from a given Free Body Diagram. Passing the first six tests is required to earn the grade of D; passing the next three for C; the next three for B; and the final three for A. Evaluations include coding of student responses to infer student reasoning. Learning cycles occur as students repeat the same topics, and their progress is assessed by passing rates and by comparing evolving responses to the same test topics. • Concept Questions that elicit qualitative responses and written explanations are deployed at least weekly. The learning cycle here consists of students answering a question, seeing the overall class results (but without the correct answer), having a chance to explore the question with other students and the instructor, and finally an opportunity to re-answer the same question, perhaps a few minutes or up to a couple days later. Sometimes, that same question is given a third time to encourage further effort or progress. To date, results from both cycles appear to agree on one important conclusion: the rate of demonstrated learning is quite low. For example, each Mastery Test has a passing rate of 20%-30%, including for students with several repeats. With the Concept Questions, typically no more than half of the students who answered incorrectly change to the correct answer by the time of the final poll. The final article will provide quantitative and qualitative results from each type of cycle, including tracking coded responses on Mastery Tests, written responses on Concept Questions, and cross-comparisons thereof. Additional results will be presented from student surveys. Since the Mastery Tests and Concept Questions follow typical Statics topics, this work has potential to lead to a standardized set of benchmarks and standards for measuring student learning – and its rate – in Statics. 

   more » « less
3. Instructor Use of Movable Furniture and Technology in Flexible Classroom Spaces

   Johnson, A.W. ; Swenson, J.E.S. ; Wiwel, C.R. ; Finelli, C.J. ( June 2019 , ASEE annual conference & exposition proceedings)

   Flexible classroom spaces, which have movable tables and chairs that can be easily rearranged into different layouts, make it easier for instructors to effectively implement active learning than a traditional lecture hall. Instructors can move throughout the room to interact with students during active learning, and they can rearrange the tables into small groups to facilitate conversation between students. Classroom technology, such as wall-mounted monitors and movable whiteboards, also facilitates active learning by allowing students to collaborate. In addition to enabling active learning, the flexible classroom can still be arranged in front-facing rows that support traditional lecture-based pedagogies. As a result, instructors do not have to make time- and effort-intensive changes to the way their courses are taught in order to use the flexible classroom. Instead, they can make small changes to add active learning. We are in the second year of a study of flexible classroom spaces funded by the National Science Foundation’s Division of Undergraduate Education. This project asks four research questions that investigate the relationships between the instructor, the students, and the classroom: 1) What pedagogy do instructors use in a flexible classroom space? 2) How do instructors take advantage of the instructional affordances (including the movable furniture, movable whiteboards, wall-mounted whiteboards, and wall-mounted monitors) of a flexible classroom? 3) What is the impact of faculty professional development on instructors’ use of flexible classroom spaces? and 4) How does the classroom influence the ways students interpret and engage in group learning activities? In the first year of our study we have developed five research instruments to answer these questions: a three-part classroom observation protocol, an instructor interview protocol, two instructor surveys, and a student survey. We have collected data from nine courses taught in one of ten flexible classrooms at the University of Michigan during the Fall 2018 semester. Two of these courses were first-year introduction to engineering courses co-taught by two instructors, and the other seven courses were sophomore- and junior-level core technical courses taught by one instructor. Five instructors participated in a faculty learning community that met three times during the semester to discuss active learning, to learn how to make the best use of the flexible classroom affordances, and to plan activities to implement in their courses. In each course we gathered data from the perspective of the instructor (through pre- and post-semester interviews), the researcher (through observations of three class meetings with our observation protocol), and the students (through conducting a student survey at the end of the semester). This poster presents qualitative and qualitative analyses of these data to answer our research questions, along with evidence based best practices for effectively using a flexible classroom. 

   more » « less
4. Demographic Mediation of the Relationship Between Engagement and Performance in a Blended Dynamics Engineering Course

   Haney, C. ; Dridi, A.M. ; Rhoads, J. ; DeBoer, J. ; Berger, E. ( December 2021 , Proceedings of REES AAEE 2021)

   ABSTRACT CONTEXT This paper examines an engineering dynamics course at Purdue University that was specifically designed to create an active, blended, and collaborative environment. In addition to in-person classes and support, students have access to blended content such as solution videos, mechanics visualizations, a course discussion forum, and interactive simulations. PURPOSE Many studies have shown that students’ engagement in an online discussion forum enhances their learning performance (Davies & Graff, 2005; Hrastinski, 2008). However, our previous research showed that students’ engagement in the online forum of our dynamics course differed significantly across students’ demographics. We showed that women, white, or Asian American students were more likely to be involved in online discussions than men, international, or Hispanic students (Duan et al., 2018). In this paper, we take the previous analysis further by examining whether the observed differences in online student engagement mediate or moderate student performance. APPROACH To answer our research question, we will first investigate the mediation effect by creating two models. A first model with race/international status as the mediating variable and gender identity as a control variable, and a second model with gender identity as the mediating variable and race/international status as a control. Second, we will investigate the moderation effect of demographic factors by creating a regression model including interaction terms to show the relationship of each demographic’s discussion forum engagement to overall performance. The goal of investigating these interaction terms is to determine if a moderating relationship exists where demographic factors impact online engagement, which in turn impact course performance. CONCLUSIONS We find that gender identity is the only significant demographic factor that moderates the effect of a student’s engagement on their performance. Based on the findings of our previous work, students of various racial and ethnic identities do engage differently in the discussion forum. However, this analysis was unable to detect any significant difference in student engagement based on demographics. Our paper contributes to understanding the mechanisms through which students’ engagement can translate into academic performance by focusing on their demographic background. The moderating role of students’ demographic background calls for a more targeted design of instructional tools in blended and collaborative environments to better support students from various demographic backgrounds. 

   more » « less
5. #lets-discuss: Analyzing student affect in course forums using emoji.

   Blobstein, A. ; Gal, K. ; Karger, D. ; Facciotti, M. ; Kim, H. ; Almahmoud, J. ; Sripathi, K. ( July 2022 , Proceedings of the 15th International Conference on Educational Data Mining)

   Mitrovic, A. ; Bosch, N. (Ed.)

   Emoji are commonly used in social media to convey attitudes and emotions. While popular, their use in educational contexts has been sparsely studied. This paper reports on the students’ use of emoji in an online course forum in which students annotate and discuss course material in the margins of the online textbook. For this study, instructors created 11 custom emoji-hashtag pairs that enabled students to quickly communicate affects and reactions in the forum that they experienced while interacting with the course material. Example reporting includes, inviting discussion about a topic, declaring a topic as interesting, or requesting assistance about a topic. We analyze emoji usage by over 1,800 students enrolled in multiple offerings of the same course across multiple academic terms. The data show that some emoji frequently appear together in posts associated with the same paragraphs, suggesting that students use the emoji in this way to communicating complex affective states. We explore the use of computational models for predicting emoji at the post level, even when posts are lacking emoji. This capability can allow instructors to infer information about students’ affective states during their ”at home” interactions with course readings. Finally, we show that partitioning the emoji into distinct groups, rather than trying to predict individual emoji, can be both of pedagogical value to instructors and improve the predictive performance of our approach using the BERT language model. Our procedure can be generalized to other courses and for the benefit of other instructors. 

   more » « less