The ultimate goal of using learning analytics dashboards is to improve teaching and learning processes. Instructors that use an analytics dashboard are presented with data about their students and/or about their teaching practices. Despite growing research in analytics dashboards, little is known about how instructors make sense of the data they receive and reflect on it. Moreover, there is limited evidence on how instructors who use these dashboards take further actions and improve their pedagogical practices. My dissertation work addresses these issues by examining instructors’ sense making, reflective practice and subsequent actions taken from classroom analytics in three phases: (a) problem analysis from systematic literature review (current), (b) implementation and examination of instructors’ sense-making and reflective practice (current) and (c) human-centered approaches to co-designing instructors’ dashboards with stakeholders (current). The findings will contribute to the conceptual basis of instructors’ change of their pedagogical practices and practical implications of human-centered principles in designing effective dashboards.
more »
« less
A Closer Look at Instructor Use and Sensemaking Processes of Analytics Dashboards: Past, Present, and Future
There is a growing interest in the research and use of automated feedback dashboards that display classroom analytics; yet little is known about the detailed processes instructors use to make sense of these tools, and to determine the impact on their teaching practices. This research was conducted at a public Midwestern university within the context of an automated classroom observation and feedback implementation project. Fifteen engineering instructors engaged in this research. The overarching goal was to investigate instructor teaching beliefs, pedagogical practices, and sensemaking processes regarding dashboard use. A grounded theory approach was used to identify categories related to instructor perceptions. Results revealed that instructor experiences inform both their present use of the dashboard and consequential future actions. A model is presented that illustrates categories included in instructor pre-use, use, and post-use of an automated feedback dashboard. An extension to this model is presented and accompanied by recommendations for a more effective future use of automated dashboards. The model’s practical implications inform both instructors and designers on effective design and use of dashboards, ultimately paving a way to improve pedagogical practices and instruction
more »
« less
- Award ID(s):
- 2021118
- PAR ID:
- 10556637
- Publisher / Repository:
- Journal of Learning Analytics
- Date Published:
- Journal Name:
- Journal of Learning Analytics
- Volume:
- 11
- Issue:
- 2
- ISSN:
- 1929-7750
- Page Range / eLocation ID:
- 1 to 22
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This complete research paper details an investigation into the influence of instructors' pedagogical knowledge on their classroom practices in the context of teaching first-year engineering courses. Background and Motivation: First-year engineering courses serve as the foundational setting in which students are introduced to the field of engineering as well as the pedagogies specific to engineering teaching and learning. These courses are pivotal in equipping students with essential knowledge and skills, setting the stage for their success in more advanced engineering topics. Understanding how instructors' pedagogical knowledge affects their teaching practices is crucial. Pedagogical knowledge encompasses a wide range of techniques to effectively manage a classroom and engage students. This includes the use of instructional strategies that cater to diverse student needs, the design of impactful and engaging lesson plans, etc. There is, however, limited research on how instructors’ pedagogical knowledge influences their classroom practices in first-year engineering courses. Hence, it seems opportune and essential to conduct additional research on engineering instructors' classroom practices. Research Question: The central question driving this research is: How does instructors' pedagogical knowledge influence their pedagogical practices for first-year engineering courses? Method: For this study, we chose the model of teacher professional knowledge and skill (TPK&S) that includes pedagogical content knowledge (PCK). The model recognizes the fundamental importance of pedagogical knowledge and contextualizes PCK within that framework, encompassing the intricate nature of teaching and learning. A descriptive case study was utilized as a methodology for this work to delve into the phenomenon. The context of the study was a first-year introductory engineering course offered at a large public research institution. This is a pilot study for an NSF-funded project (blinded for review), the study involved two instructors, Chandler and Joey (pseudonyms), chosen through purposive sampling, with varying levels of teaching experience. Data collection involved direct classroom observation using the Teaching Dimensions Observation Protocol (TDOP) and semi-structured interviews conducted after the observations. The interviews were conducted after classroom observations, allowing the researcher to explore specific findings from the observations. Results: Thematic analysis was used to categorize the data based on the constructs of the theoretical framework. The analysis revealed three major themes: (a) Instructors' topic-specific professional knowledge significantly influences their pedagogical practices. Both instructors adapt their teaching methods based on their understanding of course material and students' difficulties. (b) The interaction between instructors' personal pedagogical content knowledge (PCK) and the classroom context shapes their classroom practices. (c) Instructors' beliefs and prior knowledge act as amplifiers or filters based on the situation. They filter out their teaching practices that do not align with their beliefs and prior knowledge. Conclusion: The findings presented in this paper provide valuable insights into the complex interplay between instructors' pedagogical knowledge and their classroom practices. This work holds significant implications for current and future first-year instructors in that this paper will showcase how instructors use their understanding of the content and their students to teach, which is a critical aspect of helping students successfully integrate into engineering.more » « less
-
null (Ed.)Little is known regarding the use of, and factors related with, interaction-oriented practices. In this study we investigate instructors’ use of interaction-oriented practices in online college courses. We begin by drawing on several strands of literature to offer a person-purpose interaction framework for categorizing interaction-oriented practices. The framework’s six sub-domains integrate for whom students are interacting (instructor, student, content) with the interaction’s pedagogical purpose (academic, social, managerial). Subsequently, we examine factors that predict instructors’ use of these six domains of practices, including instructors’ characteristics and their perceptions of online learning, using a sample of (n = 126) community college instructors teaching online courses. The results show that instructors using more interaction-oriented practices consistently have greater employment status and teaching load, greater self-efficacy for using learning management systems, and greater perceived benefits of online learning for students, with subtle distinctions found across sub-domains. The findings have several implications for future research examining pedagogical behavior, as well as the design of professional development activities aimed at enhancing the use of effective online instructional practices among college instructors.more » « less
-
Classroom orchestration requires teachers to concurrently manage multiple activities across multiple social levels (individual, group, and class) and under various constraints. Real-time dashboards can support teachers; however, designing actionable dashboards is a huge challenge. This paper describes a participatory design study to identify and inform critical features of a dashboard for displaying relevant, actionable, real-time data. We leveraged a Sense-Assess-Act framework to present dashboard mockups to teachers for feedback. Although the participating teachers differed in how they would use the presented information (during class or after class as a post hoc analysis tool), two common emerging themes were that they wanted to use the data to a) better support their students and b) to make broader instructional decisions. We present data from our study and propose a customizable, mobile dashboard, that can be adapted to a teacher's specific needs at a specific time, to help them better facilitate learning activities.more » « less
-
null (Ed.)Abstract Students are more likely to learn in college science, technology, engineering, and math (STEM) classrooms when instructors use teacher discourse moves (TDMs) that encourage student engagement and learning. However, although teaching practices are well studied, TDMs are not well understood in college STEM classrooms. In STEM courses at a minority-serving institution (MSI; n = 74), we used two classroom observation protocols to investigate teaching practices and TDMs across disciplines, instructor types, years of teaching experience, and class size. We found that instructors guide students in active learning activities, but they use authoritative discourse approaches. In addition, chemistry instructors presented more than biology instructors. Also, teaching faculty had relatively high dialogic, interactive discourse, and neither years of faculty teaching experience nor class size had an impact on teaching practices or TDMs. Our results have implications for targeted teaching professional development efforts across instructor and course characteristics to improve STEM education at MSIs.more » « less
An official website of the United States government

