An official website of the United States government
There is a growing interest in the research and use of automated feedback dashboards that display classroom analytics; yet little is known about the detailed processes instructors use to make sense of these tools, and to determine the impact on their teaching practices. This research was conducted at a public Midwestern university within the context of an automated classroom observation and feedback implementation project. Fifteen engineering instructors engaged in this research. The overarching goal was to investigate instructor teaching beliefs, pedagogical practices, and sensemaking processes regarding dashboard use. A grounded theory approach was used to identify categories related to instructor perceptions. Results revealed that instructor experiences inform both their present use of the dashboard and consequential future actions. A model is presented that illustrates categories included in instructor pre-use, use, and post-use of an automated feedback dashboard. An extension to this model is presented and accompanied by recommendations for a more effective future use of automated dashboards. The model’s practical implications inform both instructors and designers on effective design and use of dashboards, ultimately paving a way to improve pedagogical practices and instruction
more »
« less
From Data to Actions: Unfolding Instructors’ Sense-making and Reflective Practice with Classroom Analytics
The ultimate goal of using learning analytics dashboards is to improve teaching and learning processes. Instructors that use an analytics dashboard are presented with data about their students and/or about their teaching practices. Despite growing research in analytics dashboards, little is known about how instructors make sense of the data they receive and reflect on it. Moreover, there is limited evidence on how instructors who use these dashboards take further actions and improve their pedagogical practices. My dissertation work addresses these issues by examining instructors’ sense making, reflective practice and subsequent actions taken from classroom analytics in three phases: (a) problem analysis from systematic literature review (current), (b) implementation and examination of instructors’ sense-making and reflective practice (current) and (c) human-centered approaches to co-designing instructors’ dashboards with stakeholders (current). The findings will contribute to the conceptual basis of instructors’ change of their pedagogical practices and practical implications of human-centered principles in designing effective dashboards.
more »
« less
- Award ID(s):
- 2021118
- PAR ID:
- 10356601
- Date Published:
- Journal Name:
- Proceedings of 12th International Conference on Learning Analytics and Knowledge (LAK22)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Simulation-based learning has become a cornerstone of healthcare education, fostering essential skills like communication, teamwork or decision-making in safe, controlled environments. However, participants’ reflection on simulations often rely on subjective recollections, limiting their effectiveness in promoting learning. This symposium explores how multimodal analytics and AI can enhance simulation-based education by automating teamwork analysis data, providing structured feedback, and supporting reflective practices. The papers examine real-time analytics for closed-loop communication in cardiac arrest simulations, multimodal data use to refine feedback in ICU nursing simulations, generative AI-powered chatbots facilitating nursing students' interpretation of multimodal learning analytics dashboards, and culturally sensitive, AI-based scenarios for Breaking Bad News in an Indian context. Collectively, these contributions highlight the transformative potential of using data and AI-enhanced solutions, emphasizing personalization, cultural sensitivity, and human-centered design, and invite dialogue on the pedagogical, technological and ethical implications of introducing data-based practices and AI-based tools in medical education.more » « less
-
Although many efforts are being made to provide educators with dashboards and tools to understand student behaviors within specific technological environments (learning analytics), there is a lack of work in supporting educators in making data-informed design decisions when designing a blended course and planning learning activities. In this paper, we introduce concept-level design analytics, a knowledge-based visualization, which uncovers facets of the learning activities that are being authored. The visualization is integrated into a (blended) learning design authoring tool, edCrumble. This new approach is explored in the context of a higher education programming course, where teaching assistants design labs and home practice sessions with online smart learning content on a weekly basis. We performed a within-subjects user study to compare the use of the design tool both with and without the visualization. We studied the differences in terms of cognitive load, design outcomes and user actions within the system to compare both conditions to the objective of evaluating the impact of using design analytics during the decision-making phase of course design.more » « less
-
Abstract Computational analysis methods and machine learning techniques introduce innovative ways to capture classroom interactions and display data on analytics dashboards. Automated classroom analytics employ advanced data analysis, providing educators with comprehensive insights into student participation, engagement, and behavioral trends within classroom settings. Through the provision of context-sensitive feedback, automated classroom analytics systems can be integrated into the evidence-based pedagogical decision-making and reflective practice processes of faculty members in higher education institutions. This paper presents TEACHActive, an automated classroom analytics system, by detailing its design and implementation. It outlines the processes of stakeholder engagement and mapping, elucidates the benefits and obstacles associated with a comprehensive classroom analytics system design, and concludes by discussing significant implications. These implications propose user-centric design approaches for higher education researchers and practitioners to consider.more » « less
-
This complete research paper details an investigation into the influence of instructors' pedagogical knowledge on their classroom practices in the context of teaching first-year engineering courses. Background and Motivation: First-year engineering courses serve as the foundational setting in which students are introduced to the field of engineering as well as the pedagogies specific to engineering teaching and learning. These courses are pivotal in equipping students with essential knowledge and skills, setting the stage for their success in more advanced engineering topics. Understanding how instructors' pedagogical knowledge affects their teaching practices is crucial. Pedagogical knowledge encompasses a wide range of techniques to effectively manage a classroom and engage students. This includes the use of instructional strategies that cater to diverse student needs, the design of impactful and engaging lesson plans, etc. There is, however, limited research on how instructors’ pedagogical knowledge influences their classroom practices in first-year engineering courses. Hence, it seems opportune and essential to conduct additional research on engineering instructors' classroom practices. Research Question: The central question driving this research is: How does instructors' pedagogical knowledge influence their pedagogical practices for first-year engineering courses? Method: For this study, we chose the model of teacher professional knowledge and skill (TPK&S) that includes pedagogical content knowledge (PCK). The model recognizes the fundamental importance of pedagogical knowledge and contextualizes PCK within that framework, encompassing the intricate nature of teaching and learning. A descriptive case study was utilized as a methodology for this work to delve into the phenomenon. The context of the study was a first-year introductory engineering course offered at a large public research institution. This is a pilot study for an NSF-funded project (blinded for review), the study involved two instructors, Chandler and Joey (pseudonyms), chosen through purposive sampling, with varying levels of teaching experience. Data collection involved direct classroom observation using the Teaching Dimensions Observation Protocol (TDOP) and semi-structured interviews conducted after the observations. The interviews were conducted after classroom observations, allowing the researcher to explore specific findings from the observations. Results: Thematic analysis was used to categorize the data based on the constructs of the theoretical framework. The analysis revealed three major themes: (a) Instructors' topic-specific professional knowledge significantly influences their pedagogical practices. Both instructors adapt their teaching methods based on their understanding of course material and students' difficulties. (b) The interaction between instructors' personal pedagogical content knowledge (PCK) and the classroom context shapes their classroom practices. (c) Instructors' beliefs and prior knowledge act as amplifiers or filters based on the situation. They filter out their teaching practices that do not align with their beliefs and prior knowledge. Conclusion: The findings presented in this paper provide valuable insights into the complex interplay between instructors' pedagogical knowledge and their classroom practices. This work holds significant implications for current and future first-year instructors in that this paper will showcase how instructors use their understanding of the content and their students to teach, which is a critical aspect of helping students successfully integrate into engineering.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
