More Like this

1. Leveraging cultural forms in human‐centred learning analytics design

   https://doi.org/10.1111/bjet.13384  

   Campos, Fabio ; Nguyen, Ha ; Ahn, June ; Jackson, Kara ( September 2023 , British Journal of Educational Technology)

   In this article, we offer theory‐grounded narratives of a 4‐year participatory design process of a Learning Analytics tool with K‐12 educators. We describe how wedesign‐in‐partnershipby leveraging educators' routines, values and cultural representations into the designs of digital dashboards. We make our long‐term reasoning visible by reflecting upon how design decisions were made, discussing key tensions and analysing to what extent the developed tools were taken up in practice. Through thick design narratives, we reflect upon how cultural forms—recognizable cultural constructs that might cue and facilitate specific activities—were identified among educators and informed the design of a dashboard. We then examined the extent to which the designed tool supported coaches and teachers to engage in Generative Uncertainty, an interpretive stance in which educators manifest productive inquiries towards data. Our analysis highlights that attuning to cultural forms is a valuable first step but not enough towards designing LA tools for systems in ways that fit institutionalized practices, challenge instrumental uses and spur productive inquiry. We conclude by offering two key criteria for making culturally‐grounded design decisions in the context of long‐term partnerships.

   What is already known about this topic

   Participatory design can invite stakeholders to directly inform the creation of LA artefacts that fit their needs, context and cultural markers.

   What this paper adds

   Cultural forms can be identified and leveraged in the design of LA tools.

   HCLA scholars ought to design forsystems—the complex body of organizational routines, cultural practices and interactions among multiple stakeholders—and not just forusers.

   Implications for practice and/or policy

   Leveraging cultural forms in LA needs to be accompanied by a critical view of which practices, behaviours, values and structures are suggested by such forms.

   Designing features that are easy to use, are associated with concrete tasks, and fit into existing cultural practices are three criteria for embedding cultural forms into LA design.

    

   more » « less
2. Co‐designing teacher support technology for problem‐based learning in middle school science

   https://doi.org/10.1111/bjet.13363  

   Hutchins, Nicole M. ; Biswas, Gautam ( July 2023 , British Journal of Educational Technology)

   This paper provides an experience report on a co‐design approach with teachers to co‐create learning analytics‐based technology to support problem‐based learning in middle school science classrooms. We have mapped out a workflow for such applications and developed design narratives to investigate the implementation, modifications and temporal roles of the participants in the design process. Our results provide precedent knowledge on co‐designing with experienced and novice teachers and co‐constructing actionable insight that can help teachers engage more effectively with their students' learning and problem‐solving processes during classroom PBL implementations.

   What is already known about this topic

   Success of educational technology depends in large part on the technology's alignment with teachers' goals for their students, teaching strategies and classroom context.

   Teacher and researcher co‐design of educational technology and supporting curricula has proven to be an effective way for integrating teacher insight and supporting their implementation needs.

   Co‐designing learning analytics and support technologies with teachers is difficult due to differences in design and development goals, workplace norms, and AI‐literacy and learning analytics background of teachers.

   What this paper adds

   We provide a co‐design workflow for middle school teachers that centres on co‐designing and developing actionable insights to support problem‐based learning (PBL) by systematic development of responsive teaching practices using AI‐generated learning analytics.

   We adapt established human‐computer interaction (HCI) methods to tackle the complex task of classroom PBL implementation, working with experienced and novice teachers to create a learning analytics dashboard for a PBL curriculum.

   We demonstrate researcher and teacher roles and needs in ensuring co‐design collaboration and the co‐construction of actionable insight to support middle school PBL.

   Implications for practice and/or policy

   Learning analytics researchers will be able to use the workflow as a tool to support their PBL co‐design processes.

   Learning analytics researchers will be able to apply adapted HCI methods for effective co‐design processes.

   Co‐design teams will be able to pre‐emptively prepare for the difficulties and needs of teachers when integrating middle school teacher feedback during the co‐design process in support of PBL technologies.

    

   more » « less
3. A learning analytics approach towards understanding collaborative inquiry in a problem‐based learning environment

   https://doi.org/10.1111/bjet.13198  

   Saleh, Asmalina ; Phillips, Tanner M. ; Hmelo‐Silver, Cindy E. ; Glazewski, Krista D. ; Mott, Bradford W. ; Lester, James C. ( February 2022 , British Journal of Educational Technology)

   This exploratory paper highlights how problem‐based learning (PBL) provided the pedagogical framework used to design and interpret learning analytics from Crystal Island: EcoJourneys, a collaborative game‐based learning environment centred on supporting science inquiry. In Crystal Island: EcoJourneys, students work in teams of four, investigate the problem individually and then utilize a brainstorming board, an in‐game PBL whiteboard that structured the collaborative inquiry process. The paper addresses a central question: how can PBL support the interpretation of the observed patterns in individual actions and collaborative interactions in the collaborative game‐based learning environment? Drawing on a mixed method approach, we first analyzed students' pre‐ and post‐test results to determine if there were learning gains. We then used principal component analysis (PCA) to describe the patterns in game interaction data and clustered students based on the PCA. Based on the pre‐ and post‐test results and PCA clusters, we used interaction analysis to understand how collaborative interactions unfolded across selected groups. Results showed that students learned the targeted content after engaging with the game‐based learning environment. Clusters based on the PCA revealed four main ways of engaging in the game‐based learning environment: students engaged in low to moderate self‐directed actions with (1) high and (2) moderate collaborative sense‐making actions, (3) low self‐directed with low collaborative sense‐making actions and (4) high self‐directed actions with low collaborative sense‐making actions. Qualitative interaction analysis revealed that a key difference among four groups in each cluster was the nature of verbal student discourse: students in the low to moderate self‐directed and high collaborative sense‐making cluster actively initiated discussions and integrated information they learned to the problem, whereas students in the other clusters required more support. These findings have implications for designing adaptive support that responds to students' interactions with in‐game activities.

   What is already known about this topic

   Learning analytic methods have been effective for understanding student learning interactions for the purposes of assessment, profiling student behaviour and the effectiveness of interventions.

   However, the interpretation of analytics from these diverse data sets are not always grounded in theory and challenges of interpreting student data are further compounded in collaborative inquiry settings, where students work in groups to solve a problem.

   What this paper adds

   Problem‐based learning as a pedagogical framework allowed for the design to focus on individual and collaborative actions in a game‐based learning environment and, in turn, informed the interpretation of game‐based analytics as it relates to student's self‐directed learning in their individual investigations and collaborative inquiry discussions.

   The combination of principal component analysis and qualitative interaction analysis was critical in understanding the nuances of student collaborative inquiry.

   Implications for practice and/or policy

   Self‐directed actions in individual investigations are critical steps to collaborative inquiry. However, students may need to be encouraged to engage in these actions.

   Clustering student data can inform which scaffolds can be delivered to support both self‐directed learning and collaborative inquiry interactions.

   All students can engage in knowledge‐integration discourse, but some students may need more direct support from teachers to achieve this.

    

   more » « less
4. How teachers conceptualise shared control with an AI co‐orchestration tool: A multiyear teacher‐centred design process

   https://doi.org/10.1111/bjet.13372  

   Lawrence, LuEttaMae ; Echeverria, Vanessa ; Yang, Kexin ; Aleven, Vincent ; Rummel, Nikol ( August 2023 , British Journal of Educational Technology)

   Artificial intelligence (AI) can enhance teachers' capabilities by sharing control over different parts of learning activities. This is especially true for complex learning activities, such as dynamic learning transitions where students move between individual and collaborative learning in un‐planned ways, as the need arises. Yet, few initiatives have emerged considering how shared responsibility between teachers and AI can support learning and how teachers' voices might be included to inform design decisions. The goal of our article is twofold. First, we describe a secondary analysis of our co‐design process comprising six design methods to understand how teachers conceptualise sharing control with an AI co‐orchestration tool, calledPair‐Up. We worked with 76 middle school math teachers, each taking part in one to three methods, to create a co‐orchestration tool that supports dynamic combinations of individual and collaborative learning using two AI‐based tutoring systems. We leveraged qualitative content analysis to examine teachers' views about sharing control withPair‐Up, and we describe high‐level insights about the human‐AI interaction, including control, trust, responsibility, efficiency, and accuracy. Secondly, we use our results as an example showcasing how human‐centred learning analytics can be applied to the design of human‐AI technologies and share reflections for human‐AI technology designers regarding the methods that might be fruitful to elicit teacher feedback and ideas. Our findings illustrate the design of a novel co‐orchestration tool to facilitate the transitions between individual and collaborative learning and highlight considerations and reflections for designers of similar systems.

   What is already known about this topic:

   Artificial Intelligence (AI) can help teachers facilitate complex classroom activities, such as having students move between individual and collaborative learning in unplanned ways.

   Designers should use human‐centred design approaches to give teachers a voice in deciding what AI might do in the classroom and if or how they want to share control with it.

   What this paper adds:

   Presents teacher views about how they want to share control with AI to support students moving between individual and collaborative learning.

   Describes how we adapted six design methods to design AI features.

   Illustrates a complete, iterative process to create human‐AI interactions to support teachers as they facilitate students moving from individual to collaborative learning.

   Implications for practice:

   We share five implications for designers that teachers highlighted as necessary when designing AI‐features, including control, trust, responsibility, efficiency and accuracy.

   Our work also includes a reflection on our design process and implications for future design processes.

    

   more » « less
5. Supporting collaborative classroom networks through technology: An actor network theory approach to understanding social behaviours and design

   https://doi.org/10.1111/bjet.13274  

   Kumar, Vishesh ; Tissenbaum, Mike ( September 2022 , British Journal of Educational Technology)

   This paper presents an implementation of Connected Spaces (CxS)—an ambient help seeking interface designed and developed for a project‐based computing classroom. We use actor network theory (ANT) to provide an underutilized posthumanist lens to understand the creation of collaborative connections in this Computational Action‐based implementation. Posthumanism offers an emerging and critical extension to sociocultural perspectives on understanding learning, by pushing us to decenter the human, and consider the active roles that human and non‐human entities play in learning environments by actively shaping each other. We analyse how students in this class adjusted their help‐seeking and collaborative habits following the introduction of CxS, a tool designed to foster (more inter‐group) collaboration. ANT proposes generalized symmetry—a principle of considering human, non‐human and more than human entities with equivalent and comparable agency, leading to describing phenomena as networks of actors in different evolving relationships with each other. Analysing collaborative interactions as fostered by CxS using an ANT approach supports design‐based research—an iterative design revision process highlighting understandings about design as well as learning—by providing a temporal and informative lens into the relationship between actors and tools within the environment. Our key findings include a framing of technologies in classrooms as bridgingagentic gapsbetween students and becoming actors engaging in different behaviours; learners enacting new agencies through technologies (for instance a more comfortable non‐intrusive help seeker), and the need for voicing and teachers to connect help networks in CxS equipped classrooms.

   What is already known about this topic

   Collaborative learning is a valuable skill and practice; opportunities to mentor others are critical in empowering minoritized learners, especially in STEM and computing disciplines.

   School norms solidify a power and expertise hierarchy between teachers and learners and fail to productively support learners in learning from each other.

   Additionally, lack of awareness about peers' knowledge is a common hindrance in students knowing who to ask for help and how.

   What this paper adds

   An example of a designed interface called Connected Spaces with potential to foster more inter‐student collaboration, especially outside of mandated within‐group collaboration—in the form of cross‐group help seeking and help giving.

   A design based research study using actor network theory highlighting the limitations of Connected Spaces in sparking notable behaviour change among students by itself but being retooled as a teacher support tool in enabling cross‐group collaborations.

   Presenting conceptions of collaboration through technologies as bridging agentic gaps and acting with new agencies in performing help‐seeking related actions.

   Provoking the idea of testing emerging technologies in classrooms along with sharing our analyses and reflections with the classroom as a key idea in computing education—surfacing the gap between designed intentions and the different kinds of extra social work needed in the on‐ground success of different technologies.

   Implications for practice and/or policy

   Designers and researchers should create and test more interfaces alongside teachers across different classrooms and contexts aimed at supporting different kinds of voluntary collaborative interactions.

   Curricula, standards and school practices should further center providing students with opportunities to engage as mentors and build communities of learning across disciplines to empower minoritized students.

   Researchers engaging in design based research should consider using more posthumanist lenses to examine educational technologies and how they affect change in learning environments.

    

   more » « less