skip to main content

Explore Research Products in the NSF-PAR

Title: Designing a Dashboard for Student Teamwork Analysis
Classroom dashboards are designed to help instructors effectively orchestrate classrooms by providing summary statistics, activity tracking, and other information [12]. Existing dashboards are generally specific to an LMS or platform and they generally summarize individual work, not group behaviors. However, CS courses typically involve constellations of tools and mix on- and offline collaboration. Thus, cross-platform monitoring of individuals and teams is important to develop a full picture of the class. In this work, we describe our work on Concert, a data integration platform that collects data about student activities from several sources such as Piazza, My Digital Hand, and GitHub and uses it to support classroom monitoring through analysis and visualizations. We discuss team visualizations that we have developed to support effective group management and to help instructors identify teams in need of intervention.  more » « less
Award ID(s):
1821475
NSF-PAR ID:
10392592
Author(s) / Creator(s):
Editor(s):
Merkle, Larry; Doyle, Maureen; Sheard, Judithe; Soh, Leen-Kiat; Dorn, Brian
Date Published:
Journal Name:
Proceedings of the 53rd ACM Technical Symposium on Computer Science Education
Page Range / eLocation ID:
446-452
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ( , Frontiers in education)
    Michalsky, Tova ; Moos, Daniel (Ed.)
    Teachers’ ability to self-regulate their own learning is closely related to their competency to enhance self-regulated learning (SRL) in their students. Accordingly, there is emerging research for the design of teacher dashboards that empower instructors by providing access to quantifiable evidence of student performance and SRL processes. Typically, they capture evidence of student learning and performance to be visualized through activity traces (e.g., bar charts showing correct and incorrect response rates, etc.) and SRL data (e.g., eye-tracking on content, log files capturing feature selection, etc.) in order to provide teachers with monitoring and instructional tools. Critics of the current research on dashboards used in conjunction with advanced learning technologies (ALTs) such as simulations, intelligent tutoring systems, and serious games, argue that the state of the field is immature and has 1) focused only on exploratory or proof-of-concept projects, 2) investigated data visualizations of performance metrics or simplistic learning behaviors, and 3) neglected most theoretical aspects of SRL including teachers’ general lack of understanding their’s students’ SRL. Additionally, the work is mostly anecdotal, lacks methodological rigor, and does not collect critical process data (e.g. frequency, duration, timing, or fluctuations of cognitive, affective, metacognitive, and motivational (CAMM) SRL processes) during learning with ALTs used in the classroom. No known research in the areas of learning analytics, teacher dashboards, or teachers’ perceptions of students’ SRL and CAMM engagement has systematically and simultaneously examined the deployment, temporal unfolding, regulation, and impact of all these key processes during complex learning. In this manuscript, we 1) review the current state of ALTs designed using SRL theoretical frameworks and the current state of teacher dashboard design and research, 2) report the important design features and elements within intelligent dashboards that provide teachers with real-time data visualizations of their students’ SRL processes and engagement while using ALTs in classrooms, as revealed from the analysis of surveys and focus groups with teachers, and 3) propose a conceptual system design for integrating reinforcement learning into a teacher dashboard to help guide the utilization of multimodal data collected on students’ and teachers’ CAMM SRL processes during complex learning. 
    more » « less
  2. ; ; ; ; ; ( , ASEE annual conference & exposition proceedings)
    We describe and analyze our efforts to support Learning Assistants (LAs)—undergraduate peer educators who simultaneously take a 3-credit pedagogy course—in fostering equitable team dynamics and collaboration within a project-based engineering design course. Tonso and others have shown that (a) inequities can “live” in mundane interactions such as those among students within design teams and (b) those inequities both reflect and (re)produce broader cultural patterns and narratives (e.g. Wolfe & Powell, 2009; Tonso, 1996, 2006a, 2006b; McLoughlin, 2005). LAs could be well-positioned to notice and potentially disrupt inequitable patterns of participation within design teams. In this paper, we explore (1) How do LAs notice, diagnose, and consider responding to teamwork troubles within design teams, and (2) What ideological assumptions plausibly contribute to LAs’ sensemaking around their students’ teamwork troubles? To do so, we analyze how the LAs notice and consider responding to issues of equitable teamwork and participation, as exhibited in three related activities: (i) an in-class roleplay, (ii) observing and diagnosing teamwork troubles (TTs) in the engineering design teams, and (iii) imagining possible instructional responses to those troubles, and students’ possible reactions. We articulate three modes of thinking that roughly capture patterns in LAs’ descriptions and diagnoses of, and imagined responses to, the teamwork troubles: individual accountability, where the trouble is seen as caused by individual(s) described as “off task” or “checked out” or demonstrating some level of incompetence; delegation of work, where the trouble was located in the team leader’s inability to delegate tasks effectively to team members, or in the group’s general lack of communication about what tasks need to be completed, who should execute the tasks, and what work other groups in the team were doing; and emergent systems, where trouble was described as a group-level phenomenon emerging from the patterns of interaction amongst group members, contextual features, and larger structural forces. We find that LAs drew on individual accountability and delegation of work to evaluate TTs. Much rarer were ascriptions of TTs to interactional dynamics between teammates. We connected these modes to the underlying ideological assumptions that have consequences for how meritocracy and technocracy (Slaton, 2015; Cech, 2014) play out in an engineering design classroom and serve to ameliorate or reify engineering mindsets (Riley, 2008). The modes are asymmetric, in that emergent systems based interpretations hold more potential for elucidating ongoing social processes, for challenging meritocracy and socio-technical duality, and for seeing power differentials within interpersonal and institutional contexts. We argue for the need to better understand the ideological assumptions underlying how peer-educators—and other instructors—interpret classroom events. 
    more » « less
  3. ; ; ; ( , Teachers College Record: The Voice of Scholarship in Education)
    Background/Context: After-school programs that focus on integrating computer programming and mathematics in authentic environments are seldomly accessible to students from culturally and linguistically diverse backgrounds, particularly bilingual Latina students in rural contexts. Providing a context that broadens Latina students’ participation in mathematics and computer programming requires educators to carefully examine how verbal and nonverbal language is used to interact and to position students as they learn new concepts in middle school. This is also an important stage for adolescents because they are likely to make decisions about their future careers in STEM. Having access to discourse and teaching practices that invite students to participate in mathematics and computer programming affords them opportunities to engage with these fields. Purpose/Focus of Study: This case study analyzes how small-group interactions mediated the positionings of Cindy, a bilingual Latina, as she learned binary numbers in an after-school program that integrated computer programming and mathematics (CPM). Setting: The Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) program was held in a rural bilingual (Spanish and English) middle school in the Southwest. The after-school program was designed to provide experiences for primarily Latinx students to learn how to integrate mathematics with computer programming using Raspberry Pi and Python as a platform. Our case study explores how Cindy was positioned as she interacted with two undergraduate engineering students who served as facilitators while learning binary numbers with a group of three middle school students. Research Design: This single intrinsic case focused on exploring how small-group interactions among four students mediated Cindy’s positionings as she learned binary numbers through her participation in AOLME. Data sources included twelve 90-minute video sessions and Cindy’s journal and curriculum binder. Video logs were created, and transcripts were coded to describe verbal and nonverbal interactions among the facilitators and Cindy. Analysis of select episodes was conducted using systemic functional linguistics (SFL), specifically language modality, to identify how positioning took place. These episodes and positioning analysis describe how Cindy, with others, navigated the process of learning binary numbers under the stereotype that female students are not as good at mathematics as male students. Findings: From our analysis, three themes that emerged from the data portray Cindy’s experiences learning binary numbers. The major themes are: (1) Cindy’s struggle to reveal her understanding of binary numbers in a competitive context, (2) Cindy’s use of “fake it until you make it” to hide her cognitive dissonance, and (3) the use of Spanish and peers’ support to resolve Cindy’s understanding of binary numbers. The positioning patterns observed help us learn how, when Cindy’s bilingualism was viewed and promoted as an asset, this social context worked as a generative axis that addressed the challenges of learning binary numbers. The contrasting episodes highlight the facilitators’ productive teaching strategies and relations that nurtured Cindy’s social and intellectual participation in CPM. Conclusions/Recommendations: Cindy’s case demonstrates how the facilitator’s teaching, and participants’ interactions and discourse practices contributed to her qualitatively different positionings while she learned binary numbers, and how she persevered in this process. Analysis of communication acts supported our understanding of how Cindy’s positionings underpinned the discourse; how the facilitators’ and students’ discourse formed, shaped, or shifted Cindy’s positioning; and how discourse was larger than gender storylines that went beyond classroom interactions. Cindy’s case reveals the danger of placing students in “struggle” instead of a “productive struggle.” The findings illustrated that when Cindy was placed in struggle when confronting responding moves by the facilitator, her “safe” reaction was hiding and avoiding. In contrast, we also learned about the importance of empathetic, nurturing supporting responses that encourage students’ productive struggle to do better. We invite instructors to notice students’ hiding or avoiding and consider Cindy’s case. Furthermore, we recommend that teachers notice their choice of language because this is important in terms of positioning students. We also highlight Cindy’s agency as she chose to take up her friend’s suggestion to “fake it” rather than give up. 
    more » « less
  4. ; ; ( , ICOTS)
    It is widely accepted that active learning and group work generally enhance learning in the statistics classroom, but how should those groups be formed? This study aims to better understand the characteristics of a productive team in the undergraduate introductory statistics course. Specifically, we explore the relationship between the attitudes of a student’s teammates and that student’s academic performance in both individual and group settings. We find moderate evidence that positive teammate attitudes towards statistics are associated with greater improvement from a student’s individual to the team exam score. If we can better understand what combination of student characteristics results in productive teams, instructors can be intentional with how they form groups in the classroom, realizing the full efficacy of active learning. 
    more » « less
  5. ; ; ; ; ; ( , International Conference on Artificial Intelligence in Education)
    Classroom orchestration requires teachers to concurrently manage multiple activities across multiple social levels (individual, group, and class) and under various constraints. Real-time dashboards can support teachers; however, designing actionable dashboards is a huge challenge. This paper describes a participatory design study to identify and inform critical features of a dashboard for displaying relevant, actionable, real-time data. We leveraged a Sense-Assess-Act framework to present dashboard mockups to teachers for feedback. Although the participating teachers differed in how they would use the presented information (during class or after class as a post hoc analysis tool), two common emerging themes were that they wanted to use the data to a) better support their students and b) to make broader instructional decisions. We present data from our study and propose a customizable, mobile dashboard, that can be adapted to a teacher's specific needs at a specific time, to help them better facilitate learning activities. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email