An official website of the United States government
The Next Generation Science Standards and the National Research Council recognize systems thinking as an essential skill to address the global challenges of the 21st century. But the habits of mind needed to understand complex systems are not readily learned through traditional approaches. Recently large-scale interactive multi-user immersive simulations are being used to expose the learners to diverse topics that emulate real-world complex systems phenomena. These modern-day mixed reality simulations are unique in that the learners are an integral part of the evolving dynamics. The decisions they make and the actions that follow, collectively impact the simulated complex system, much like any real-world complex system. But the learners have difficulty understanding these coupled complex systems processes, and often get “lost” or “stuck,” and need help navigating the problem space. Formative feedback is the traditional way educators support learners during problem solving. Traditional goal-based and learner-centered approaches don’t scale well to environments that allow learners to explore multiple goals or solutions, and multiple solution paths (Mallavarapu & Lyons, 2020). In this work, we reconceptualize formative feedback for complex systems-based learning environments, formative fugues, (a term derived from music by Reitman, 1964) to allow learners to make informed decisions about their own exploration paths. We discuss a novel computational approach that employs causal inference and pattern matching to characterize the exploration paths of prior learners and generate situationally relevant formative feedback. We extract formative fugues from the data collected from an ecological complex systems simulation installed at a museum. The extracted feedback does not presume the goals of the learners, but helps the learners understand what choices and events led to the current state of the problem space, and what paths forward are possible. We conclude with a discussion of implications of using formative fugues for complex systems education.
more »
« less
Exploration Maps, Beyond Top Scores: Designing Formative Feedback for Open-Ended Problems
Learners are being exposed to abstract skills like innovation, creativity and reasoning through collaborative open-ended problems. Most of these problems, like their real-world counterparts, have no definite starting or ending point, and have no fixed strategies to solve them. To help the learners explore the multiple perspectives of the problem solutions there is an urgent need for designing formative feedback in these environments. Unfortunately, there are barriers to us- ing existing EDM approaches to provide formative feedback to learners in these environments: (1) due to the vast so- lution space, and the lack of verifiability of the solutions it is impossible to create task and expert models, thus mak- ing the detection of the learners progress impractical; (2) formative feedback based on individual learner models does not scale well when many learners are collaborating to solve the same problem. In this work, we redefine formative feed- back as reshaping the learning environment and learners’ exploration paths by exposing/enlisting “fugues” as defined by Reitman [28]. Through a case study approach we, (1) val- idate methods to extract learners’ “fugues” from a collabora- tive open-ended museum exhibit, (2) design formative feed- back for learners and educators using these extracted fugues in real-time, (3) evaluate the impact of exposing fugues to group of learners interacting with the exhibit.
more »
« less
- Award ID(s):
- 1822864
- PAR ID:
- 10290783
- Editor(s):
- Anna N. Rafferty, Jacob Whitehill
- Date Published:
- Journal Name:
- Proceedings of The 13th International Conference on Educational Data Mining
- Page Range / eLocation ID:
- 790 - 795
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In schools and colleges around the world, open-ended home-work assignments are commonly used. However, such assignments require substantial instructor effort for grading, and tend not to support opportunities for repeated practice. We propose UpGrade, a novel learnersourcing approach that generates scalable learning opportunities using prior student solutions to open-ended problems. UpGrade creates interactive questions that offer automated and real-time feedback, while enabling repeated practice. In a two-week experiment in a college-level HCI course, students answering UpGrade-created questions instead of traditional open-ended assignments achieved indistinguishable learning outcomes in ~30% less time. Further, no manual grading effort is required. To enhance quality control, UpGrade incorporates a psychometric approach using crowd workers' answers to automatically prune out low quality questions, resulting in a question bank that exceeds reliability standards for classroom use.more » « less
-
null (Ed.)Open-ended programming increases students' motivation by allowing them to solve authentic problems and connect programming to their own interests. However, such open-ended projects are also challenging, as they often encourage students to explore new programming features and attempt tasks that they have not learned before. Code examples are effective learning materials for students and are well-suited to supporting open-ended programming. However, there is little work to understand how novices learn with examples during open-ended programming, and few real-world deployments of such tools. In this paper, we explore novices' learning barriers when interacting with code examples during open-ended programming. We deployed Example Helper, a tool that offers galleries of code examples to search and use, with 44 novice students in an introductory programming classroom, working on an open-ended project in Snap. We found three high-level barriers that novices encountered when using examples: decision, search, and integration barriers. We discuss how these barriers arise and design opportunities to address them.more » « less
-
null; null; null (Ed.)We examined the regulation of shared problem solving in a museum exhibit. We found that we had to augment our dialogue codes to properly embrace the dynamic nature of the observed learning regulation. These changes reflect aspects of shared regulation that occur when learning takes place (1) in an immersive open-ended learning environment, where (2) learners work together in large groups. We present preliminary results, arguing that designers and researchers may benefit from recognizing how planning and evaluation acts can be tactically embedded in immersive learning environments.more » « less
-
Introduction: The work reported here subscribes to the idea that the best way to learn - and thus, improve student educational outcomes - is through solving problems, yet recognizes that engineering students are generally provided insufficient opportunities to engage problems as they will be engaged in practice. Attempts to incorporate more open-ended, ill-structured experiences have increased but are challenging for faculty to implement because there are no systematic methods or approaches that support the educator in designing these learning experiences. Instead, faculty often start from the anchor of domain-specific concepts, an anchoring that is further reinforced by available textbook problems that are rarely open in nature. Open-ended problems are then created in ad-hoc ways, and in doing so, the problem-solving experience is often not realized as the instructor intended. Approach: The focus in this work is the development and preliminary implementation of a reflective approach to support instructors in examining the design intent of problem experiences. The reflective method combines concept mapping as developed by Joseph Novak with the work of David Jonassen and his characterization of problems and the forms of knowledge required to solve them. Results: We report on the development of a standard approach – a template -- for concept mapping of problems. As a demonstration, we applied the approach to a relatively simple, well-structured problem used in an introductory aerospace engineering course. Educator-created concept maps provided a visual medium for examining the connectivity of problem elements and forms of knowledge. Educator reflection after looking at and discussing the concept map revealed ways in which the problem engagement may differ from the perceived design intent. Implications: We consider the potential for the proposed method to support design and facilitation activities in problem-based learning (PBL) environments. We explore broader implications of the approach as it relates to 1) facilitating a priori faculty insights regarding student navigation of problem solving, 2) instructor reflection on problem design and facilitation, and 3) supporting problem design and facilitation. Additionally, we highlight important issues to be further investigated toward quantifying the value and limitations of the proposed approach.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
