How can we teach AI students to use human concerns to guide their technical decisions? We created an AI assignment with a human context, asking students to find the safest path rather than the shortest path. This integrated assignment evaluated 120 students’ understanding of the limitations and assumptions of standard graph search algorithms, and required students to consider human impacts to propose appropriate modifications. Since the assignment focused on algorithm selection and modification, it provided the instructor with a different perspective on student understanding (compared with questions on algorithm execution). Specifically, many students: tried to solve a bottleneck problem with algorithms designed for accumulation problems, did not distinguish between calculations that could be done during the incremental construction of a path versus ones that required knowledge of the full path, and, when proposing modifications to a standard algorithm, did not present the full technical details necessary to implement their ideas. We created rubrics to analyze students’ responses. Our rubrics cover three dimensions: technical AI knowledge, consideration of human factors, and the integration of technical decisions as they align with the human context. These rubrics demonstrate how students’ skills can vary along each dimension, and also provide a template for scoring integrated assignments for other CS topics. Overall, this work demonstrates how to integrate human concerns with technical content in a way that deepens technical rigor and supports instructor pedagogical content knowledge.
more »
« less
Model AI Assignments 2020
The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of nine AI assignments from the 2020 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu.
more »
« less
- Award ID(s):
- 1724392
- NSF-PAR ID:
- 10179966
- Author(s) / Creator(s):
- ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more »
- Date Published:
- Journal Name:
- Symposium on Educational Advances in Artificial Intelligence (EAAI)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
With large enrollments (about 200-350) of primarily non-majors, engaging students in the required introductory materials science and engineering course at our university has been a longstanding challenge. In moving to the virtual format in the fall of 2020, we significantly adapted several aspects of the course, many of which have continued to the hybrid format in future semesters, with good results. The primary content was provided through asynchronous videos; this format allowed us to break content into digestible pieces. In particular, multiple mini-lectures and example videos were pre-recorded for each week, with a total viewing time per week somewhat less than the typical total class time. To provide real-time, structured interaction, one live virtual class session per week was held, centered on previously submitted student questions. Smaller teaching-assistant-led recitation sections also met live (virtually or in person), during which “clicker” questions were asked through TopHat. Assignments were also updated to take advantage of the virtual format. Multiple small assignments with lower stakes were due throughout the week: a reading/lecture quiz, a survey to submit questions, and a shortened homework assignment. Finally, we changed some content near the end of the course to allow students to connect the course to their own career aspirations, which we expect can aid in longterm retention. Specifically, students chose among several possible topics to cover in the final weeks, covered via typical pre-recorded lectures and reading, and also guest lectures. They wrote an abstract-length reflection on how they could use what they learned in this course later in their careers. Overall, students remained engaged with the course throughout the semester and provided favorable comments and evaluations of the course, including higher numerical evaluations of the course than in prior semesters.more » « less
-
Gale and Shapley’s stable assignment problem has been extensively studied, applied, and extended. In the context of school choice, mechanisms often aim at finding an assignment that is more favorable to students. We investigate two extensions introduced in this framework—legal assignments and the efficiency adjusted deferred acceptance mechanism (EADAM) algorithm—through the lens of the classic theory of stable matchings. In any instance, the set [Formula: see text] of legal assignments is known to contain all stable assignments. We prove that [Formula: see text] is exactly the set of stable assignments in another instance. Moreover, we show that essentially all optimization problems over [Formula: see text] can be solved within the same time bound needed for solving it over the set of stable assignments. A key tool for this latter result is an algorithm that finds the student-optimal legal assignment. We then generalize our algorithm to obtain the assignment output of EADAM with any given set of consenting students without sacrificing the running time, hence largely improving in both theory and practice over known algorithms. Finally, we show that the set [Formula: see text] can be much larger than the set of stable matchings, connecting legal matchings with certain concepts and open problems in the literature.more » « less
-
more » « less
Abstract Background A robotic rehabilitation gym can be defined as multiple patients training with multiple robots or passive sensorized devices in a group setting. Recent work with such gyms has shown positive rehabilitation outcomes; furthermore, such gyms allow a single therapist to supervise more than one patient, increasing cost-effectiveness. To allow more effective multipatient supervision in future robotic rehabilitation gyms, we propose an automated system that could dynamically assign patients to different robots within a session in order to optimize rehabilitation outcome.
Methods As a first step toward implementing a practical patient-robot assignment system, we present a simplified mathematical model of a robotic rehabilitation gym. Mixed-integer nonlinear programming algorithms are used to find effective assignment and training solutions for multiple evaluation scenarios involving different numbers of patients and robots (5 patients and 5 robots, 6 patients and 5 robots, 5 patients and 7 robots), different training durations (7 or 12 time steps) and different complexity levels (whether different patients have different skill acquisition curves, whether robots have exit times associated with them). In all cases, the goal is to maximize total skill gain across all patients and skills within a session.
Results Analyses of variance across different scenarios show that disjunctive and time-indexed optimization models significantly outperform two baseline schedules: staying on one robot throughout a session and switching robots halfway through a session. The disjunctive model results in higher skill gain than the time-indexed model in the given scenarios, and the optimization duration increases as the number of patients, robots and time steps increases. Additionally, we discuss how different model simplifications (e.g., perfectly known and predictable patient skill level) could be addressed in the future and how such software may eventually be used in practice.
Conclusions Though it involves unrealistically simple scenarios, our study shows that intelligently moving patients between different rehabilitation robots can improve overall skill acquisition in a multi-patient multi-robot environment. While robotic rehabilitation gyms are not yet commonplace in clinical practice, prototypes of them already exist, and our study presents a way to use intelligent decision support to potentially enable more efficient delivery of technologically aided rehabilitation.
-
The first step toward investigating the effectiveness of a treatment via a randomized trial is to split the population into control and treatment groups then compare the average response of the treatment group receiving the treatment to the control group receiving the placebo. To ensure that the difference between the two groups is caused only by the treatment, it is crucial that the control and the treatment groups have similar statistics. Indeed, the validity and reliability of a trial are determined by the similarity of two groups’ statistics. Covariate balancing methods increase the similarity between the distributions of the two groups’ covariates. However, often in practice, there are not enough samples to accurately estimate the groups’ covariate distributions. In this article, we empirically show that covariate balancing with the standardized means difference (SMD) covariate balancing measure, as well as Pocock and Simon’s sequential treatment assignment method, are susceptible to worst case treatment assignments. Worst case treatment assignments are those admitted by the covariate balance measure, but result in highest possible ATE estimation errors. We developed an adversarial attack to find adversarial treatment assignment for any given trial. Then, we provide an index to measure how close the given trial is to the worst case. To this end, we provide an optimization-based algorithm, namely adversarial treatment assignment in treatment effect trials (ATASTREET), to find the adversarial treatment assignments.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1609/aaai.v34i09.7072
