Reinforcement learning (RL) presents numerous benefits compared to rule-based approaches in various applications. Privacy concerns have grown with the widespread use of RL trained with privacy- sensitive data in IoT devices, especially for human-in-the-loop systems. On the one hand, RL methods enhance the user experience by trying to adapt to the highly dynamic nature of humans. On the other hand, trained policies can leak the user’s private information. Recent attention has been drawn to designing privacy-aware RL algorithms while maintaining an acceptable system utility. A central challenge in designing privacy-aware RL, especially for human-in-the-loop systems, is that humans have intrinsic variability, and their preferences and behavior evolve. The effect of one privacy leak mitigation can differ for the same human or across different humans over time. Hence, we can not design one fixed model for privacy-aware RL that fits all. To that end, we propose adaPARL, an adaptive approach for privacy-aware RL, especially for human-in-the-loop IoT systems. adaPARL provides a personalized privacy-utility trade-off depend- ing on human behavior and preference. We validate the proposed adaPARL on two IoT applications, namely (i) Human-in-the-Loop Smart Home and (ii) Human-in-the-Loop Virtual Reality (VR) Smart Classroom. Results obtained on these two applications validate the generality of adaPARL and its ability to provide a personalized privacy-utility trade-off. On average, adaPARL improves the utility by 57% while reducing the privacy leak by 23% on average.
more »
« less
Hammers for Robots: Designing Tools for Reinforcement Learning Agents
In this paper we explore what role humans might play in designing tools for reinforcement learning (RL) agents to interact with the world. Recent work has explored RL methods that optimize a robot’s morphology while learning to control it, effectively dividing an RL agent’s environment into the external world and the agent’s interface with the world. Taking a user-centered design (UCD) approach, we explore the potential of a human, instead of an algorithm, redesigning the agent’s tool. Using UCD to design for a machine learning agent brings up several research questions, including what it means to understand an RL agent’s experience, beliefs, tendencies, and goals. After discussing these questions, we then present a system we developed to study humans designing a 2D racecar for an RL autonomous driver. We conclude with findings and insights from exploratory pilots with twelve users using this system.
more »
« less
- NSF-PAR ID:
- 10292491
- Date Published:
- Journal Name:
- DIS '21: Designing Interactive Systems Conference 2021
- Page Range / eLocation ID:
- 1638 to 1653
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The biodesign studio: Constructions and reflections of high school youth on making with living mediamore » « less
Abstract Most constructionist efforts have focused on supporting learners with tools for designing with digital and/or physical media. Recent developments in life sciences now allow K‐12 learners to design with living materials, or to biodesign. In this paper, we report on the development of a biodesign studio activity called biocakes wherein teams of high school youth genetically modified a yeast strain to bake a nutrient fortified food product. Using a qualitative deductive analysis of classroom observations, interviews and projects, we examined high school youth experiences and reflections of these activities to answer three research questions: (1) What kind of artefacts did youth make with living media? (2) How did youth engage in biodesign? and (3) What did youth have to say about their biodesign experiences? We discuss how our analyses of biodesign applications highlight the importance of assembly practices, social engagement and imaginative design for constructionist learning. These insights provide compelling examples for designing with living media in K‐12 education.
Practitioner notes What is already known about this topic?
Constructionist perspectives shape important ideas about what we know about science learning, and notably in computer science fields.
One widely taken up example includes making, where learners use crafts to make interactive computational objects.
In life science, constructionism also provides insights about learning using digital media to model biological systems or interact with living materials.
What this paper adds?
This paper extends these perspectives by examining production and engagement when learners construct
with living materials—an approach that has only recently been possible with the development of accessible wet lab tools.We frame learning activities as a studio model—that emphasised application design, iteration and critique—to better assess the roles assembly, construction and speculative design play in production that uses living materials.
Our findings suggest that assembly is important for creating accessible points of entry to complex biological fabrication processes.
We also find that speculative design provides an opportunity for learners to extend their existing abilities beyond what is otherwise available given their expertise or access to resources, and thus expansively explore related ideas.
Implications for practice and/or policy
Assembly and speculative design have important places in constructionist‐driven production with living materials and could, therefore, be leveraged in practice.
-
We study adaptive video streaming for multiple users in wireless access edge networks with unreliable channels. The key challenge is to jointly optimize the video bitrate adaptation and resource allocation such that the users' cumulative quality of experience is maximized. This problem is a finite-horizon restless multi-armed multi-action bandit problem and is provably hard to solve. To overcome this challenge, we propose a computationally appealing index policy entitled Quality Index Policy, which is well-defined without the Whittle indexability condition and is provably asymptotically optimal without the global attractor condition. These two conditions are widely needed in the design of most existing index policies, which are difficult to establish in general. Since the wireless access edge network environment is highly dynamic with system parameters unknown and time-varying, we further develop an index-aware reinforcement learning (RL) algorithm dubbed QA-UCB. We show that QA-UCB achieves a sub-linear regret with a low-complexity since it fully exploits the structure of the Quality Index Policy for making decisions. Extensive simulations using real-world traces demonstrate significant gains of proposed policies over conventional approaches. We note that the proposed framework for designing index policy and index-aware RL algorithm is of independent interest and could be useful for other large-scale multi-user problems.more » « less
-
more » « less
Abstract Much attention in constructionism has focused on designing tools and activities that support learners in designing fully finished and functional applications and artefacts to be shared with others. But helping students learn to debug their applications often takes on a surprisingly more instructionist stance by giving them checklists, teaching them strategies or providing them with test programmes. The idea of designing bugs for learning—or
debugging by design —makes learners agents of their own learning and, more importantly, of making and solving mistakes. In this paper, we report on our implementation of ‘Debugging by Design’ activities in a high school classroom over a period of 8 hours as part of an electronic textiles unit. Students were tasked to craft the electronic textile artefacts with problems or bugs for their peers to solve. Drawing on observations and interviews, we answer the following research questions: (1) How did students participate in making bugs for others? (2) What did students gain from designing and solving bugs for others? In the discussion, we address the opportunities and challenges that designing personally and socially meaningful failure artefacts provides for becoming objects‐to‐think‐with and objects‐to‐share‐with in student learning and promoting new directions in constructionism.Practitioner notes What is already known about this topic
There is substantial evidence for the benefits of learning programming and debugging in the context of constructing personally relevant and complex artefacts, including electronic textiles.
Related, work on productive failure has demonstrated that providing learners with strategically difficult problems (in which they ‘fail’) equips them to better handle subsequent challenges.
What this paper adds
In this paper, we argue that designing bugs or ‘failure artefacts’ is as much a constructionist approach to learning as is designing fully functional artefacts.
We consider how ‘failure artefacts’ can be both objects‐to‐learn‐with and objects‐to‐share‐with.
We introduce the concept of ‘Debugging by Design’ (DbD) as a means to expand application of constructionism to the context of developing ‘failure artifacts’.
Implications for practice and/or policy
We conceptualise a new way to enable and empower students in debugging—by designing creative, multimodal buggy projects for others to solve.
The DbD approach may support students in near‐transfer of debugging and the beginning of a more systematic approach to debugging in later projects and should be explored in other domains beyond e‐textiles.
New studies should explore learning, design and teaching that empower students to design bugs in projects in mischievous and creative ways.
-
BACKGROUND: Natureculture (Haraway, 2003; Fuentes, 2010) constructs offer a powerful framework for science education to explore learners’ interactions with and understanding of the natural world. Technologies such as Augmented Reality (AR) designed to reveal pets’ sensory worlds and companionship with pets can facilitate learners’ harmonious relationships with significant others in naturecultures. METHODS: At a two-week virtual summer camp, we engaged teens in inquiring into dogs’ and cats’ senses using selective color filters, investigations, experience design projects, and understanding how the umwelt (von Uexküll, 2001) of pets impacts their lives with humans. We qualitatively analyzed participants’ talk, extensive notes, and projects completed at the workshop. FINDINGS: We found that teens engaged in the science and engineering practices of planning and carrying out investigations, constructing explanations and designing solutions, and questioning while investigating specific aspects of their pets’ lives. Further, we found that teens checking and taking pets’ perspectives while caring for them shaped their productive engagement in these practices. The relationship between pets and humans facilitated an ecological and relational approach to science learning. CONTRIBUTION: Our findings suggest that relational practices of caring and perspective-taking coexist with scientific practices and enrich scientific inquiry.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3461778.3462029
