This paper introduces a novel multi-armed bandits framework, termed Contextual Restless Bandits (CRB), for complex online decision-making. This CRB framework incorporates the core features of contextual bandits and restless bandits, so that it can model both the internal state transitions of each arm and the influence of external global environmental contexts. Using the dual decomposition method, we develop a scalable index policy algorithm for solving the CRB problem, and theoretically analyze the asymptotical optimality of this algorithm. In the case when the arm models are unknown, we further propose a model-based online learning algorithm based on the index policy to learn the arm models and make decisions simultaneously. Furthermore, we apply the proposed CRB framework and the index policy algorithm specifically to the demand response decision-making problem in smart grids. The numerical simulations demonstrate the performance and efficiency of our proposed CRB approaches.
more »
« less
Diagnosing environmental problems and their possible policy responses: A tool for assessing initial options
The complexities of many environmental problems make the task of identifying potential solutions daunting. We present a diagnostic framework to help guide environmental policy analysts and practitioners to think more systematically about the major types of environmental problems and their possible policy responses. Our framework helps the user classify a problem into 1 of the 3 main problem categories, and then for each of the problem types think about contextual factors that will influence the choice of policy responses. The main problem types are (1) common-pool resource (CPR) problems (e.g., overfishing, groundwater depletion, and forest degradation); (2) pollution problems (e.g., greenhouse gas emissions, eutrophication, acid rain, and smog); and (3) hazards (natural and human-made hazards, including hurricanes, wildfires, and levy collapse). For each of these problems, the framework asks users to consider several contextual factors that are known to influence the likely effectiveness of different policy responses, particularly fast-thinking behavior. The framework is a heuristic tool that will help novice analysts develop a deeper understanding of the problems at hand and an appreciation for the complexities involved in coming up with workable solutions to environmental challenges. The proposed framework is not prescriptive but analytical in that it asks users guiding questions to assess multiple aspects of a problem. The resulting problem assessment helps to narrow down the number of viable options for environmental policy responses, each of which may, in turn, be assessed with an eye toward their legal, political, and social viability.
more »
« less
- PAR ID:
- 10515168
- Publisher / Repository:
- University of California Press
- Date Published:
- Journal Name:
- Elem Sci Anth
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2325-1026
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take—including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses—and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.more » « less
-
Holistic engineering is an approach to the engineering profession, rather than an engineering discipline such as civil, electrical, or mechanical engineering. It is inspired by the realization that traditional engineering does not adequately harness professional skills in its problem-solving repertoire. Holistic engineering asks engineers to look outward, beyond the fields of math and science, in search of solutions to entire problems. While engineering graduates are well prepared in the technical aspects of the engineering profession, they lack non-technical professional skills (e.g., strategic communication, social science perspective of engineering problems, and others) that can help them think through diverse social aspects posed by current complex engineering grand challenges. In this paper, we review the concept and origins of holistic engineering and we present an application of this concept in a Holistic Engineering Project Course (HEPC) developed as part of a National Science Foundation (NSF) grant. HEPC is developed in such a way that engineering students work with social science students on a complex and open-ended engineering grand challenge problem. We hypothesize that such collaborations can significantly improve the professional formation of well-rounded, and effective engineers. The paper also draws lessons learned from the first offering of the course, titled Technology Innovations: Engineering, Economics, and Public Relations, which was offered in the spring semester of 2020 in the Wadsworth Department of Civil and Environmental Engineering in coordination with the John Chambers Department of Economics and the Reed College of Media in West Virginia University.more » « less
-
How can global megacities respond to increased threats from natural hazards? Looking at hazard events that produced drinking water crises in Bogota, Colombia, and Mexico City, Mexico, we compare these cities’ efforts to decrease potable water consumption. We ask how and why the socioeconomic and biophysical contexts shape city policymakers’ responses to water crises and consider the immediate and lasting impacts of technical and behavioral interventions. Leveraging our 2 case studies in Latin America, we identify how the unique policy contexts affected the interventions used and helped determine their outcomes. Four factors are identified as particularly relevant as follows: the differential roles of scientific and technical perspectives in each context, the role of ideology, the complexity of the environmental policy problems in each setting, and the varied roles of policy entrepreneurs in Bogota and Mexico City.more » « less
-
Abstract End users are increasingly using trigger-action platforms like If-This-Then-That (IFTTT) to create applets to connect smart-home devices and services. However, there are inherent implicit risks in using such applets—even non-malicious ones—as sensitive information may leak through their use in certain contexts ( e.g., where the device is located, who can observe the resultant action). This work aims to understand to what extent end users can assess this implicit risk. More importantly we explore whether usage context makes a difference in end-users’ perception of such risks. Our work complements prior work that has identified the impact of usage context on expert evaluation of risks in IFTTT by focusing the impact of usage context on end-users’ risk perception. Through a Mechanical Turk survey of 386 participants on 49 smart-home IFTTT applets, we found that participants have a nuanced view of contextual factors and that different values for contextual factors impact end-users’ risk perception differently. Further, our findings show that nudging the participants to think about different usage contexts led them to think deeper about the associated risks and raise their concern scores.more » « less
An official website of the United States government

