An official website of the United States government
This paper introduces a novel multi-armed bandits framework, termed Contextual Restless Bandits (CRB), for complex online decision-making. This CRB framework incorporates the core features of contextual bandits and restless bandits, so that it can model both the internal state transitions of each arm and the influence of external global environmental contexts. Using the dual decomposition method, we develop a scalable index policy algorithm for solving the CRB problem, and theoretically analyze the asymptotical optimality of this algorithm. In the case when the arm models are unknown, we further propose a model-based online learning algorithm based on the index policy to learn the arm models and make decisions simultaneously. Furthermore, we apply the proposed CRB framework and the index policy algorithm specifically to the demand response decision-making problem in smart grids. The numerical simulations demonstrate the performance and efficiency of our proposed CRB approaches.
more »
« less
Diagnosing environmental problems and their possible policy responses: A tool for assessing initial options
The complexities of many environmental problems make the task of identifying potential solutions daunting. We present a diagnostic framework to help guide environmental policy analysts and practitioners to think more systematically about the major types of environmental problems and their possible policy responses. Our framework helps the user classify a problem into 1 of the 3 main problem categories, and then for each of the problem types think about contextual factors that will influence the choice of policy responses. The main problem types are (1) common-pool resource (CPR) problems (e.g., overfishing, groundwater depletion, and forest degradation); (2) pollution problems (e.g., greenhouse gas emissions, eutrophication, acid rain, and smog); and (3) hazards (natural and human-made hazards, including hurricanes, wildfires, and levy collapse). For each of these problems, the framework asks users to consider several contextual factors that are known to influence the likely effectiveness of different policy responses, particularly fast-thinking behavior. The framework is a heuristic tool that will help novice analysts develop a deeper understanding of the problems at hand and an appreciation for the complexities involved in coming up with workable solutions to environmental challenges. The proposed framework is not prescriptive but analytical in that it asks users guiding questions to assess multiple aspects of a problem. The resulting problem assessment helps to narrow down the number of viable options for environmental policy responses, each of which may, in turn, be assessed with an eye toward their legal, political, and social viability.
more »
« less
- PAR ID:
- 10515168
- Publisher / Repository:
- University of California Press
- Date Published:
- Journal Name:
- Elem Sci Anth
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2325-1026
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract A hallmark of adaptation in humans and other animals is our ability to control how we think and behave across different settings. Research has characterized the various forms cognitive control can take—including enhancement of goal-relevant information, suppression of goal-irrelevant information, and overall inhibition of potential responses—and has identified computations and neural circuits that underpin this multitude of control types. Studies have also identified a wide range of situations that elicit adjustments in control allocation (e.g., those eliciting signals indicating an error or increased processing conflict), but the rules governing when a given situation will give rise to a given control adjustment remain poorly understood. Significant progress has recently been made on this front by casting the allocation of control as a decision-making problem. This approach has developed unifying and normative models that prescribe when and how a change in incentives and task demands will result in changes in a given form of control. Despite their successes, these models, and the experiments that have been developed to test them, have yet to face their greatest challenge: deciding how to select among the multiplicity of configurations that control can take at any given time. Here, we will lay out the complexities of the inverse problem inherent to cognitive control allocation, and their close parallels to inverse problems within motor control (e.g., choosing between redundant limb movements). We discuss existing solutions to motor control's inverse problems drawn from optimal control theory, which have proposed that effort costs act to regularize actions and transform motor planning into a well-posed problem. These same principles may help shed light on how our brains optimize over complex control configuration, while providing a new normative perspective on the origins of mental effort.more » « less
-
Holistic engineering is an approach to the engineering profession, rather than an engineering discipline such as civil, electrical, or mechanical engineering. It is inspired by the realization that traditional engineering does not adequately harness professional skills in its problem-solving repertoire. Holistic engineering asks engineers to look outward, beyond the fields of math and science, in search of solutions to entire problems. While engineering graduates are well prepared in the technical aspects of the engineering profession, they lack non-technical professional skills (e.g., strategic communication, social science perspective of engineering problems, and others) that can help them think through diverse social aspects posed by current complex engineering grand challenges. In this paper, we review the concept and origins of holistic engineering and we present an application of this concept in a Holistic Engineering Project Course (HEPC) developed as part of a National Science Foundation (NSF) grant. HEPC is developed in such a way that engineering students work with social science students on a complex and open-ended engineering grand challenge problem. We hypothesize that such collaborations can significantly improve the professional formation of well-rounded, and effective engineers. The paper also draws lessons learned from the first offering of the course, titled Technology Innovations: Engineering, Economics, and Public Relations, which was offered in the spring semester of 2020 in the Wadsworth Department of Civil and Environmental Engineering in coordination with the John Chambers Department of Economics and the Reed College of Media in West Virginia University.more » « less
-
This paper presents an innovative approach to improve engineering students’ problem-solving skills by implementing think-aloud exercises. Sometimes engineering students claim they do not know where to start with the problem-solving process, or they are not sure how to proceed to the next steps when they get stuck. A systematic training that focuses on the problem-solving process and the justification of each step could help. Think-aloud techniques help make the invisible mental processes visible to learners. Engineering think-aloud technique engages students and helps them make their way through a solving process step-by-step, reasoning along with them. In this study, a multiple faceted systematic approach that integrates think-aloud exercises through video assignments and oral exams were developed and implemented in two pilot engineering classes. We present our think-aloud exercises and oral exams structures in each of the courses and their impacts on students' learning outcomes, and students’ perceptions towards the pedagogical approach. Both quantitative and qualitative results show that the think-aloud exercise assignments and oral exams enhance students’ problem-solving skills and promote learning.more » « less
-
Much of environmental law and policy rests on an unspoken premise that accomplishing environmental goals may not require addressing root causes of environmental problems. For example, rather than regulating risks directly, society may adopt warnings that merely avoid risk, and rather than limiting plastic use and reducing plastic waste, society may adopt recycling programs. Such approaches may be well-intended and may come at a relatively low economic or political cost. However, they often prove ineffective or even harmful, and they may mislead society into believing that further responses are unnecessary. This Article proposes the concept of “too-easy solutions” to describe these approaches. Too easy solutions can be classified into three subcategories: fig leaves—policy approaches that appear to do something about a problem without necessarily solving it; pipe dreams—policy approaches that are adopted with the good faith expectation of solving the problem but are inherently flawed; and myopic solutions—approaches that address part of the problem but may impede its overall resolution. Too-easy solutions analysis can serve as a powerful mechanism for evaluating policies and improving decisionmaking in the environmental arena and other areas as well.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1525/elementa.2022.00137
