An official website of the United States government
Sequential decision-making problems in the context of uncertainty naturally arise in healthcare settings. In general, the frequency at which decisions can be made or changed is determined by physical limitations, such as the frequency of doctor’s visits or transplantation offers. Quantifying the benefits of increasing the frequency of decision-making allows us to quantify the value of changing these physical constraints and thus improve the quality of care. In this article, we study the value provided by having additional decision-making opportunities in each epoch. We model this problem using a Markov Decision Process (MDP) framework. We provide structural properties of the optimal policies and quantify the difference in optimal values between MDP problems of different decision-making frequencies. We analyze numerical examples using liver transplantation in high-risk patients and treatment initiation in chronic kidney disease to illustrate our findings.
more »
« less
Reframing wildlife disease management problems with decision analysis
Abstract Contemporary wildlife disease management is complex because managers need to respond to a wide range of stakeholders, multiple uncertainties, and difficult trade‐offs that characterize the interconnected challenges of today. Despite general acknowledgment of these complexities, managing wildlife disease tends to be framed as a scientific problem, in which the major challenge is lack of knowledge. The complex and multifactorial process of decision‐making is collapsed into a scientific endeavor to reduce uncertainty. As a result, contemporary decision‐making may be oversimplified, rely on simple heuristics, and fail to account for the broader legal, social, and economic context in which the decisions are made. Concurrently, scientific research on wildlife disease may be distant from this decision context, resulting in information that may not be directly relevant to the pertinent management questions. We propose reframing wildlife disease management challenges as decision problems and addressing them with decision analytical tools to divide the complex problems into more cognitively manageable elements. In particular, structured decision‐making has the potential to improve the quality, rigor, and transparency of decisions about wildlife disease in a variety of systems. Examples of management of severe acute respiratory syndrome coronavirus 2, white‐nose syndrome, avian influenza, and chytridiomycosis illustrate the most common impediments to decision‐making, including competing objectives, risks, prediction uncertainty, and limited resources.
more »
« less
- Award ID(s):
- 2200310
- PAR ID:
- 10645420
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- Conservation Biology
- Volume:
- 38
- Issue:
- 4
- ISSN:
- 0888-8892
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Modeling is essential to characterize and explore complex societal and environmental issues in systematic and collaborative ways. Socio-environmental systems (SES) modeling integrates knowledge and perspectives into conceptual and computational tools that explicitly recognize how human decisions affect the environment. Depending on the modeling purpose, many SES modelers also realize that involvement of stakeholders and experts is fundamental to support social learning and decision-making processes for achieving improved environmental and social outcomes. The contribution of this paper lies in identifying and formulating grand challenges that need to be overcome to accelerate the development and adaptation of SES modeling. Eight challenges are delineated: bridging epistemologies across disciplines; multi-dimensional uncertainty assessment and management; scales and scaling issues; combining qualitative and quantitative methods and data; furthering the adoption and impacts of SES modeling on policy; capturing structural changes; representing human dimensions in SES; and leveraging new data types and sources. These challenges limit our ability to effectively use SES modeling to provide the knowledge and information essential for supporting decision making. Whereas some of these challenges are not unique to SES modeling and may be pervasive in other scientific fields, they still act as barriers as well as research opportunities for the SES modeling community. For each challenge, we outline basic steps that can be taken to surmount the underpinning barriers. Thus, the paper identifies priority research areas in SES modeling, chiefly related to progressing modeling products, processes and practices.more » « less
-
From out-competing grandmasters in chess to informing high-stakes healthcare decisions, emerging methods from artificial intelligence are increasingly capable of making complex and strategic decisions in diverse, high-dimensional and uncertain situations. But can these methods help us devise robust strategies for managing environmental systems under great uncertainty? Here we explore how reinforcement learning (RL), a subfield of artificial intelligence, approaches decision problems through a lens similar to adaptive environmental management: learning through experience to gradually improve decisions with updated knowledge. We review where RL holds promise for improving evidence-informed adaptive management decisions even when classical optimization methods are intractable and discuss technical and social issues that arise when applying RL to adaptive management problems in the environmental domain. Our synthesis suggests that environmental management and computer science can learn from one another about the practices, promises and perils of experience-based decision-making. This article is part of the theme issue ‘Detecting and attributing the causes of biodiversity change: needs, gaps and solutions’.more » « less
-
Many resource management problems require sequential decision-making under uncertainty, where the only uncertainty affecting the decision outcomes are exogenous variables outside the control of the decision-maker. We model these problems as Exo-MDPs (Markov Decision Processes with Exogenous Inputs) and design a class of data-efficient algorithms for them termed Hindsight Learning (HL). Our HL algorithms achieve data efficiency by leveraging a key insight: having samples of the exogenous variables, past decisions can be revisited in hindsight to infer counterfactual consequences that can accelerate policy improvements. We compare HL against classic baselines in the multi-secretary and airline revenue management problems. We also scale our algorithms to a business-critical cloud resource management problem – allocating Virtual Machines (VMs) to physical machines, and simulate their performance with real datasets from a large public cloud provider. We find that HL algorithms outperform domain-specific heuristics, as well as state-of-the-art reinforcement learning methods.more » « less
-
null (Ed.)Existing collaborations among public health practitioners, veterinarians, and ecologists do not sufficiently consider illegal wildlife trade in their surveillance, biosafety, and security (SB&S) efforts even though the risks to health and biodiversity from these threats are significant. We highlight multiple cases to illustrate the risks posed by existing gaps in understanding the intersectionality of the illegal wildlife trade and zoonotic disease transmission. We argue for more integrative science in support of decision-making using the One Health approach. Opportunities abound to apply transdisciplinary science to sustainable wildlife trade policy and programming, such as combining on-the-ground monitoring of health, environmental, and social conditions with an understanding of the operational and spatial dynamics of illicit wildlife trade. We advocate for (1) a surveillance sample management system for enhanced diagnostic efficiency in collaboration with diverse and local partners that can help establish new or link existing surveillance networks, outbreak analysis, and risk mitigation strategies; (2) novel analytical tools and decision support models that can enhance self-directed local livelihoods by addressing monitoring, detection, prevention, interdiction, and remediation; (3) enhanced capacity to promote joint SB&S efforts that can encourage improved human and animal health, timely reporting, emerging disease detection, and outbreak response; and, (4) enhanced monitoring of illicit wildlife trade and supply chains across the heterogeneous context within which they occur. By integrating more diverse scientific disciplines, and their respective scientists with indigenous people and local community insight and risk assessment data, we can help promote a more sustainable and equitable wildlife trade.more » « less
