An official website of the United States government
Reinforcement learning (RL) is the subfield of machine learning focused on optimal sequential decision making under uncertainty. An optimal RL strategy maximizes cumulative utility by experimenting only if and when the information generated by experimentation is likely to outweigh associated short-term costs. RL represents a holistic approach to decision making that evaluates the impact of every action (ie, data collection, allocation of resources, and treatment assignment) in terms of short-term and long-term utility to stakeholders. Thus, RL is an ideal model for a number of complex decision problems that arise in public health, including resource allocation in a pandemic, monitoring or testing, and adaptive sampling for hidden populations. Nevertheless, although RL has been applied successfully in a wide range of domains, including precision medicine, it has not been widely adopted in public health. The purposes of this review are to introduce key ideas in RL and to identify challenges and opportunities associated with the application of RL in public health.
more »
« less
Learning what to read: Focused machine reading
Recent efforts in bioinformatics have achieved tremendous progress in the machine reading of biomedical literature, and the assembly of the extracted biochemical interactions into large-scale models such as protein signaling pathways. However, batch machine reading of literature at today’s scale (PubMed alone indexes over 1 million papers per year) is unfeasible due to both cost and processing overhead. In this work, we introduce a focused reading approach to guide the machine reading of biomedical literature towards what literature should be read to answer a biomedical query as efficiently as possible. We introduce a family of algorithms for focused reading, including an intuitive, strong baseline, and a second approach which uses a reinforcement learning (RL) framework that learns when to explore (widen the search) or exploit (narrow it). We demonstrate that the RL approach is capable of answering more queries than the baseline, while being more efficient, i.e., reading fewer documents.
more »
« less
- Award ID(s):
- 1740858
- PAR ID:
- 10111660
- Date Published:
- Journal Name:
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Page Range / eLocation ID:
- 2905 to 2910
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The rise in popularity of two-photon polymerization (TPP) as an additive manufacturing technique has impacted many areas of science and engineering, particularly those related to biomedical applications. Compared with other fabrication methods used for biomedical applications, TPP offers 3D, nanometer-scale fabrication dexterity (free-form). Moreover, the existence of turnkey commercial systems has increased accessibility. In this review, we discuss the diversity of biomedical applications that have benefited from the unique features of TPP. We also present the state of the art in approaches for patterning and reading 3D TPP-fabricated structures. The reading process influences the fidelity for both in situ and ex situ characterization methods. We also review efforts to leverage machine learning to facilitate process control for TPP. Finally, we conclude with a discussion of both the current challenges and exciting opportunities for biomedical applications that lie ahead for this intriguing and emerging technology.more » « less
-
Serverless Function-as-a-Service (FaaS) offers improved programmability for customers, yet it is not server-“less” and comes at the cost of more complex infrastructure management (e.g., resource provisioning and scheduling) for cloud providers. To maintain function service-level objectives (SLOs) and improve resource utilization efficiency, recent research has been focused on applying online learning algorithms such as reinforcement learning (RL) to manage resources. Compared to rule-based solutions with heuristics, RL-based approaches eliminate humans in the loop and avoid the painstaking generation of heuristics. Despite the initial success of applying RL, we first show in this paper that the state-of-the-art single-agent RL algorithm (S-RL) suffers up to 4.8x higher p99 function latency degradation on multi-tenant serverless FaaS platforms compared to isolated environments and is unable to converge during training. We then design and implement a scalable and incremental multi-agent RL framework based on Proximal Policy Optimization (SIMPPO). Our experiments on widely used serverless benchmarks demonstrate that in multi-tenant environments, SIMPPO enables each RL agent to efficiently converge during training and provides online function latency performance comparable to that of S-RL trained in isolation (which we refer to as the baseline for assessing RL performance) with minor degradation (<9.2%). In addition, SIMPPO reduces the p99 function latency by 4.5x compared to S-RL in multi-tenant cases.more » « less
-
We consider the problem of learning to simplify medical texts. This is important because most reliable, up-to-date information in biomedicine is dense with jargon and thus practically inaccessible to the lay audience. Furthermore, manual simplification does not scale to the rapidly growing body of biomedical literature, motivating the need for automated approaches. Unfortunately, there are no large-scale resources available for this task. In this work we introduce a new corpus of parallel texts in English comprising technical and lay summaries of all published evidence pertaining to different clinical topics. We then propose a new metric based on likelihood scores from a masked language model pretrained on scientific texts. We show that this automated measure better differentiates between technical and lay summaries than existing heuristics. We introduce and evaluate baseline encoder-decoder Transformer models for simplification and propose a novel augmentation to these in which we explicitly penalize the decoder for producing “jargon” terms; we find that this yields improvements over baselines in terms of readability.more » « less
-
Abstract The ever increasing popularity of machine learning methods in virtually all areas of science, engineering and beyond is poised to put established statistical modeling approaches into question. Environmental statistics is no exception, as popular constructs such as neural networks and decision trees are now routinely used to provide forecasts of physical processes ranging from air pollution to meteorology. This presents both challenges and opportunities to the statistical community, which could contribute to the machine learning literature with a model‐based approach with formal uncertainty quantification. Should, however, classical statistical methodologies be discarded altogether in environmental statistics, and should our contribution be focused on formalizing machine learning constructs? This work aims at providing some answers to this thought‐provoking question with two time series case studies where selected models from both the statistical and machine learning literature are compared in terms of forecasting skills, uncertainty quantification and computational time. Relative merits of both class of approaches are discussed, and broad open questions are formulated as a baseline for a discussion on the topic.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18653/v1/D17-1313
