An official website of the United States government
Large language models (LLMs) demonstrate impressive reasoning abilities, but translating reasoning into actions in the real world remains challenging. In particular, it is unclear how to complete a given task provably within a minimum number of interactions with the external environment, e.g., through an internal mechanism of reasoning. To this end, we propose the first framework with provable regret guarantees to orchestrate reasoning and acting, which we call “reason for future, act for now” (RAFA). Specifically, we design a prompt template for reasoning that learns from the memory buffer and plans a future trajectory over a long horizon (“reason for future”). At each step, the LLM agent takes the initial action of the planned trajectory (“act for now”), stores the collected feedback in the memory buffer, and reinvokes the reasoning routine to replan the future trajectory from the new state. The key idea is to cast reasoning in LLMs as learning and planning in Bayesian adaptive Markov decision processes (MDPs). Correspondingly, we prompt LLMs with the memory buffer to estimate the unknown environment (learning) and generate an optimal trajectory for multiple future steps that maximize a value function (planning). The learning and planning subroutines are performed in an “incontext” manner to emulate the actor-critic update for MDPs. Our theoretical analysis establishes a √T regret, while our experimental validation demonstrates superior empirical performance. Here, T denotes the number of online interactions.
more »
« less
Data Analysis Codes for: "Determining the rate-limiting processes for cell division in Escherichia coli"
Data Analysis Codes for article: Determining the rate-limiting processes for cell division in Escherichia coli; (Nat. Comm. 2024)
more »
« less
- Award ID(s):
- 2313719
- PAR ID:
- 10582962
- Publisher / Repository:
- Zenodo
- Date Published:
- Format(s):
- Medium: X
- Right(s):
- Creative Commons Attribution 4.0 International
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
AI is beginning to transform every aspect of society. With the dramatic increases in AI, K-12 students need to be prepared to understand AI. To succeed as the workers, creators, and innovators of the future, students must be introduced to core concepts of AI as early as elementary school. However, building a curriculum that introduces AI content to K-12 students present significant challenges, such as connecting to prior knowledge, and developing curricula that are meaningful for students and possible for teachers to teach. To lay the groundwork for elementary AI education, we conducted a qualitative study into the design of AI curricular approaches with elementary teachers and students. Interviews with elementary teachers and students suggests four design principles for creating an effective elementary AI curriculum to promote uptake by teachers. This example will present the co-designed curriculum with teachers (PRIMARYAI) and describe how these four elements were incorporated into real-world problem-based learning scenarios.more » « less
-
Additive manufacturing (AM) simulations are effective for materials that are well characterized and published; however, for newer or proprietary materials, they cannot provide accurate results due to the lack of knowledge of the material properties. This work demonstrates the process of the application of mathematical search algorithms to develop an optimized material dataset which results in accurate simulations for the laser directed energy deposition (DED) process. This was performed by first using a well-characterized material, Ti-64, to show the error in the predicted melt pool was accurate, and the error was found to be less than two resolution steps. Then, for 7000-series aluminum using a generic material property dataset from sister alloys, the error was found to be over 600%. The Nelder–Mead search algorithm was then applied to the problem and was able to develop an optimized dataset that had a combined width and depth error of just 9.1%, demonstrating that it is possible to develop an optimized material property dataset that facilitates more accurate simulation of an under-characterized material.more » « less
-
Crowdsourcing is an effective and efficient paradigm for obtaining labels for unlabeled corpus employing crowd workers. This work considers the budget allocation problem for a generalized setting on a graph of instances to be labeled where edges encode instance dependencies. Specifically, given a graph and a labeling budget, we propose an optimal policy to allocate the budget among the instances to maximize the overall labeling accuracy. We formulate the problem as a Bayesian Markov Decision Process (MDP), where we define our task as an optimization problem that maximizes the overall label accuracy under budget constraints. Then, we propose a novel stage-wise reward function that considers the effect of worker labels on the whole graph at each timestamp. This reward function is utilized to find an optimal policy for the optimization problem. Theoretically, we show that our proposed policies are consistent when the budget is infinite. We conduct extensive experiments on five real-world graph datasets and demonstrate the effectiveness of the proposed policies to achieve a higher label accuracy under budget constraints.more » « less
