Search for: All records

The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development. We build upon this observation by formalizing an algorithm for learning from natural language feedback at training time instead, which we call Imitation learning from Language Feedback (ILF). ILF requires only a small amount of human-written feedback during training and does not require the same feedback at test time, making it both user-friendly and sample-efficient. We further show that ILF can be seen as a form of minimizing the KL divergence to the target distribution and demonstrate proof-of-concepts on text summarization and program synthesis tasks. For code generation, ILF improves a Codegen-Mono 6.1B model’s pass@1 rate from 22% to 36% on the MBPP benchmark, outperforming both fine-tuning on MBPP and on human- written repaired programs. For summarization, we show that ILF can be combined with learning from human preferences to improve a GPT-3 model’s summarization performance to be comparable to human quality, outperforming fine-tuning on human-written summaries. Overall, our results suggest that ILF is both more effective and sample-efficient than training exclusively on demonstrations for improving an LLM’s performance on a variety of tasks. 

Large language models (LLMs) have achieved widespread success on a variety of in-context few shot tasks, but this success is typically evaluated via correctness rather than consistency. We argue that self-consistency is an important criteria for valid multi-step reasoning in tasks where the solution is composed of the answers to multiple sub-steps. We propose two types of self consistency that are particularly important for multi-step reasoning – hypothetical consistency (a model’s ability to predict what its output would be in a hypothetical other context) and compositional consistency (consistency of a model’s final outputs when intermediate sub-steps are replaced with the model’s outputs for those steps). We demonstrate that multiple variants of the GPT-3/-4 models exhibit poor consistency rates across both types of consistency on a variety of tasks. 

A popular approach to protein design is to combine a generative model with a discriminative model for conditional sampling. The generative model samples plausible sequences while the discriminative model guides a search for sequences with high fitness. Given its broad success in conditional sampling, classifier-guided diffusion modeling is a promising foundation for protein design, leading many to develop guided diffusion models for structure with inverse folding to recover sequences. In this work, we propose diffusioN Optimized Sampling (NOS), a guidance method for discrete diffusion models that follows gradients in the hidden states of the denoising network. NOS makes it possible to perform design directly in sequence space, circumventing significant limitations of structure-based methods, including scarce data and challenging inverse design. Moreover, we use NOS to generalize LaMBO, a Bayesian optimization procedure for sequence design that facilitates multiple objectives and edit-based constraints. The resulting method, LaMBO-2, enables discrete diffusions and stronger performance with limited edits through a novel application of saliency maps. We apply LaMBO-2 to a real-world protein design task, optimizing antibodies for higher expression yield and binding affinity to several therapeutic targets under locality and developability constraints, attaining a 99% expression rate and 40% binding rate in exploratory in vitro experiments. 

Abstract Recent rapid thinning of West Antarctic ice shelves are believed to be caused by intrusions of warm deep water that induce basal melting and seaward meltwater export. This study uses data from three bottom-mounted mooring arrays to show seasonal variability and local forcing for the currents moving into and out of the Dotson ice shelf cavity. A southward flow of warm, salty water had maximum current velocities along the eastern channel slope, while northward outflows of freshened ice shelf meltwater spread at intermediate depth above the western slope. The inflow correlated with the local ocean surface stress curl. At the western slope, meltwater outflows followed the warm influx along the eastern slope with a ~2–3 month delay. Ocean circulation near Dotson Ice Shelf, affected by sea ice distribution and wind, appears to significantly control the inflow of warm water and subsequent ice shelf melting on seasonal time-scales. 

We report measurements of production cross sections for ${\rho }^{+}$, ${\rho }^0$, $\omega$, $K^{*+}$, $K^{*0}$, $\varphi$, $\eta$, $K_S^0$, $f_0\left(980\right)$, $D^{+}$, $D^0$, $D_s^{+}$, $D^{*+}$, $D^{*0}$, and $D_s^{*+}$in $e^{+}{e}^{-}$collisions at a center-of-mass energy near 10.58 GeV. The data were recorded by the Belle experiment, consisting of $571{\mathrm{fb}}^{-1}$at 10.58 GeV and $74{\mathrm{fb}}^{-1}$at 10.52 GeV. Production cross sections are extracted as a function of the fractional hadron momentum $x_p$. The measurements are compared to Monte Carlo generator predictions with various fragmentation settings, including those that have increased fragmentation into vector mesons over pseudoscalar mesons. The cross sections measured for light hadrons are consistent with no additional increase of vector over pseudoscalar mesons. The charmed-meson cross sections are compared to earlier measurements—when available—including older Belle results, which they supersede. They are in agreement before application of an improved initial-state radiation correction procedure that causes slight changes in their $x_p$shapes. Published by the American Physical Society2025 

We present a framework for compactly summarizing many recent results in efficient and/or biologically plausible online training of recurrent neural networks (RNN). The framework organizes algorithms according to several criteria: (a) past vs. future facing, (b) tensor structure, (c) stochastic vs. deterministic, and (d) closed form vs. numerical. These axes reveal latent conceptual connections among several recent advances in online learning. Furthermore, we provide novel mathematical intuitions for their degree of success. Testing various algorithms on two synthetic tasks shows that performances cluster according to our criteria. Although a similar clustering is also observed for gradient alignment, alignment with exact methods does not alone explain ultimate performance, especially for stochastic algorithms. This suggests the need for better comparison metrics. 

In the bottomonium sector, the hindered magnetic dipole transitions between P-wave states $h_b\left(2P\right)\to {\chi }_{bJ}\left(1P\right)\gamma$, $J=0$, 1, 2, are expected to be severely suppressed according to the relativized quark model, due to the spin flip of the $b$quark. Nevertheless, a recent model following the coupled-channel approach predicts the corresponding branching fractions to be enhanced by orders of magnitude. In this Letter, we report the first search for such transitions. We find no significant signals and set upper limits at 90% confidence level on the corresponding branching fractions: $\mathcal{B}\left[h_b\right(2P)\to \gamma {\chi }_{b0}(1P\left)\right]<2.7\times {10}^{-1}$, $\mathcal{B}\left[h_b\right(2P)\to \gamma {\chi }_{b1}(1P\left)\right]<5.4\times {10}^{-3}$and $\mathcal{B}\left[h_b\right(2P)\to \gamma {\chi }_{b2}(1P\left)\right]<1.3\times {10}^{-2}$. These values help to constrain the parameters of the coupled-channel models. The results are obtained using a $121.4{\mathrm{fb}}^{-1}$data sample taken around $\sqrt{s}=10.860\mathrm{GeV}$with the Belle detector at the KEKB asymmetric-energy $e^{+}{e}^{-}$collider. Published by the American Physical Society2025 

We report the first evidence for the $h_b\left(2\mathrm{P}\right)\to \mathrm{\Upsilon }\left(1\mathrm{S}\right)\eta$transition with a significance of 3.5 standard deviations. The decay branching fraction is measured to be $\mathcal{B}\left[h_b\right(2\mathrm{P})\to \mathrm{\Upsilon }(1\mathrm{S}\left)\eta \right]=({7.1}_{-3.2}^{+3.7}\pm 0.8)\times {10}^{-3}$, which is noticeably smaller than expected. We also set upper limits on ${\pi }^0$transitions of $\mathcal{B}\left[h_b\right(2\mathrm{P})\to \mathrm{\Upsilon }(1\mathrm{S}\left){\pi }^0\right]<1.8\times {10}^{-3}$, and $\mathcal{B}\left[h_b\right(1\mathrm{P})\to \mathrm{\Upsilon }(1\mathrm{S}\left){\pi }^0\right]<1.8\times {10}^{-3}$, at the 90% confidence level. These results are obtained with a $131.4{\mathrm{fb}}^{-1}$data sample collected near the $\mathrm{\Upsilon }\left(5\mathrm{S}\right)$resonance with the Belle detector at the KEKB asymmetric-energy $e^{+}{e}^{-}$collider. Published by the American Physical Society2024