Search for: All records

Does language help make sense of the visual world? How important is it to actually see the world rather than having it described with words? These basic questions about the na- ture of intelligence have been difficult to answer because we only had one example of an intelligent system – humans – and limited access to cases that isolated language or vision. How- ever, the development of sophisticated Vision-Language Mod- els (VLMs) by artificial intelligence researchers offers us new opportunities to explore the contributions that language and vi- sion make to learning about the world. We ablate components from the cognitive architecture of these models to identify their contributions to learning new tasks from limited data. We find that a language model leveraging all components recovers a majority of a VLM’s performance, despite its lack of visual in- put, and that language seems to allow this by providing access to prior knowledge and reasoning. 

Abstract Model-observation comparisons of type-I X-ray bursts (XRBs) can reveal the properties of accreting neutron star systems, including the neutron star compactness. XRBs are powered by nuclear burning, and a handful of reactions have been shown to impact the model results. Reactions in the NiCu cycles, featuring a competition between⁵⁹Cu(p,γ)⁶⁰Zn and⁵⁹Cu(p,α)⁵⁶Ni, have been shown to be among the most important reactions as they are a critical checkpoint inrp-process flow and significantly impact the light curves and burst ashes. We report a direct measurement of⁵⁹Cu(p,α)⁵⁶Ni, bringing stringent constraints on this reaction rate. New results rule out a strong NiCu cycle in XRBs, with a negligible degree of recycling, ≤5% up to 1.5 GK. The new reaction rate, when varied within new uncertainty limits, shows no impact on one-zone XRB model light curves tailored for clocked-bursterGS1826–24, hence removing an important nuclear physics uncertainty in the model-observation comparison. 

Aims.Global particle-in-cell (PIC) simulations of pulsar magnetospheres are performed with volume-, surface-, and pair-production-based plasma injection schemes to systematically investigate the transition between electrosphere and force-free pulsar magnetospheric regimes. Methods.We present a new extension of the PIC code OSIRIS that can be used to model pulsar magnetospheres with a two-dimensional axisymmetric spherical grid. The subalgorithms of the code and thorough benchmarks are presented in detail, including a new first-order current deposition scheme that conserves charge to machine precision. Results.We show that all plasma injection schemes produce a range of magnetospheric regimes. Active solutions can be obtained with surface and volume injection schemes when using artificially large plasma-injection rates, and with pair-production-based plasma injection for sufficiently large separation between kinematic and pair-production energy scales. 

The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development. We build upon this observation by formalizing an algorithm for learning from natural language feedback at training time instead, which we call Imitation learning from Language Feedback (ILF). ILF requires only a small amount of human-written feedback during training and does not require the same feedback at test time, making it both user-friendly and sample-efficient. We further show that ILF can be seen as a form of minimizing the KL divergence to the target distribution and demonstrate proof-of-concepts on text summarization and program synthesis tasks. For code generation, ILF improves a Codegen-Mono 6.1B model’s pass@1 rate from 22% to 36% on the MBPP benchmark, outperforming both fine-tuning on MBPP and on human- written repaired programs. For summarization, we show that ILF can be combined with learning from human preferences to improve a GPT-3 model’s summarization performance to be comparable to human quality, outperforming fine-tuning on human-written summaries. Overall, our results suggest that ILF is both more effective and sample-efficient than training exclusively on demonstrations for improving an LLM’s performance on a variety of tasks. 

Large language models (LLMs) have achieved widespread success on a variety of in-context few shot tasks, but this success is typically evaluated via correctness rather than consistency. We argue that self-consistency is an important criteria for valid multi-step reasoning in tasks where the solution is composed of the answers to multiple sub-steps. We propose two types of self consistency that are particularly important for multi-step reasoning – hypothetical consistency (a model’s ability to predict what its output would be in a hypothetical other context) and compositional consistency (consistency of a model’s final outputs when intermediate sub-steps are replaced with the model’s outputs for those steps). We demonstrate that multiple variants of the GPT-3/-4 models exhibit poor consistency rates across both types of consistency on a variety of tasks. 

Abstract Neural responses evoked by a stimulus reduce upon repetition. While this adaptation allows the sensory system to attend to novel cues, does information about the recurring stimulus particularly its intensity get compromised? We explored this issue in the locust olfactory system. We found that locusts’ innate behavioral response to odorants varied with repetition and stimulus intensity. Counter-intuitively, the stimulus-intensity dependent differences became significant only after adaptation had set in. Adaptation also altered responses of individual neurons in the antennal lobe (neural network downstream to insect antenna). These response variations to repetitions of the same stimulus were unpredictable and inconsistent across intensities. Although both adaptation and intensity decrements resulted in an overall reduction in spiking activities across neurons, these changes could be disentangled and information about stimulus intensity robustly maintained by ensemble neural responses. In sum, these results show how information about odor intensity can be preserved in an adaptation-invariant manner. 

Language models (LMs) are pretrained to imitate internet text, including content that would violate human preferences if generated by an LM: falsehoods, offensive comments, personally identifiable information, low-quality or buggy code, and more. Here, we explore alternative objectives for pretraining LMs in a way that also guides them to generate text aligned with human preferences. We benchmark five objectives for pretraining with human feedback across three tasks and study how they affect the trade-off between alignment and capabilities of pretrained LMs. We find a Pareto optimal and simple approach among those we explored: conditional training, or learning distribution over tokens conditional on their human preference scores given by a reward model. Conditional training reduces the rate of undesirable content by up to an order of magnitude, both when generating without a prompt and with an adversarially chosen prompt. Moreover, conditional training maintains the downstream task performance of standard LM pretraining, both before and after task-specific finetuning. Pretraining with human feedback results in much better preference satisfaction than standard LM pretraining followed by finetuning with feedback, i.e., learning and then unlearning undesirable behavior. Our results suggest that we should move beyond imitation learning when pretraining LMs and incorporate human preferences from the start of training.