Attention:The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, October 15th until 12:00 AM ET on Friday, October 16th due to maintenance. We apologize for the inconvenience.


Title: Task Vectors in In-Context Learning: Emergence, Formation, and Benefits
This paper studies the emergence, formation, and benefits of task vectors in in-context learning.  more » « less
Award ID(s):
2339978
PAR ID:
10690631
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Conference on Language Modeling
Date Published:
Edition / Version:
arXiv:2501.09240v1
Subject(s) / Keyword(s):
in-context learning task vectors transformers machine learning
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Bar-Cohen, Y; Zacny, K (Ed.)
    The need for scientific ice drilling in glaciers and ice sheets has been driven by many fields of science, including drilling ice cores for evidence of past environment and paleoclimate information, and drilling access holes through the ice to gather data relevant to glacial dynamics, history of glacier extent, sediment sampling, and discovery of ecosystems within and beneath the ice. Many nations have contributed to drilling technologies relevant to each of these fields, and developments in any one nation often build on prior designs from other nations. A description of the very early polar ice coring endeavors in Greenland and Antarctica is provided in Langway (2008). Ice drilling and coring technologies that were developed before 2008 are well described in Bentley et al (2009), including a wide array of ice coring drills, drills designed to create holes in ice only, and autonomous instruments that melt their way through ice. The text by [Talalay 2016] provides a review of mechanical ice drilling technology that includes design, parameters and performance of an assortment of tools and drills for making holes in snow, firn and ice. Described in detail are direct-push drilling, hand- and power-driven portable drills, percussion drills, conventional machine-driven rotary drill rigs, flexible drill-stem drill rigs, cable-suspended electromechanical auger drills, cable-suspended electromechanical drills with bottom-hole circulation, and drilling challenges and perspective for future development. In this chapter our goal is to describe new ice drilling and coring technologies that have been designed, built, and used in the field in the most recent decade. Some of these technologies are improvements on prior drills, while other technologies such as a replicate ice coring drill, geologic drilling underneath many meters of glacial ice, and the rapid access isotope drill are the first of their kind. There are many additional ice drilling and sampling designs currently in the design or development stage that are not included in this chapter; rather our goal in this chapter is to describe proven ice drilling technologies that have been developed since 2009. 
    more » « less
  2. The 116,118,120,122,124In nuclei have been popu- lated as fission fragments in reactions induced by heavy ions. Level schemes have been built from γ-rays detected using the Gammasphere array. Medium-spin states of 118,120In69,71 nuclei have been identified for the first time, while the level schemes of 116,122,124In67,73,75 were enriched. The observed states at lower excitations and at medium spin can be described by the coupling of the proton g9/2 hole to the neutron or neutron-hole in the h11/2 orbital. This coupling can now be followed in all odd-odd In isotopes from 104In to 126In 
    more » « less
  3. While recent reforms in science education envision engaging students in doing science as a way to learn science, less is known about how such an engagement can take hold in the classroom. In an effort to address this gap, this study examines the dynamics of students’ disciplinary engagement in small groups in a middle school science classroom. Using multimodal discourse analysis, we conducted a comparative case analysis of three groups of students to examine the dynamics of their engagement along conceptual, epistemological, social, and affective dimensions. Through this analysis, our findings highlight tensions that emerge along these dimensions and the ways in which students negotiate these tensions in ways that support or hinder disciplinary engagement. 
    more » « less
  4. Schaeffer, J; Novogrodsky, R; Perovic, A; Prévost, P; Tuller, L (Ed.)
    Alongside the behavioral features of autism, this neurodevelopmental disorder is characterized by important differences in the neural circuitry underlying language processing. Regarding brain structure, most neurotypical individuals have larger left-hemisphere volumes of brain regions that are important for language, compared to the same regions in the right hemisphere (the right half of the brain). This asymmetry is due to neural specialization of left hemisphere regions for the purpose of language functions. In contrast, the brains of autistic individuals seem to be more symmetrical, suggesting that language difficulties are associated with reduced left hemisphere specialization for language in the brain. The activity of brain regions involved in language also differs in autism. Examining brain activity reveals nuanced and important differences in the processes underlying language production and comprehension in neurotypical and autistic individuals, even when their language behavior appears similar. 
    more » « less
  5. null (Ed.)
    The brain is a complex network that accounts for only 5% of human mass but consumes 20% of our energy. Uncovering the mysteries of the brain’s functions in motion, memory, learning, behavior, and mental health remains a hot but challenging topic. Neurochemicals in the brain, such as neurotransmitters, neuromodulators, gliotransmitters, hormones, and metabolism substrates and products, play vital roles in mediating and modulating normal brain function, and their abnormal release or imbalanced concentrations can cause various diseases, such as epilepsy, Alzheimer’s disease, and Parkinson’s disease. A wide range of techniques have been used to probe the concentrations of neurochemicals under normal, stimulated, diseased, and drug-induced conditions in order to understand the neurochemistry of drug mechanisms and develop diagnostic tools or therapies. Recent advancements in detection methods, device fabrication, and new materials have resulted in the development of neurochemical sensors with improved performance. However, direct in vivo measurements require a robust sensor that is highly sensitive and selective with minimal fouling and reduced inflammatory foreign body responses. Here, we review recent advances in neurochemical sensor development for in vivo studies, with a focus on electrochemical and optical probes. Other alternative methods are also compared. We discuss in detail the in vivo challenges for these methods and provide an outlook for future directions. 
    more » « less