Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
-
Long-context capabilities are essential for a wide range of applications, including document and video understanding, in-context learning, and inference-time scaling, all of which require models to process and reason over long sequences of text and multimodal data. In this work, we introduce a efficient training recipe for building ultra-long context LLMs from aligned instruct model, pushing the boundaries of context lengths from 128K to 1M, 2M, and 4M tokens. Our approach leverages efficient continued pretraining strategies to extend the context window and employs effective instruction tuning to maintain the instruction-following and reasoning abilities. Our UltraLong-8B, built on Llama3.1-Instruct with our recipe, achieves state-of-the-art performance across a diverse set of long-context benchmarks. Importantly, models trained with our approach maintain competitive performance on standard benchmarks, demonstrating balanced improvements for both long and short context tasks. We further provide an in-depth analysis of key design choices, highlighting the impacts of scaling strategies and data composition. Our findings establish a robust framework for efficiently scaling context lengths while preserving general model capabilities. We release all model weights at: https://ultralong.github.io/.more » « lessFree, publicly-accessible full text available July 2, 2027
-
Dynamic mutations in some human genes containing trinucleotide repeats are associated with severe neurodegenerative and neuromuscular disorders—known as Trinucleotide (or Triplet) Repeat Expansion Diseases (TREDs)—which arise when the repeat number of triplets expands beyond a critical threshold. While the mechanisms causing the DNA triplet expansion are complex and remain largely unknown, it is now recognized that the expandable repeats lead to the formation of nucleotide configurations with atypical structural characteristics that play a crucial role in TREDs. These nonstandard nucleic acid forms include single-stranded hairpins, Z-DNA, triplex structures, G-quartets and slipped-stranded duplexes. Of these, hairpin structures are the most prolific and are associated with the largest number of TREDs and have therefore been the focus of recent single- molecule FRET experiments and molecular dynamics investigations. Here, we review the structural and dynamical properties of nucleic acid hairpins that have emerged from these studies and the implications for repeat expansion mechanisms. The focus will be on CAG, GAC, CTG and GTC hairpins and their stems, their atomistic structures, their stability, and the important role played by structural interrupts.more » « less
-
The trapped residual magnetic flux during the cool-down due to the incomplete Meissner state is a significant source of radio frequency losses in superconducting radio frequency cavities. Here, we clearly correlate the niobium microstructure in elliptical cavity geometry and flux expulsion behavior. In particular, a traditionally fabricated Nb cavity half-cell from an annealed poly-crystalline Nb sheet after an 800 C heat treatment leads to a bi-modal microstructure that ties in with flux trapping and inefficient flux expulsion. This non-uniform microstructure is related to varying strain profiles along the cavity shape. A novel approach to prevent this non-uniform microstructure is presented by fabricating a 1.3 GHz single cell Nb cavity with a cold-worked sheet and subsequent heat treatment leading to better flux expulsion after 800 ∘C/3 h. Microstructural evolution by electron backscattered diffraction-orientation imaging microscopy on cavity cutouts, and flux pinning behavior by dc-magnetization on coupon samples confirms a reduction in flux pinning centers with increased heat treatment temperature. The heat treatment temperature-dependent mechanical properties and thermal conductivity are reported. The significant impact of cold work in this study demonstrates clear evidence for the importance of the microstructure required for high-performance superconducting cavities with reduced losses caused by magnetic flux trapping.more » « less
-
null (Ed.)Ribonucleoside monophosphate (rNMP) incorporation in DNA is a natural and prominent phenomenon resulting in DNA structural change and genome instability. While DNA polymerases have different rNMP incorporation rates, little is known whether these enzymes incorporate rNMPs following specific sequence patterns. In this study, we analyzed a series of rNMP incorporation datasets, generated from three rNMP mapping techniques, and obtained from Saccharomyces cerevisiae cells expressing wild-type or mutant replicative DNA polymerase and ribonuclease H2 genes. We performed computational analyses of rNMP sites around early and late firing autonomously replicating sequences (ARS’s) of the yeast genome, from which bidirectional, leading and lagging DNA synthesis starts. We find the preference of rNMP incorporation on the leading strand in wild-type DNA polymerase yeast cells. The leading/lagging-strand ratio of rNMP incorporation changes dramatically within 500 nt from ARS’s, highlighting the Pol δ - Pol ε handoff during early leading-strand synthesis. Furthermore, the pattern of rNMP incorporation is markedly distinct between the leading the lagging strand. Overall, our results show the different counts and patterns of rNMP incorporation during DNA replication from ARS, which reflects the different labor of division and rNMP incorporation pattern of Pol δ and Pol ε.more » « less
-
Ribonucleoside monophosphate (rNMP) incorporation in DNA is a natural and prominent phenomenon resulting in DNA structural change and genome instability. While DNA polymerases have different rNMP incorporation rates, little is known whether these enzymes incorporate rNMPs following specific sequence patterns. In this study, we analyzed a series of rNMP incorporation datasets, generated from three rNMP mapping techniques, and obtained from Saccharomyces cerevisiae cells expressing wild-type or mutant replicative DNA polymerase and ribonuclease H2 genes. We performed computational analyses of rNMP sites around early and late firing autonomously replicating sequences (ARS’s) of the yeast genome, from which bidirectional, leading and lagging DNA synthesis starts. We find the preference of rNMP incorporation on the leading strand in wild-type DNA polymerase yeast cells. The leading/lagging-strand ratio of rNMP incorporation changes dramatically within 500 nt from ARS’s, highlighting the Pol δ - Pol ε handoff during early leading-strand synthesis. Furthermore, the pattern of rNMP incorporation is markedly distinct between the leading the lagging strand. Overall, our results show the different counts and patterns of rNMP incorporation during DNA replication from ARS, which reflects the different labor of division and rNMP incorporation pattern of Pol δ and Pol ε.more » « less
An official website of the United States government

Full Text Available