Search for: All records

Large language models (LLMs) are being increasingly deployed as part of pipelines that repeatedly process or generate data of some sort. However, a common barrier to deployment are the frequent and often unpredictable errors that plague LLMs. Acknowledging the inevitability of these errors, we proposedata quality assertionsto identify when LLMs may be making mistakes. We present spade, a method for automatically synthesizing data quality assertions that identify bad LLM outputs. We make the observation that developers often identify data quality issues during prototyping prior to deployment, and attempt to address them by adding instructions to the LLM prompt over time. spade therefore analyzes histories of prompt versions over time to create candidate assertion functions and then selects a minimal set that fulfills both coverage and accuracy requirements. In testing across nine different real-world LLM pipelines, spade efficiently reduces the number of assertions by 14% and decreases false failures by 21% when compared to simpler baselines. spade has been deployed as an offering within LangSmith, LangChain's LLM pipeline hub, and has been used to generate data quality assertions for over 2000 pipelines across a spectrum of industries. 

The synthesis of heavy elements in supernovae is affected by low-energy $(n,p)$and $(p,n)$reactions on unstable nuclei, yet experimental data on such reaction rates are scarce. The SECAR (SEparator for CApture Reactions) recoil separator at FRIB (Facility for Rare Isotope Beams) was originally designed to measure astrophysical reactions that change the mass of a nucleus significantly. We used a novel approach that integrates machine learning with ion-optical simulations to find an ion-optical solution for the separator that enables the measurement of $(p,n)$reactions, despite the reaction leaving the mass of the nucleus nearly unchanged. A new measurement of the ${}^{58}\mathrm{Fe}(p,n){}^{58}\mathrm{Co}$reaction in inverse kinematics with a $3.66\pm 0.12$MeV/nucleon ${}^{58}\mathrm{Fe}$beam (corresponding to $3.69\pm 0.12$MeV proton energy in normal kinematics) yielded a cross-section of $20.3\pm 6.3$ mb and served as a proof of principle experiment for the new technique demonstrating its effectiveness in achieving the required performance criteria. This novel approach paves the way for studying astrophysically important $(p,n)$reactions on unstable nuclei produced at FRIB. Published by the American Physical Society2025