Search for: All records

Large language models (LLMs) have achieved remarkable performance on various natural language tasks. However, they are trained on static corpora and their knowledge can become outdated quickly in the fast-changing world. This motivates the development of knowledge editing (KE) to update specific knowledge in LLMs without changing unrelated others or compromising their pre-trained capabilities. Previous efforts sought to update a small amount of parameters of a LLM and proved effective for making selective updates. Nonetheless, the edited LLM often exhibits degraded ability to reason about the new knowledge. In this work, we identify a key issue: \textit{heterogeneous token overfitting} (HTO), where the LLM overfits different tokens in the provided knowledge at varying rates. To tackle this, we propose {\NAME}, a token-level smoothing method that mitigates HTO by adaptively refining the target distribution. Theoretically, {\NAME} offers better parameter updates with negligible computation overhead. It also induces an implicit DPO but does not require preference data pairs. Extensive experiments across four editing methods, two LLMs, and diverse scenarios demonstrate the effectiveness and versatility of our method. 

Abstract This paper is a collaborative effort that originated at the International Space Science Institute Workshop on “Physical links between Weather and Climate in Space and the Lower Atmosphere” held 22–26 January 2024. Many scientists attended that workshop and contributed their expertise related to polar vortex impacts on upper atmosphere variability. This paper summarizes well-known and newly reported signatures of polar vortex weakening on mesosphere–lower-thermosphere (MLT) temperature, winds, composition, planetary waves, gravity waves, tides, and ionospheric foF2. A variety of observational and modeling results are shown and are consistent with previously published variations in the dynamical and chemical state of the MLT and ionosphere during weak vortex events. We present Superposed Epoch Analysis (SEA) of upper atmosphere diagnostics and phenomena where day 0 is the onset of major SSWs. We also present SEAs where day 0 is the onset of stratopause warmings followed by elevated stratopause events. Our goal in performing two SEAs is to test the sensitivity of 10 hPa versus 1 hPa winds to predict upper atmosphere variability. Results suggest that zonal winds and the semidiurnal migrating solar tide (SW2) in the MLT are more sensitive to zonal wind reversals at 1 hPa rather than 10 hPa. Alternatively, the non-migrating DW2 tide in the equatorial upper mesosphere is best predicted by planetary wave-1 amplitudes in the winter high-latitude upper stratosphere rather than zonal wind reversals. A notable aspect of both SEAs is extremely large event-to-event variability in all diagnostics. Thus, conclusions drawn based on any one event are less robust than those based on many events.