Search for: All records

Preferences within a group of people are not uniform but follow a distribution. While existing alignment methods like Direct Preference Optimization (DPO) attempt to steer models to reflect human preferences, they struggle to capture the distributional pluralistic preferences within a group. These methods often skew toward dominant preferences, overlooking the diversity of opinions, especially when conflicting preferences arise. To address this issue, we propose Group Distributional Preference Optimization (GDPO), a novel framework that aligns language models with the distribution of preferences within a group by incorporating the concept of beliefs that shape individual preferences. GDPO calibrates a language model using statistical estimation of the group's belief distribution and aligns the model with belief-conditioned preferences, offering a more inclusive alignment framework than traditional methods. In experiments using both synthetic controllable opinion generation and real-world movie review datasets, we show that DPO fails to align with the targeted belief distributions, while GDPO consistently reduces this alignment gap during training. Additionally, our evaluation metrics demonstrate that GDPO outperforms existing approaches in aligning with group distributional preferences, marking a significant advance in pluralistic alignment. 

Abstract Measurements of oxygen and hydrogen stable isotope ratios (δ¹⁸O and δD) in meteoric waters provide insight to overlapping effects of evaporation, precipitation, and mixing on basin scale hydrology. This study of waters collected between 2016 and 2021 in the Turkana Basin, northern Kenya, uses δ¹⁸O and δD to understand water balance in Lake Turkana, a large, low‐latitude, alkaline desert lake. The Omo River, a major river system in the Ethiopian Highlands, is historically understood to provide approximately 90% of the water input to Lake Turkana. Discharge of the Omo is prohibitively difficult to measure, but stable isotope ratios in the lake may provide a meaningful method for monitoring the lake's response to changes in input. Precipitation in the Turkana Basin is low (<200 mm/year) with negligible rainfall on the lake's surface, and all water loss from the lake is evaporative. We compare new measurements with previous data from the region and records of lake height and precipitation from the same time period. We show that a Bayesian approach to modeling evaporation using atmospheric conditions and river δ¹⁸O and δD yields results consistent with published water balance models. Continued sampling of lake and meteoric waters in the Turkana Basin will be a useful way to monitor the lake's response to regional and global climate change.