More Like this

1. Text as Policy: Measuring Policy Similarity through Bill Text Reuse

   https://doi.org/10.1111/psj.12257  

   Linder, Fridolin; Desmarais, Bruce; Burgess, Matthew; Giraudy, Eugenia (October 2018, Policy Studies Journal)
2. Controlled Text Generation as Continuous Optimization with Multiple Constraints

   Kumar, Sachin; Malmi, Eric; Severyn, Aliaksei; Tsvetkov, Yulia (October 2022, Conference on Neural Information Processing Systems (NeurIPS))
3. ClusterLLM: Large Language Models as a Guide for Text Clustering

   https://doi.org/10.18653/v1/2023.emnlp-main.858  

   Zhang, Yuwei; Wang, Zihan; Shang, Jingbo (January 2023, ACL SIGDAT Empirical Methods in Natural Language Processing (EMNLP) 2023)
4. BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers

   https://doi.org/10.18653/v1/2024.emnlp-main.1241  

   Xu, Ran; Shi, Wenqi; Yu, Yue; Zhuang, Yuchen; Zhu, Yanqiao; Wang, May Dongmei; Ho, Joyce C; Zhang, Chao; Yang, Carl (January 2024, Association for Computational Linguistics)
5. Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

   Gruver, N; Sriram, A; Madotto, A; Wilson, AG; Zitnick, LC; Ulissi, Z (May 2024, International Conference on Learning Representations)

   We propose fine-tuning large language models for generation of stable materials. While unorthodox, fine-tuning large language models on text-encoded atomistic data is simple to implement yet reliable, with around 90% of sampled structures obeying physical constraints on atom positions and charges. Using energy above hull calculations from both learned ML potentials and gold-standard DFT calculations, we show that our strongest model (fine-tuned LLaMA-2 70B) can generate materials predicted to be metastable at about twice the rate (49% vs 28%) of CDVAE, a competing diffusion model. Because of text prompting's inherent flexibility, our models can simultaneously be used for unconditional generation of stable material, infilling of partial structures and text-conditional generation. Finally, we show that language models' ability to capture key symmetries of crystal structures improves with model scale, suggesting that the biases of pretrained LLMs are surprisingly well-suited for atomistic data. 

   more » « less