An official website of the United States government
Future sea-level change is characterized by both quantifiable and unquantifiable uncertainties. Effective communication of both types of uncertainty is a key challenge in translating sea-level science to inform long-term coastal planning. Scientific assessments play a key role in the translation process and have taken diverse approaches to communicating sea-level projection uncertainty. Here we review how past IPCC and regional assessments have presented sea-level projection uncertainty, how IPCC presentations have been interpreted by regional assessments and how regional assessments and policy guidance simplify projections for practical use. This information influenced the IPCC Sixth Assessment Report presentation of quantifiable and unquantifiable uncertainty, with the goal of preserving both elements as projections are adapted for regional application.
more »
« less
Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications
Certainty and uncertainty are fundamental to science communication. Hedges have widely been used as proxies for uncertainty. However, certainty is a complex construct, with authors expressing not only the degree but the type and aspects of uncertainty in order to give the reader a certain impression of what is known. Here, we introduce a new study of certainty that models both the level and the aspects of certainty in scientific findings. Using a new dataset of 2167 annotated scientific findings, we demonstrate that hedges alone account for only a partial explanation of certainty. We show that both the overall certainty and individual aspects can be predicted with pre-trained language models, providing a more complete picture of the author’s intended communication. Downstream analyses on 431K scientific findings from news and scientific abstracts demonstrate that modeling sentence-level and aspect-level certainty is meaningful for areas like science communication. Both the model and datasets used in this paper are released at https://blablablab.si.umich.edu/projects/certainty/.
more »
« less
- Award ID(s):
- 1850221
- PAR ID:
- 10336737
- Date Published:
- Journal Name:
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Page Range / eLocation ID:
- 9959 to 10011
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Future sea-level rise projections are characterized by both quantifiable uncertainty and unquantifiable structural uncertainty. Thorough scientific assessment of sea-level rise projections requires analysis of both dimensions of uncertainty. Probabilistic sea-level rise projections evaluate the quantifiable dimension of uncertainty; comparison of alternative probabilistic methods provides an indication of structural uncertainty. Here we describe the Framework for Assessing Changes To Sea-level (FACTS), a modular platform for characterizing different probability distributions for the drivers of sea-level change and their consequences for global mean, regional, and extreme sea-level change. We demonstrate its application by generating seven alternative probability distributions under multiple emissions scenarios for both future global mean sea-level change and future relative and extreme sea-level change at New York City. These distributions, closely aligned with those presented in the Intergovernmental Panel on Climate Change Sixth Assessment Report, emphasize the role of the Antarctic and Greenland ice sheets as drivers of structural uncertainty in sea-level change projections.more » « less
-
*Uncertainty expressions* such as ‘probably’ or ‘highly unlikely’ are pervasive in human language. While prior work has established that there is population-level agreement in terms of how humans quantitatively interpret these expressions, there has been little inquiry into the abilities of language models in the same context. In this paper, we investigate how language models map linguistic expressions of uncertainty to numerical responses. Our approach assesses whether language models can employ theory of mind in this setting: understanding the uncertainty of another agent about a particular statement, independently of the model’s own certainty about that statement. We find that 7 out of 10 models are able to map uncertainty expressions to probabilistic responses in a human-like manner. However, we observe systematically different behavior depending on whether a statement is actually true or false. This sensitivity indicates that language models are substantially more susceptible to bias based on their prior knowledge (as compared to humans). These findings raise important questions and have broad implications for human-AI and AI-AI communication.more » « less
-
Large Language Models (LLMs) are reshaping many aspects of materials science and chemistry research, enabling advances in molecular property prediction, materials design, scientific automation, knowledge extraction, and more. Recent developments demonstrate that the latest class of models are able to integrate structured and unstructured data, assist in hypothesis generation, and streamline research workflows. To explore the frontier of LLM capabilities across the research lifecycle, we review applications of LLMs through 34 total projects developed during the second annual Large Language Model Hackathon for Applications in Materials Science and Chemistry, a global hybrid event. These projects spanned seven key research areas: (1) molecular and material property prediction, (2) molecular and material design, (3) automation and novel interfaces, (4) scientific communication and education, (5) research data management and automation, (6) hypothesis generation and evaluation, and (7) knowledge extraction and reasoning from the scientific literature. Collectively, these applications illustrate how LLMs serve as versatile predictive models, platforms for rapid prototyping of domain-specific tools, and much more. In particular, improvements in both open source and proprietary LLM performance through the addition of reasoning, additional training data, and new techniques have expanded effectiveness, particularly in low-data environments and interdisciplinary research. As LLMs continue to improve, their integration into scientific workflows presents both new opportunities and new challenges, requiring ongoing exploration, continued refinement, and further research to address reliability, interpretability, and reproducibility.more » « less
-
Abstract Large language models (LLMs) are reshaping many aspects of materials science and chemistry research, enabling advances in molecular property prediction, materials design, scientific automation, knowledge extraction, and more. Recent developments demonstrate that the latest class of models are able to integrate structured and unstructured data, assist in hypothesis generation, and streamline research workflows. To explore the frontier of LLM capabilities across the research lifecycle, we review applications of LLMs through 32 total projects developed during the second annual LLM hackathon for applications in materials science and chemistry, a global hybrid event. These projects spanned seven key research areas: (1) molecular and material property prediction, (2) molecular and material design, (3) automation and novel interfaces, (4) scientific communication and education, (5) research data management and automation, (6) hypothesis generation and evaluation, and (7) knowledge extraction and reasoning from the scientific literature. Collectively, these applications illustrate how LLMs serve as versatile predictive models, platforms for rapid prototyping of domain-specific tools, and much more. In particular, improvements in both open source and proprietary LLM performance through the addition of reasoning, additional training data, and new techniques have expanded effectiveness, particularly in low-data environments and interdisciplinary research. As LLMs continue to improve, their integration into scientific workflows presents both new opportunities and new challenges, requiring ongoing exploration, continued refinement, and further research to address reliability, interpretability, and reproducibility.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18653/v1/2021.emnlp-main.784
