An official website of the United States government
Abstract An ensemble postprocessing method is developed to improve the probabilistic forecasts of extreme precipitation events across the conterminous United States (CONUS). The method combines a 3D vision transformer (ViT) for bias correction with a latent diffusion model (LDM), a generative artificial intelligence (AI) method, to postprocess 6-hourly precipitation ensemble forecasts and produce an enlarged generative ensemble that contains spatiotemporally consistent precipitation trajectories. These trajectories are expected to improve the characterization of extreme precipitation events and offer skillful multiday accumulated and 6-hourly precipitation guidance. The method is tested using the Global Ensemble Forecast System (GEFS) precipitation forecasts out to day 6 and is verified against the Climatology-Calibrated Precipitation Analysis (CCPA) data. Verification results indicate that the method generated skillful ensemble members with improved continuous ranked probabilistic skill scores (CRPSSs) and Brier skill scores (BSSs) over the raw operational GEFS and a multivariate statistical postprocessing baseline. It showed skillful and reliable probabilities for events at extreme precipitation thresholds. Explainability studies were further conducted, which revealed the decision-making process of the method and confirmed its effectiveness on ensemble member generation. This work introduces a novel, generative AI–based approach to address the limitation of small numerical ensembles and the need for larger ensembles to identify extreme precipitation events. Significance StatementWe use a new artificial intelligence (AI) technique to improve extreme precipitation forecasts from a numerical weather prediction ensemble, generating more scenarios that better characterize extreme precipitation events. This AI-generated ensemble improved the accuracy of precipitation forecasts and probabilistic warnings for extreme precipitation events. The study explores AI methods to generate precipitation forecasts and explains the decision-making mechanisms of such AI techniques to prove their effectiveness.
more »
« less
Applying Generative Artificial Intelligence to cognitive models of decision making
IntroductionGenerative Artificial Intelligence has made significant impacts in many fields, including computational cognitive modeling of decision making, although these applications have not yet been theoretically related to each other. This work introduces a categorization of applications of Generative Artificial Intelligence to cognitive models of decision making. MethodsThis categorization is used to compare the existing literature and to provide insight into the design of an ablation study to evaluate our proposed model in three experimental paradigms. These experiments used for model comparison involve modeling human learning and decision making based on both visual information and natural language, in tasks that vary in realism and complexity. This comparison of applications takes as its basis Instance-Based Learning Theory, a theory of experiential decision making from which many models have emerged and been applied to a variety of domains and applications. ResultsThe best performing model from the ablation we performed used a generative model to both create memory representations as well as predict participant actions. The results of this comparison demonstrates the importance of generative models in both forming memories and predicting actions in decision-modeling research. DiscussionIn this work, we present a model that integrates generative and cognitive models, using a variety of stimuli, applications, and training methods. These results can provide guidelines for cognitive modelers and decision making researchers interested in integrating Generative AI into their methods.
more »
« less
- Award ID(s):
- 2229881
- PAR ID:
- 10699340
- Publisher / Repository:
- Frontiers in psychology
- Date Published:
- Journal Name:
- Frontiers in Psychology
- Volume:
- 15
- ISSN:
- 1664-1078
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract BackgroundInfluenza has a significant impact on morbidity and mortality, with disproportionate impact on non-White populations. Forecasts of flu timing/intensity have the potential to reduce morbidity, mortality, and health disparities by supporting decision-making by public health officials and clinicians. However, uptake and use of forecasts on local levels is low, with limited communication between model developers and end-users. The goals of this study are to: 1) understand the seasonal flu intervention decision-making process from the perspective of local public health officials and health care providers; and 2) identify these stakeholders’ data needs and priorities for flu forecasting models. MethodsThis mixed methods study included a brief survey and two rounds of focus groups with local public health officials and clinicians in a mid-sized metropolitan area in the Northeast US (N=16). Authors used descriptive statistics to analyze survey responses and content analysis to analyze qualitative data. ResultsParticipants described a decision-making process that included using data from forecast models and other sources to inform health interventions, health communication, and resource allocation. Primary outcomes for decision-making included disease prevention and health care preparedness. Participants articulated a variety of ways that forecasting models could assist them in delivering evidence-informed public health and clinical services, and data needs including sociodemographic characteristics and surveillance information at smaller spatial scales. There was a desire for functionality of models that reduced the time spent gathering information. ConclusionsFindings support the need for a participatory modeling approach to the design of influenza forecasts that improves uptake by including the goals and desires of end-users.more » « less
-
Abstract Artificial intelligence (AI) can be used to improve performance across a wide range of Earth system prediction tasks. As with any application of AI, it is important for AI to be developed in an ethical and responsible manner to minimize bias and other effects. In this work, we extend our previous work demonstrating how AI can go wrong with weather and climate applications by presenting a categorization of bias for AI in the Earth sciences. This categorization can assist AI developers to identify potential biases that can affect their model throughout the AI development life cycle. We highlight examples from a variety of Earth system prediction tasks of each category of bias.more » « less
-
Abstract BackgroundHigh‐quality feedback is crucial for academic success, driving student motivation and engagement while research explores effective delivery and student interactions. Advances in artificial intelligence (AI), particularly natural language processing (NLP), offer innovative methods for analyzing complex qualitative data such as feedback interactions. PurposeWe developed a framework to train sentence transformers using generative AI–created synthetic data to categorize student‐feedback interactions in engineering studios. We compared traditional thematic analysis with modern methods to evaluate the realism of synthetic datasets and their effectiveness in training NLP models by exploring how generative AI can aid qualitative coding. MethodsWe deidentified and transcribed eight audio recordings from engineering studios. Synthetic feedback transcripts were generated using three locally hosted large language models: Llama 3.1, Gemma 2.0, and Mistral NeMo, adjusting parameters to produce datasets mimicking the real transcripts. We assessed the quality of synthetic transcripts using our framework and used a sentence transformer model (trained on both real and synthetic data) to compare changes in the model's percent accuracy when qualitatively coding feedback interactions. ResultsSynthetic data improved the NLP model's performance in classifying feedback interactions, boosting the average accuracy from 68.4% to 81% with Llama 3.1. Although incorporating synthetic data improved classification, all models produced transcripts that occasionally included extraneous details and failed to capture instructor‐dominant discourse. ConclusionsSynthetic data offers an opportunity to expand qualitative research, particularly in contexts where real data for NLP training is limited or hard to obtain; however, transparency in its use is paramount to maintain research integrity.more » « less
-
Abstract Collective intelligence (CI) is the capacity of groups to outperform individuals in tasks such as decision-making, coordination and problem-solving. A significant challenge to studying CI is that it encompasses a wide variety of behaviours that are underpinned by different mechanisms. To balance inclusivity with conceptual clarity, we propose a typology of CI mechanisms including self-organized coordination, distributed decision-making, cooperative problem-solving, statistical aggregation and culture and cultural improvement. This typology helps clarify the minimum requirements for CI and how other, more cognitive complex mechanisms may have layered on top of these. As model comparative species, we focus on two ecologically successful generalists, rats (Rattus) and cockroaches (Blattodea). Cockroaches show evidence of collective behaviours, but these rely on a more ancestral set of mechanisms that lack cognitive complexity. Conversely, rats share important ecological traits with cockroaches but exhibit social traits found in primates, including cultural learning, distributed decision-making, empathy and cooperation. This comparison can show how similar collective outcomes can emerge even with disparate mechanisms and provide a way to uncover evolutionary design principles of CI. Ultimately, we can use such comparisons to determine which features of CI are universal properties of collectives and which are contingent on specific cognitive adaptations. This article is part of the theme issue ‘The evolution of collective intelligence’.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.3389/fpsyg.2024.1387948
