An official website of the United States government
Abstract Pure artificial intelligence (AI)-based weather prediction (AIWP) models have made waves within the scientific community and the media, claiming superior performance to numerical weather prediction (NWP) models. However, these models often lack impactful output variables such as precipitation. One exception is Google DeepMind’s GraphCast model, which became the first mainstream AIWP model to predict precipitation, but performed only limited verification. We present an analysis of the ECMWF’s Integrated Forecasting System (IFS)-initialized (GRAPIFS) and the NCEP’s Global Forecast System (GFS)-initialized (GRAPGFS) GraphCast precipitation forecasts over the contiguous United States and compare to results from the GFS and IFS models using 1) grid-based, 2) neighborhood, and 3) object-oriented metrics verified against the fifth major global reanalysis produced by ECMWF (ERA5) and the NCEP/Environmental Modeling Center (EMC) stage IV precipitation analysis datasets. We affirmed that GRAPGFSand GRAPIFSperform better than the GFS and IFS in terms of root-mean-square error and stable equitable errors in probability space, but the GFS and IFS precipitation distributions more closely align with the ERA5 and stage IV distributions. Equitable threat score also generally favored GraphCast, particularly for lower accumulation thresholds. Fractions skill score for increasing neighborhood sizes shows greater gains for the GFS and IFS than GraphCast, suggesting the NWP models may have a better handle on intensity but struggle with the location. Object-based verification for GraphCast found positive area biases at low accumulation thresholds and large negative biases at high accumulation thresholds. GRAPGFSsaw similar performance gains to GRAPIFSwhen compared to their NWP counterparts, but initializing with the less familiar GFS conditions appeared to lead to an increase in light precipitation. Significance StatementPure artificial intelligence (AI)-based weather prediction (AIWP) has exploded in popularity with promises of better performance and faster run times than numerical weather prediction (NWP) models. However, less attention has been paid to their capability to predict impactful, sensible weather like precipitation, precipitation type, or specific meteorological features. We seek to address this gap by comparing precipitation forecast performance by an AI model called GraphCast to the Global Forecast System (GFS) and the Integrated Forecasting System (IFS) NWP models. While GraphCast does perform better on many verification metrics, it has some limitations for intense precipitation forecasts. In particular, it less frequently predicts intense precipitation events than the GFS or IFS. Overall, this article emphasizes the promise of AIWP while at the same time stresses the need for robust verification by domain experts.
more »
« less
This content will become publicly available on January 1, 2026
Accelerating Community-Wide Evaluation of AI Models for Global Weather Prediction by Facilitating Access to Model Output
Abstract Numerous artificial intelligence-based weather prediction (AIWP) models have emerged over the past 2 years, mostly in the private sector. There is an urgent need to evaluate these models from a meteorological perspective, but access to the output of these models is limited. We detail two new resources to facilitate access to AIWP model output data in the hope of accelerating the investigation of AIWP models by the meteorological community. First, a 3-yr (and growing) reforecast archive beginning in October 2020 containing twice daily 10-day forecasts forFourCastNet v2-small,Pangu-Weather, andGraphCast Operationalis now available via an Amazon Simple Storage Service (S3) bucket through NOAA’s Open Data Dissemination (NODD) program (https://noaa-oar-mlwp-data.s3.amazonaws.com/index.html). This reforecast archive was initialized with both the NOAA’s Global Forecast System (GFS) and ECMWF’s Integrated Forecasting System (IFS) initial conditions in the hope that users can begin to perform the feature-based verification of impactful meteorological phenomena. Second, real-time output for these three models is visualized on our web page (https://aiweather.cira.colostate.edu) along with output from the GFS and the IFS. This allows users to easily compare output between each AIWP model and traditional, physics-based models with the goal of familiarizing users with the characteristics of AIWP models and determine whether the output aligns with expectations, is physically consistent and reasonable, and/or is trustworthy. We view these two efforts as a first step toward evaluating whether these new AIWP tools have a place in forecast operations.
more »
« less
- Award ID(s):
- 2019758
- PAR ID:
- 10596341
- Publisher / Repository:
- AMS
- Date Published:
- Journal Name:
- Bulletin of the American Meteorological Society
- Volume:
- 106
- Issue:
- 1
- ISSN:
- 0003-0007
- Page Range / eLocation ID:
- E68 to E76
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract We describe POInTbrowse, a web portal that gives access to the orthology inferences made for polyploid genomes with POInT, the Polyploidy Orthology Inference Tool. Ancient, or paleo-, polyploidy events are widely distributed across the eukaryotic phylogeny, and the combination of duplicated and lost duplicated genes that these polyploidies produce can confound the identification of orthologous genes between genomes. POInT uses conserved synteny and phylogenetic models to infer orthologous genes between genomes with a shared polyploidy. It also gives confidence estimates for those orthology inferences. POInTbrowsegives both graphical and query-based access to these inferences from 12 different polyploidy events, allowing users to visualize genomic regions produced by polyploidies and perform batch queries for each polyploidy event, downloading genes trees and coding sequences for orthologous genes meeting user-specified criteria. POInTbrowseand the associated data are online athttps://wgd.statgen.ncsu.edu.more » « less
-
Abstract The DESI Milky Way Survey (MWS) will observe ≥8 million stars between 16 < r < 19 mag, supplemented by observations of brighter targets under poor observing conditions. The survey will permit an accurate determination of stellar kinematics and population gradients, characterize diffuse substructure in the thick disk and stellar halo, enable the discovery of extremely metal-poor stars and other rare stellar types, and improve constraints on the Galaxy’s 3D dark matter distribution from halo star kinematics. MWS will also enable a detailed characterization of the stellar populations within 100 pc of the Sun, including a complete census of white dwarfs. The target catalog from the preliminary selection described here is public (Available at https://data.desi.lbl.gov/public/ets/target/catalogs/and detailed at https://desidatamodel.readthedocs.io).more » « less
-
dadi-cli: Automated and distributed population genetic model inference from allele frequency spectraAbstract Summarydadi is a popular software package for inferring models of demographic history and natural selection from population genomic data. But using dadi requires Python scripting and manual parallelization of optimization jobs. We developed dadi-cli to simplify dadi usage and also enable straighforward distributed computing. Availability and Implementationdadi-cli is implemented in Python and released under the Apache License 2.0. The source code is available athttps://github.com/xin-huang/dadi-cli. dadi-cli can be installed via PyPI and conda, and is also available through Cacao on Jetstream2https://cacao.jetstream-cloud.org/.more » « less
-
Abstract Proleptic (or prothetic) objects (Majaliwa remembered about Samson that he’s sick) present a particular puzzle because they appear to instantiate an unconstrained cross-clausal dependency between the proleptic object (Samson) and a correlate (he). The current analytical approach to prolepsis is relies on a syntactic mechanism of treating the embedded clause as a predicate, derived by merging a null operator which unselectively binds the correlate. This approach faces a number of known empirical challenges. Moreover, this work does not meaningfully engage with any of the recent semantic innovations in our understanding of embedded clauses (Kratzer 2006. Decomposing attitude verbs. Available at:http://semanticsarchive.net/Archive/DcwY2JkM/attitude-verbs2006.pdf). I offer an alternative to the CP-predicate approach, adopting three (semi-)independently motivated ideas concerning (i) the syntax of cross-clausal dependencies in Lohninger et al. (2022. From prolepsis to hyperraising.Philosophies7(32)), (ii) the semantics of embedded clauses in Kratzer (2006. Decomposing attitude verbs. Available at:http://semanticsarchive.net/Archive/DcwY2JkM/attitude-verbs2006.pdf)/Moulton (2009. Not moving clauses: Connectivity in clausal arguments.Syntax16(3). 250–291), and (iii) the semantics ofaboutin Rawlins (2013. About about.Proceedings of SALT23. 336–357)/Onea and Mardale (2020. From topic to object, grammaticalization differential object marking in Romanian.Canadian Journal of Linguistics/Revue Canadienne de Linguistique65(3). 350–392). I argue that proleptic objects are complex-NPs, roughly,the thing about Samson. The somewhat particular properties of prolepsis are natural consequences of these combined ideas. I further present empirical evidence from Japanese that the complex-NP analysis is on the right track. This analysis deepens our understanding of prolepsis by including both semantic and syntactic factors.more » « less
