More Like this

1. Accelerating Community-Wide Evaluation of AI Models for Global Weather Prediction by Facilitating Access to Model Output

   https://doi.org/10.1175/BAMS-D-24-0057.1  

   Radford, Jacob T; Ebert-Uphoff, Imme; Stewart, Jebb Q; Musgrave, Kate D; DeMaria, Robert; Tourville, Natalie; Hilburn, Kyle (January 2025, Bulletin of the American Meteorological Society)

   Abstract Numerous artificial intelligence-based weather prediction (AIWP) models have emerged over the past 2 years, mostly in the private sector. There is an urgent need to evaluate these models from a meteorological perspective, but access to the output of these models is limited. We detail two new resources to facilitate access to AIWP model output data in the hope of accelerating the investigation of AIWP models by the meteorological community. First, a 3-yr (and growing) reforecast archive beginning in October 2020 containing twice daily 10-day forecasts forFourCastNet v2-small,Pangu-Weather, andGraphCast Operationalis now available via an Amazon Simple Storage Service (S3) bucket through NOAA’s Open Data Dissemination (NODD) program (https://noaa-oar-mlwp-data.s3.amazonaws.com/index.html). This reforecast archive was initialized with both the NOAA’s Global Forecast System (GFS) and ECMWF’s Integrated Forecasting System (IFS) initial conditions in the hope that users can begin to perform the feature-based verification of impactful meteorological phenomena. Second, real-time output for these three models is visualized on our web page (https://aiweather.cira.colostate.edu) along with output from the GFS and the IFS. This allows users to easily compare output between each AIWP model and traditional, physics-based models with the goal of familiarizing users with the characteristics of AIWP models and determine whether the output aligns with expectations, is physically consistent and reasonable, and/or is trustworthy. We view these two efforts as a first step toward evaluating whether these new AIWP tools have a place in forecast operations. 

   more » « less
2. Creating a biomedical knowledge base by addressing GPT inaccurate responses and benchmarking context

   https://doi.org/10.1101/2024.10.16.618663  

   Darnell, S Solomon; Overall, Rupert W; Guarracino, Andrea; Colonna, Vicenza; Villani, Flavia; Garrison, Erik; Isaac, Arun; Muli, Priscilla; Muriithi, Frederick Muriuki; Kabui, Alexander; et al (October 2024, bioRxiv)

   We created GNQA, a generative pre-trained transformer (GPT) knowledge base driven by a performant retrieval augmented generation (RAG) with a focus on aging, dementia, Alzheimer’s and diabetes. We uploaded a corpus of three thousand peer reviewed publications on these topics into the RAG. To address concerns about inaccurate responses and GPT ‘hallucinations’, we implemented a context provenance tracking mechanism that enables researchers to validate responses against the original material and to get references to the original papers. To assess the effectiveness of contextual information we collected evaluations and feedback from both domain expert users and ‘citizen scientists’ on the relevance of GPT responses. A key innovation of our study is automated evaluation by way of a RAG assessment system (RAGAS). RAGAS combines human expert assessment with AI-driven evaluation to measure the effectiveness of RAG systems. When evaluating the responses to their questions, human respondents give a “thumbs-up” 76% of the time. Meanwhile, RAGAS scores 90% on answer relevance on questions posed by experts. And when GPT-generates questions, RAGAS scores 74% on answer relevance. With RAGAS we created a benchmark that can be used to continuously assess the performance of our knowledge base. Full GNQA functionality is embedded in the freeGeneNetwork.orgweb service, an open-source system containing over 25 years of experimental data on model organisms and human. The code developed for this study is published under a free and open-source software license athttps://git.genenetwork.org/gn-ai/tree/README.md. 

   more » « less
3. SMART Cables Observing the Oceans and Earth

   https://doi.org/10.4031/MTSJ.56.5.3  

   Rowe, Charlotte A.; Howe, Bruce M.; Fouch, Matthew J.; Angove, Michael; Aucan, Jerome; Barnes, Christopher R.; Bayliff, Nigel; Becker, Nathan C.; Carrilho, Fernando; Fry, Bill; et al (October 2022, Marine Technology Society Journal)

   Abstract The Joint Task Force, Science Monitoring And Reliable Telecommunications (SMART) Subsea Cables is working to integrate environmental sensors (temperature, pressure, seismic acceleration) into submarine telecommunications cables. This will support climate and ocean observation, sea-level monitoring, observations of Earth structure, tsunami and earthquake early warning, and disaster risk reduction. Recent advances include regional SMART pilot systems that are the initial steps to trans-ocean and global implementation. Building on the OceanObs'19conference and community white paper (https://doi.org/10.3389/fmars.2019.00424), this paper presents an overview of the initiative and a description of ongoing projects including: InSea wet demonstration project off Sicily; Vanuatu and New Caledonia; Indonesia; CAM-2 ring system connecting the Portuguese mainland, Azores, and Madeira; New Zealand; and Antarctica. In addition to the diverse scientific and societal benefits, the telecommunications industry's mission of societal connectivity will also benefit because environmental awareness improves both individual cable system integrity and the resilience of the overall global communications network. 

   more » « less
4. On the Annual Asymmetry of High‐Latitude Sporadic F

   https://doi.org/10.1029/2019SW002305  

   Chartier, Alex T.; Huba, J. D.; Mitchell, Cathryn N. (November 2019, Space Weather)

   Abstract The polarFregion ionosphere frequently exhibits sporadic variability (e.g., Meek, 1949,https://doi.org/10.1029/JZ054i004p00339; Hill, 1963,https://doi.org/10.1175/1520‐0469(1963)020<0492:SEOLII>2.0.CO;2). Recent satellite data analysis (Noja et al., 2013,https://doi.org/10.1002/rds.20033; Chartier et al., 2018,https://doi.org/10.1002/2017JA024811) showed that the high‐latitudeFregion ionosphere exhibits sporadic enhancements more frequently in January than in July in both the northern and southern hemispheres. The same pattern has been seen in statistics of the degradation and total loss of GPS service onboard low‐Earth orbit satellites (Xiong et al. 2018,https://doi.org/10.5194/angeo‐36‐679‐2018). Here, we confirm the existence of this annual pattern using ground GPS‐based images of TEC from the MIDAS algorithm. Images covering January and July 2014 confirm that the high‐latitude (>70 MLAT)Fregion exhibits a substantially larger range of values in January than in July in both the northern and southern hemispheres. The range of TEC values observed in the polar caps is 38–57 TECU (north‐south) in January versus 25–37 TECU in July. First‐principle modeling using SAMI3 reproduces this pattern, and indicates that it is caused by an asymmetry in plasma levels (30% higher in January than in July across both polar caps), as well as 17% longer O⁺plasma lifetimes in northern hemisphere winter, compared to southern hemisphere winter. 

   more » « less
5. Peering into the Black Box: Forward Modeling of the Uncertainty Budget of High-resolution Spectroscopy of Exoplanet Atmospheres

   https://doi.org/10.3847/1538-3881/ada27e  

   Savel, Arjun B; Bedell, Megan; Kempton, Eliza M-R; Smith, Peter_C B; Bean, Jacob L; Zhao, Lily L; Wong, Kaze_W K; Sanchez, Jorge A; Line, Michael R (February 2025, The Astronomical Journal)

   Abstract Ground-based high-resolution cross-correlation spectroscopy (HRCCS;R ≳ 15,000) is a powerful complement to space-based studies of exoplanet atmospheres. By resolving individual spectral lines, HRCCS can precisely measure chemical abundance ratios, directly constrain atmospheric dynamics, and robustly probe multidimensional physics. But the subtleties of HRCCS data sets—e.g., the lack of exoplanetary spectra visible by eye and the statistically complex process of telluric removal—can make interpreting them difficult. In this work, we seek to clarify the uncertainty budget of HRCCS with a forward-modeling approach. We present an HRCCS observation simulator,scope,⁵⁵https://github.com/arjunsavel/scopethat incorporates spectral contributions from the exoplanet, star, tellurics, and instrument. This tool allows us to control the underlying data set, enabling controlled experimentation with complex HRCCS methods. Simulating a fiducial hot Jupiter data set (WASP-77Ab emission with IGRINS), we first confirm via multiple tests that the commonly used principal component analysis does not bias the planetary signal when few components are used. Furthermore, we demonstrate that mildly varying tellurics and moderate wavelength solution errors induce only mild decreases in HRCCS detection significance. However, limiting-case, strongly varying tellurics can bias the retrieved velocities and gas abundances. Additionally, in the low signal-to-noise ratio limit, constraints on gas abundances become highly non-Gaussian. Our investigation of the uncertainties and potential biases inherent in HRCCS data analysis enables greater confidence in scientific results from this maturing method. 

   more » « less