Search for: All records

Abstract Artificial intelligence (AI) and machine learning (ML) pose a challenge for achieving science that is both reproducible and replicable. The challenge is compounded in supervised models that depend on manually labeled training data, as they introduce additional decision‐making and processes that require thorough documentation and reporting. We address these limitations by providing an approach to hand labeling training data for supervised ML that integrates quantitative content analysis (QCA)—a method from social science research. The QCA approach provides a rigorous and well‐documented hand labeling procedure to improve the replicability and reproducibility of supervised ML applications in Earth systems science (ESS), as well as the ability to evaluate them. Specifically, the approach requires (a) the articulation and documentation of the exact decision‐making process used for assigning hand labels in a “codebook” and (b) an empirical evaluation of the reliability” of the hand labelers. In this paper, we outline the contributions of QCA to the field, along with an overview of the general approach. We then provide a case study to further demonstrate how this framework has and can be applied when developing supervised ML models for applications in ESS. With this approach, we provide an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS. 

Abstract During near-0°C surface conditions, diverse precipitation types (p-types) are possible, including rain, drizzle, freezing rain, freezing drizzle, ice pellets, wet snow, snow, and snow pellets. Near-0°C precipitation affects wide swaths of the United States and Canada, impacting aviation, road transportation, power generation and distribution, winter recreation, ecology, and hydrology. Fundamental challenges remain in observing, diagnosing, simulating, and forecasting near-0°C p-types, particularly during transitions and within complex terrain. Motivated by these challenges, the field phase of the Winter Precipitation Type Research Multi-scale Experiment (WINTRE-MIX) was conducted from 1 February – 15 March 2022 to better understand how multiscale processes influence the variability and predictability of p-type and amount under near-0°C surface conditions. WINTRE-MIX took place near the US / Canadian border, in northern New York and southern Quebec, a region with plentiful near-0°C precipitation influenced by terrain. During WINTRE-MIX, existing advanced mesonets in New York and Quebec were complemented by deployment of: (1) surface instruments, (2) the National Research Council Convair-580 research aircraft with W- and X-band Doppler radars and in situ cloud and aerosol instrumentation, (3) two X-band dual-polarization Doppler radars and a C-band dual-polarization Doppler radar from University of Illinois, and (4) teams collecting manual hydrometeor observations and radiosonde measurements. Eleven intensive observing periods (IOPs) were coordinated. Analysis of these WINTRE-MIX IOPs is illuminating how synoptic dynamics, mesoscale dynamics, and microscale processes combine to determine p-type and its predictability under near-0°C conditions. WINTRE-MIX research will contribute to improving nowcasts and forecasts of near-0°C precipitation through evaluation and refinement of observational diagnostics and numerical forecast models. 

Abstract Global Forecast System (GFS), North American Mesoscale Forecast System (NAM), and High-Resolution Rapid Refresh (HRRR) 2-m temperature, 10-m wind speed, and precipitation accumulation forecasts initialized at 1200 UTC are verified against New York State Mesonet (NYSM) observations from 1 January 2018 through 31 December 2021. NYSM observations at 126 site locations are used to calculate standard error statistics (e.g., forecast error, root-mean-square error) for temperature and wind speed and contingency table statistics for precipitation across forecast hours, meteorological seasons, and regions. The majority of the focus is placed on the first 18 forecast hours to allow for comparison among all three models. A daily NYSM station-mean temperature error analysis identified a slight cold bias at temperatures below 25°C in the GFS, a cool-to-warm bias as forecast temperatures warm in the HRRR, and a warm bias at temperatures above 30°C in each model. Differences arise when considering temperature biases with respect to lead times and seasons. Wind speeds are overforecast at all ranges in each season, and forecast wind speeds ≥ 18 m s⁻¹are rarely observed. Performance diagrams indicate overall good forecast performance at precipitation thresholds of 0.1–1.5 mm, but with a high frequency bias in the GFS and NAM. This paper provides an overview of deterministic forecast performance across New York State, with the aim of sharing common biases associated with temperature, wind speed, and precipitation with operational forecasters and is the first step in developing a real-time model forecast uncertainty prediction tool.