skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: HydroBench: Jupyter supported reproducible hydrological model benchmarking and diagnostic tool
Evaluating whether hydrological models are right for the right reasons demands reproducible model benchmarking and diagnostics that evaluate not just statistical predictive model performance but also internal processes. Such model benchmarking and diagnostic efforts will benefit from standardized methods and ready-to-use toolkits. Using the Jupyter platform, this work presents HydroBench, a model-agnostic benchmarking tool consisting of three sets of metrics: 1) common statistical predictive measures, 2) hydrological signature-based process metrics, including a new time-linked flow duration curve and 3) information-theoretic diagnostics that measure the flow of information among model variables. As a test case, HydroBench was applied to compare two model products (calibrated and uncalibrated) of the National Hydrologic Model - Precipitation Runoff Modeling System (NHM-PRMS) at the Cedar River watershed, WA, United States. Although the uncalibrated model has the highest predictive performance, particularly for high flows, the signature-based diagnostics showed that the model overestimates low flows and poorly represents the recession processes. Elucidating why low flows may have been overestimated, the information-theoretic diagnostics indicated a higher flow of information from precipitation to snowmelt to streamflow in the uncalibrated model compared to the calibrated model, where information flowed more directly from precipitation to streamflow. This test case demonstrated the capability of HydroBench in process diagnostics and model predictive and functional performance evaluations, along with their tradeoffs. Having such a model benchmarking tool not only provides modelers with a comprehensive model evaluation system but also provides an open-source tool that can further be developed by the hydrological community.  more » « less
Award ID(s):
1928406
PAR ID:
10488044
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
Frontiers Media S.A.
Date Published:
Journal Name:
Frontiers in Earth Science
Volume:
10
ISSN:
2296-6463
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract How precipitation (P) is translated into streamflow (Q) and over what timescales (i.e., “memory”) is difficult to predict without calibration of site‐specific models or using geochemical approaches, posing barriers to prediction in ungauged basins or advancement of general theories. Here, we used a data‐driven approach to identify regional patterns and exogenous controls on P–Q interactions. We applied an information flow analysis, which quantifies uncertainty reduction, to a daily time series of P and Q from 671 watersheds across the conterminous United States. We first demonstrated that information transfer from P to Q primarily reflects the quickflow component of water‐budgets, based on a watershed model. Readily quantifiable information flows show a functional relationship with model parameters, suggesting utility for model calibration. Second, applied to real watersheds, P–Q information flows exhibit seasonally varying behavior within regions in a manner consistent with dominant runoff generation mechanisms. However, the timing and the magnitude of information flows also reflect considerable subregional heterogeneity, likely attributable to differences in watershed size, baseflow contributions, and variation in aerial coverage of preferential flow paths. A regression analysis showed that a combination of climate and watershed characteristics are predictive of P–Q information flows. Though information flows cannot, in most cases, uniquely determine dominant runoff mechanisms, they provide a means to quantify the heterogeneous outcomes of those mechanisms within regions, thereby serving as a benchmarking tool for models developed at the regional scale. Last, information flows characterize regionally specific ways in which catchment connectivity changes from the wet to dry season. 
    more » « less
  2. Hydrologic signatures are quantitative metrics that describe streamflow statistics and dynamics. Signatures have many applications, including assessing habitat suitability and hydrologic alteration, calibrating and evaluating hydrologic models, defining similarity between watersheds and investigating watershed processes. Increasingly, signatures are being used in large sample studies to guide flow management and modelling at continental scales. Using signatures in studies involving 1000s of watersheds brings new challenges as it becomes impractical to examine signature parameters and behaviour in each watershed. For example, we might wish to check that signatures describing flood event characteristics have correctly identified event periods, that signature values have not been biassed by data errors, or that human and natural influences on signature values have been correctly interpreted. In this commentary, we draw from our collective experience to present case studies where naïve application of signatures fails to correctly identify streamflow dynamics. These include unusual precipitation or flow regimes, data quality issues, and signature use in human-influenced watersheds. We conclude by providing guidance and recommendations on applying signatures in large sample studies. 
    more » « less
  3. Assessing the uncertainty associated with projections of climate change impacts on hydrological processes can be challenging due to multiple sources of uncertainties within and between climate and hydrological models. Here we compare the effects of parameter uncertainty in a hydrological model to inter-model spread from climate projections on hydrological projections of urban streamflow in response to climate change. Four hourly climate model outputs from the RCP8.5 scenario were used as inputs to a distributed hydrologic model (SWMM) calibrated using a Bayesian approach to summarize uncertainty intervals for both model parameters and streamflow predictions. Continuous simulation of 100 years of streamflow generated 90 % prediction intervals for selected exceedance probabilities and flood frequencies prediction intervals from single climate models were compared to the inter climate model spread resulting from a single calibration of the SWMM model. There will be an increase in future flows with exceedance probabilities of 0.5 %-50 % and 2-year floods for all climate projections and all 21st century periods, for the modeled Ohio (USA) watershed. Floods with return periods of ≥ 5 years increase relative to the historical from mid-century (2046–2070) for most climate projections and parameter sets. Across the four climate models, the 90th percentile increase in flows and floods ranges from 17-108 % and 11–63 % respectively. Using multiple calibration parameter sets and climate projections helped capture the most likely hydrologic outcomes, as well as upper and lower bounds of future predictions. For this watershed, hydrological model parameter uncertainty was large relative to inter climate model spread, for near term moderate to high flows and for many flood frequencies. The uncertainty quantification and comparison approach developed here may be helpful in decision-making and design of engineering infrastructure in urban watersheds. 
    more » « less
  4. Abstract Hurricanes have been the most destructive and expensive hydrometeorological event in U.S. history, causing catastrophic winds and floods. Hurricane dynamics can significantly impact the amount and spatial extent of storm precipitation. However, the complex interactions of hurricane intensity and precipitation and the impacts of improving hurricane dynamics on streamflow forecasts are not well established yet. This paper addresses these gaps by comprehensively characterizing the role of vertical diffusion in improving hurricane intensity and streamflow forecasts under different planetary boundary layer, microphysics, and cumulus parameterizations. To this end, the Weather Research and Forecasting (WRF) atmospheric model is coupled with the WRF hydrological (WRF-Hydro) model to simulate four major hurricanes landfalling in three hurricane-prone regions in the United States. First, a stepwise calibration is carried out in WRF-Hydro, which remarkably reduces streamflow forecast errors compared to the U.S. Geological Survey (USGS) gauges. Then, 60 coupled hydrometeorological simulations were conducted to evaluate the performance of current weather parameterizations. All schemes were shown to underestimate the observed intensity of the considered major hurricanes since their diffusion is overdissipative for hurricane flow simulations. By reducing the vertical diffusion, hurricane intensity forecasts were improved by ∼39.5% on average compared to the default models. These intensified hurricanes generated more intense and localized precipitation forcing. This enhancement in intensity led to ∼16% and ∼34% improvements in hurricane streamflow bias and correlation forecasts, respectively. The research underscores the role of improved hurricane dynamics in enhancing flood predictions and provides new insights into the impacts of vertical diffusion on hurricane intensity and streamflow forecasts. Significance StatementDespite significant recent improvements, numerical weather prediction models struggle to accurately forecast hurricane intensity and track due to many reasons such as inaccurate physical parameterization for hurricane flows. Furthermore, the performance of existing physics schemes is not well studied for hurricane flood forecasting. This study bridges these knowledge gaps by extensively evaluating different physical parameterizations for hurricane track, intensity, and flood forecasts using an atmospheric model coupled with a hydrological model. Then, a reduced diffusion boundary layer scheme is developed, making remarkable improvements in hurricane intensity forecasts due to the overdissipative nature of the considered schemes for major hurricane simulations. This reduced diffusion model is shown to significantly enhance hurricane flood forecasts, indicating the significance of hurricane dynamics on its induced precipitation. 
    more » « less
  5. Abstract Numerous studies have examined the reliability of various precipitation products over the Mekong River Basin (MRB) and modeled its basin hydrology. However, there is a lack of comprehensive studies on precipitation‐induced uncertainties in hydrological simulations using process‐based land surface models. This study examines the propagation of precipitation uncertainty into hydrological simulations over the entire MRB using the Community Land Model version 5 (CLM5) at a high spatial resolution of 0.05° (∼5 km) and without any parameter calibration. Simulations conducted using different precipitation datasets are compared to investigate the discrepancies in streamflow, terrestrial water storage (TWS), soil moisture, and evapotranspiration (ET) caused by precipitation uncertainty. Results indicate that precipitation is a key determinant of simulated streamflow in the MRB; peak flow and soil moisture are particularly sensitive to precipitation input. Further, precipitation data with a higher spatial resolution did not improve the simulations, contrary to the common perception that using meteorological forcing with higher spatial resolution would improve hydrological simulations. In addition, since high flow indicators are particularly influenced by precipitation data, the choice of precipitation data could directly impact flood pulse simulations in the MRB. Notable differences are also found among TWS, soil moisture, and ET simulated using different precipitation products. Moreover, TWS, soil moisture, and ET exhibit a varying degree of sensitivity to precipitation uncertainty. This study provides crucial insights on precipitation‐induced uncertainties in process‐based hydrological modeling and uncovers these uncertainties in the MRB. 
    more » « less