An official website of the United States government
Abstract Communicating and interpreting uncertainty in ecological model predictions is notoriously challenging, motivating the need for new educational tools, which introduce ecology students to core concepts in uncertainty communication. Ecological forecasting, an emerging approach to estimate future states of ecological systems with uncertainty, provides a relevant and engaging framework for introducing uncertainty communication to undergraduate students, as forecasts can be used as decision support tools for addressing real‐world ecological problems and are inherently uncertain. To provide critical training on uncertainty communication and introduce undergraduate students to the use of ecological forecasts for guiding decision‐making, we developed a hands‐on teaching module within the Macrosystems Environmental Data‐Driven Inquiry and Exploration (EDDIE;MacrosystemsEDDIE.org) educational program. Our module used an active learning approach by embedding forecasting activities in an R Shiny application to engage ecology students in introductory data science, ecological modeling, and forecasting concepts without needing advanced computational or programming skills. Pre‐ and post‐module assessment data from more than 250 undergraduate students enrolled in ecology, freshwater ecology, and zoology courses indicate that the module significantly increased students' ability to interpret forecast visualizations with uncertainty, identify different ways to communicate forecast uncertainty for diverse users, and correctly define ecological forecasting terms. Specifically, students were more likely to describe visual, numeric, and probabilistic methods of uncertainty communication following module completion. Students were also able to identify more benefits of ecological forecasting following module completion, with the key benefits of using forecasts for prediction and decision‐making most commonly described. These results show promise for introducing ecological model uncertainty, data visualizations, and forecasting into undergraduate ecology curricula via software‐based learning, which can increase students' ability to engage and understand complex ecological concepts.
more »
« less
Evaluating probabilistic ecological forecasts
Abstract Probabilistic near‐term forecasting facilitates evaluation of model predictions against observations and is of pressing need in ecology to inform environmental decision‐making and effect societal change. Despite this imperative, many ecologists are unfamiliar with the widely used tools for evaluating probabilistic forecasts developed in other fields. We address this gap by reviewing the literature on probabilistic forecast evaluation from diverse fields including climatology, economics, and epidemiology. We present established practices for selecting evaluation data (end‐sample hold out), graphical forecast evaluation (times‐series plots with uncertainty, probability integral transform plots), quantitative evaluation using scoring rules (log, quadratic, spherical, and ranked probability scores), and comparing scores across models (skill score, Diebold–Mariano test). We cover common approaches, highlight mathematical concepts to follow, and note decision points to allow application of general principles to specific forecasting endeavors. We illustrate these approaches with an application to a long‐term rodent population time series currently used for ecological forecasting and discuss how ecology can continue to learn from and drive the cross‐disciplinary field of forecasting science.
more »
« less
- Award ID(s):
- 1929730
- PAR ID:
- 10449199
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Ecology
- Volume:
- 102
- Issue:
- 8
- ISSN:
- 0012-9658
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Near‐term ecological forecasting can be used to improve operational resource management in freshwater ecosystems. Here, we developed a framework that uses water temperature forecasting as a tool to predict the migrations of Atlantic salmon (Salmo salar) and European eel (Anguilla anguilla) between freshwater and the sea. We used historical observations of lake water temperature and fish migrations from an internationally important long‐term monitoring site (the Burrishoole catchment, Ireland) to generate daily probabilistic predictions (0%–100%) of when relatively large numbers of fish migrate. For this, we produced daily lake water temperature forecasts that extended up to 34 days into the future using Forecasting Lake and Reservoir Ecosystems (FLARE), an open‐source ensemble‐based forecasting system. We used this system to forecast lake water temperature conditions associated with percentile‐based fish migrations. Two metrics, P66 and P95, were used to indicate days with migrations in excess of 66% and 95%, respectively, of the historical daily fish counts. The results were first validated against water temperature observations, with an overall root mean squared error (RMSE) of 0.97°C. Our forecasts outperformed two other possible water temperature forecasting approaches, using site climatology (1.36°C) and site persistence (1.19°C). The predictions for fish migrations performed better for the P66 metric than for the more extreme P95 metric based on the continuous ranked probability score (CRPS), and the best results were obtained for the salmon downstream migration. This forecasting approach with quantified uncertainty levels has the potential to assist decision making, especially in the face of increased risks for these species. We conclude by discussing the scalability of the framework to other settings as a tool aimed at supporting management practices in real time.more » « less
-
This paper advances machine learning (ML)-based streamflow prediction by strategically selecting rainfall events, introducing a new loss function, and addressing rainfall forecast uncertainties. Focusing on the Iowa River Basin, we applied the stochastic storm transposition (SST) method to create realistic rainfall events, which were input into a hydrological model to generate corresponding streamflow data for training and testing deterministic and probabilistic ML models. Long short-term memory (LSTM) networks were employed to predict streamflow up to 12 h ahead. An active learning approach was used to identify the most informative rainfall events, reducing data generation effort. Additionally, we introduced a novel asymmetric peak loss function to improve peak streamflow prediction accuracy. Incorporating rainfall forecast uncertainties, our probabilistic LSTM model provided uncertainty quantification for streamflow predictions. Performance evaluation using different metrics improved the accuracy and reliability of our models. These contributions enhance flood forecasting and decision-making while significantly reducing computational time and costs.more » « less
-
Abstract Current earthquake forecasting approaches are mainly based on probabilistic assumptions, as earthquakes seem to occur randomly. Such apparent randomness can however be caused by deterministic chaos, rendering deterministic short‐term forecasts possible. Due to the short historical and instrumental record of earthquakes, chaos detection has proven challenging, but more frequently occurring slow slip events (SSE) are promising candidates to probe for determinism. Here, we characterize the SSE signatures obtained from GNSS position time series in the Hikurangi Subduction Zone (New Zealand) to investigate whether the seemingly random SSE occurrence is governed by chaotic determinism. We find evidence for deterministic chaos for stations recording shallow SSEs, suggesting that short‐term deterministic forecasting of SSEs, similar to weather forecasts, might indeed be possible over timescales of a few weeks. We anticipate that our findings could open the door for next‐generation SSE forecasting, adding new tools to existing probabilistic approaches.more » « less
-
Abstract Near‐term iterative forecasting is a powerful tool for ecological decision support and has the potential to transform our understanding of ecological predictability. However, to this point, there has been no cross‐ecosystem analysis of near‐term ecological forecasts, making it difficult to synthesize diverse research efforts and prioritize future developments for this emerging field. In this study, we analyzed 178 near‐term (≤10‐yr forecast horizon) ecological forecasting papers to understand the development and current state of near‐term ecological forecasting literature and to compare forecast accuracy across scales and variables. Our results indicated that near‐term ecological forecasting is widespread and growing: forecasts have been produced for sites on all seven continents and the rate of forecast publication is increasing over time. As forecast production has accelerated, some best practices have been proposed and application of these best practices is increasing. In particular, data publication, forecast archiving, and workflow automation have all increased significantly over time. However, adoption of proposed best practices remains low overall: for example, despite the fact that uncertainty is often cited as an essential component of an ecological forecast, only 45% of papers included uncertainty in their forecast outputs. As the use of these proposed best practices increases, near‐term ecological forecasting has the potential to make significant contributions to our understanding of forecastability across scales and variables. In this study, we found that forecastability (defined here as realized forecast accuracy) decreased in predictable patterns over 1–7 d forecast horizons. Variables that were closely related (i.e., chlorophyll and phytoplankton) displayed very similar trends in forecastability, while more distantly related variables (i.e., pollen and evapotranspiration) exhibited significantly different patterns. Increasing use of proposed best practices in ecological forecasting will allow us to examine the forecastability of additional variables and timescales in the future, providing a robust analysis of the fundamental predictability of ecological variables.more » « less
