An official website of the United States government
Abstract Communicating and interpreting uncertainty in ecological model predictions is notoriously challenging, motivating the need for new educational tools, which introduce ecology students to core concepts in uncertainty communication. Ecological forecasting, an emerging approach to estimate future states of ecological systems with uncertainty, provides a relevant and engaging framework for introducing uncertainty communication to undergraduate students, as forecasts can be used as decision support tools for addressing real‐world ecological problems and are inherently uncertain. To provide critical training on uncertainty communication and introduce undergraduate students to the use of ecological forecasts for guiding decision‐making, we developed a hands‐on teaching module within the Macrosystems Environmental Data‐Driven Inquiry and Exploration (EDDIE;MacrosystemsEDDIE.org) educational program. Our module used an active learning approach by embedding forecasting activities in an R Shiny application to engage ecology students in introductory data science, ecological modeling, and forecasting concepts without needing advanced computational or programming skills. Pre‐ and post‐module assessment data from more than 250 undergraduate students enrolled in ecology, freshwater ecology, and zoology courses indicate that the module significantly increased students' ability to interpret forecast visualizations with uncertainty, identify different ways to communicate forecast uncertainty for diverse users, and correctly define ecological forecasting terms. Specifically, students were more likely to describe visual, numeric, and probabilistic methods of uncertainty communication following module completion. Students were also able to identify more benefits of ecological forecasting following module completion, with the key benefits of using forecasts for prediction and decision‐making most commonly described. These results show promise for introducing ecological model uncertainty, data visualizations, and forecasting into undergraduate ecology curricula via software‐based learning, which can increase students' ability to engage and understand complex ecological concepts.
more »
« less
Evaluating probabilistic ecological forecasts
Abstract Probabilistic near‐term forecasting facilitates evaluation of model predictions against observations and is of pressing need in ecology to inform environmental decision‐making and effect societal change. Despite this imperative, many ecologists are unfamiliar with the widely used tools for evaluating probabilistic forecasts developed in other fields. We address this gap by reviewing the literature on probabilistic forecast evaluation from diverse fields including climatology, economics, and epidemiology. We present established practices for selecting evaluation data (end‐sample hold out), graphical forecast evaluation (times‐series plots with uncertainty, probability integral transform plots), quantitative evaluation using scoring rules (log, quadratic, spherical, and ranked probability scores), and comparing scores across models (skill score, Diebold–Mariano test). We cover common approaches, highlight mathematical concepts to follow, and note decision points to allow application of general principles to specific forecasting endeavors. We illustrate these approaches with an application to a long‐term rodent population time series currently used for ecological forecasting and discuss how ecology can continue to learn from and drive the cross‐disciplinary field of forecasting science.
more »
« less
- Award ID(s):
- 1929730
- PAR ID:
- 10449199
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Ecology
- Volume:
- 102
- Issue:
- 8
- ISSN:
- 0012-9658
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Current earthquake forecasting approaches are mainly based on probabilistic assumptions, as earthquakes seem to occur randomly. Such apparent randomness can however be caused by deterministic chaos, rendering deterministic short‐term forecasts possible. Due to the short historical and instrumental record of earthquakes, chaos detection has proven challenging, but more frequently occurring slow slip events (SSE) are promising candidates to probe for determinism. Here, we characterize the SSE signatures obtained from GNSS position time series in the Hikurangi Subduction Zone (New Zealand) to investigate whether the seemingly random SSE occurrence is governed by chaotic determinism. We find evidence for deterministic chaos for stations recording shallow SSEs, suggesting that short‐term deterministic forecasting of SSEs, similar to weather forecasts, might indeed be possible over timescales of a few weeks. We anticipate that our findings could open the door for next‐generation SSE forecasting, adding new tools to existing probabilistic approaches.more » « less
-
Abstract Near‐term iterative forecasting is a powerful tool for ecological decision support and has the potential to transform our understanding of ecological predictability. However, to this point, there has been no cross‐ecosystem analysis of near‐term ecological forecasts, making it difficult to synthesize diverse research efforts and prioritize future developments for this emerging field. In this study, we analyzed 178 near‐term (≤10‐yr forecast horizon) ecological forecasting papers to understand the development and current state of near‐term ecological forecasting literature and to compare forecast accuracy across scales and variables. Our results indicated that near‐term ecological forecasting is widespread and growing: forecasts have been produced for sites on all seven continents and the rate of forecast publication is increasing over time. As forecast production has accelerated, some best practices have been proposed and application of these best practices is increasing. In particular, data publication, forecast archiving, and workflow automation have all increased significantly over time. However, adoption of proposed best practices remains low overall: for example, despite the fact that uncertainty is often cited as an essential component of an ecological forecast, only 45% of papers included uncertainty in their forecast outputs. As the use of these proposed best practices increases, near‐term ecological forecasting has the potential to make significant contributions to our understanding of forecastability across scales and variables. In this study, we found that forecastability (defined here as realized forecast accuracy) decreased in predictable patterns over 1–7 d forecast horizons. Variables that were closely related (i.e., chlorophyll and phytoplankton) displayed very similar trends in forecastability, while more distantly related variables (i.e., pollen and evapotranspiration) exhibited significantly different patterns. Increasing use of proposed best practices in ecological forecasting will allow us to examine the forecastability of additional variables and timescales in the future, providing a robust analysis of the fundamental predictability of ecological variables.more » « less
-
Few-shot machine learning attempts to predict outputs given only a very small number of training examples. The key idea behind most few-shot learning approaches is to pre-train the model with a large number of instances from a different but related class of data, classes for which a large number of instances are available for training. Few-shot learning has been most successfully demonstrated for classification problems using Siamese deep learning neural networks. Few-shot learning is less extensively applied to time-series forecasting. Few-shot forecasting is the task of predicting future values of a time-series even when only a small set of historic time-series is available. Few-shot forecasting has applications in domains where a long history of data is not available. This work describes deep neural network architectures for few-shot forecasting. All the architectures use a Siamese twin network approach to learn a difference function between pairs of time-series, rather than directly forecasting based on historical data as seen in traditional forecasting models. The networks are built using Long short-term memory units (LSTM). During forecasting, a model is able to forecast time-series types that were never seen in the training data by using the few available instances of the new time-series type as reference inputs. The proposed architectures are evaluated on Vehicular traffic data collected in California from the Caltrans Performance Measurement System (PeMS). The models were trained with traffic flow data collected at specific locations and then are evaluated by predicting traffic at different locations at different time horizons (0 to 12 hours). The Mean Absolute Error (MAE) was used as the evaluation metric and also as the loss function for training. The proposed architectures show lower prediction error than a baseline nearest neighbor forecast model. The prediction error increases at longer time horizons.more » « less
-
Abstract Ecological forecasting provides a powerful set of methods for predicting short‐ and long‐term change in living systems. Forecasts are now widely produced, enabling proactive management for many applied ecological problems. However, despite numerous calls for an increased emphasis on prediction in ecology, the potential for forecasting to accelerate ecological theory development remains underrealized.Here, we provide a conceptual framework describing how ecological forecasts can energize and advance ecological theory. We emphasize the many opportunities for future progress in this area through increased forecast development, comparison and synthesis.Our framework describes how a forecasting approach can shed new light on existing ecological theories while also allowing researchers to address novel questions. Through rigorous and repeated testing of hypotheses, forecasting can help to refine theories and understand their generality across systems. Meanwhile, synthesizing across forecasts allows for the development of novel theory about the relative predictability of ecological variables across forecast horizons and scales.We envision a future where forecasting is integrated as part of the toolset used in fundamental ecology. By outlining the relevance of forecasting methods to ecological theory, we aim to decrease barriers to entry and broaden the community of researchers using forecasting for fundamental ecological insight.more » « less
