skip to main content


Title: Macrosystems EDDIE Module 6: Understanding Uncertainty in Ecological Forecasts (Instructor Materials)
This EDI data package contains instructional materials necessary to teach Macrosystems EDDIE Module 6: Understanding Uncertainty in Ecological Forecasts, a ~3-hour educational module for undergraduates. Ecological forecasting is an emerging approach that provides an estimate of the future state of an ecological system with uncertainty, allowing society to prepare for changes in important ecosystem services. Forecast uncertainty is derived from multiple sources, including model parameters and driver data, among others. Knowing the uncertainty associated with a forecast enables forecast users to evaluate the forecast and make more informed decisions. This module will guide students through an exploration of the sources of uncertainty within an ecological forecast, how uncertainty can be quantified, and steps that can be taken to reduce the uncertainty in a forecast that students develop for a lake ecosystem, using data from the National Ecological Observatory Network (NEON). Students will visualize data, build a model, generate a forecast with uncertainty, and then compare the contributions of various sources of forecast uncertainty to total forecast uncertainty. The flexible, three-part (A-B-C) structure of this module makes it adaptable to a range of student levels and course structures. There are two versions of the module: an R Shiny application which does not require students to code, and an RMarkdown version which requires students to read and alter R code to complete module activities. The R Shiny application is published to shinyapps.io and is available at the following link: https://macrosystemseddie.shinyapps.io/module6/. GitHub repositories are available for both the R Shiny (https://github.com/MacrosystemsEDDIE/module6) and RMarkdown versions (https://github.com/MacrosystemsEDDIE/module6_R) of the module, and both code repositories have been published with DOIs to Zenodo (R Shiny version at https://zenodo.org/doi/10.5281/zenodo.10380759 and RMarkdown version at https://zenodo.org/doi/10.5281/zenodo.10380339). Readers are referred to the module landing page for additional information (https://serc.carleton.edu/eddie/teaching_materials/modules/module6.html).  more » « less
Award ID(s):
1926050
NSF-PAR ID:
10479679
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
Environmental Data Initiative
Date Published:
Edition / Version:
1
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Because of increased variability in populations, communities, and ecosystems due to land use and climate change, there is a pressing need to know the future state of ecological systems across space and time. Ecological forecasting is an emerging approach which provides an estimate of the future state of an ecological system with uncertainty, allowing society to preemptively prepare for fluctuations in important ecosystem services. However, forecasts must be effectively designed and communicated to those who need them to make decisions in order to realize their potential for protecting natural resources. In this module, students will explore real ecological forecast visualizations, identify ways to represent uncertainty, make management decisions using forecast visualizations, and learn decision support techniques. Lastly, students customize a forecast visualization for a specific stakeholder's decision needs. The overarching goal of this module is for students to understand how forecasts are connected to decision-making of stakeholders, or the managers, policy-makers, and other members of society who use forecasts to inform decision-making. The A-B-C structure of this module makes it flexible and adaptable to a range of student levels and course structures. This EDI data package contains instructional materials and the files necessary to teach the module. Readers are referred to the Zenodo data package (Woelmer et al. 2022; DOI: 10.5281/zenodo.7074674) for the R Shiny application code needed to run the module locally. 
    more » « less
  2. Abstract

    Communicating and interpreting uncertainty in ecological model predictions is notoriously challenging, motivating the need for new educational tools, which introduce ecology students to core concepts in uncertainty communication. Ecological forecasting, an emerging approach to estimate future states of ecological systems with uncertainty, provides a relevant and engaging framework for introducing uncertainty communication to undergraduate students, as forecasts can be used as decision support tools for addressing real‐world ecological problems and are inherently uncertain. To provide critical training on uncertainty communication and introduce undergraduate students to the use of ecological forecasts for guiding decision‐making, we developed a hands‐on teaching module within the Macrosystems Environmental Data‐Driven Inquiry and Exploration (EDDIE;MacrosystemsEDDIE.org) educational program. Our module used an active learning approach by embedding forecasting activities in an R Shiny application to engage ecology students in introductory data science, ecological modeling, and forecasting concepts without needing advanced computational or programming skills. Pre‐ and post‐module assessment data from more than 250 undergraduate students enrolled in ecology, freshwater ecology, and zoology courses indicate that the module significantly increased students' ability to interpret forecast visualizations with uncertainty, identify different ways to communicate forecast uncertainty for diverse users, and correctly define ecological forecasting terms. Specifically, students were more likely to describe visual, numeric, and probabilistic methods of uncertainty communication following module completion. Students were also able to identify more benefits of ecological forecasting following module completion, with the key benefits of using forecasts for prediction and decision‐making most commonly described. These results show promise for introducing ecological model uncertainty, data visualizations, and forecasting into undergraduate ecology curricula via software‐based learning, which can increase students' ability to engage and understand complex ecological concepts.

     
    more » « less
  3. Ecological forecasting is an emerging approach to estimate the future state of an ecological system with uncertainty, allowing society to better manage ecosystem services. Ecological forecasting is a core mission of the U.S. National Ecological Observatory Network (NEON) and several federal agencies, yet, to date, forecasting training has focused on graduate students, representing a gap in undergraduate ecology curricula. In response, we developed a teaching module for the Macrosystems EDDIE (Environmental Data-Driven Inquiry and Exploration; MacrosystemsEDDIE.org) educational program to introduce ecological forecasting to undergraduate students through an interactive online tool built with R Shiny. To date, we have assessed this module, “Introduction to Ecological Forecasting,” at ten universities and two conference workshops with both undergraduate and graduate students (N = 136 total) and found that the module significantly increased undergraduate students’ ability to correctly define ecological forecasting terms and identify steps in the ecological forecasting cycle. Undergraduate and graduate students who completed the module showed increased familiarity with ecological forecasts and forecast uncertainty. These results suggest that integrating ecological forecasting into undergraduate ecology curricula will enhance students’ abilities to engage and understand complex ecological concepts. 
    more » « less
  4. Over the last two decades, robust optimization has emerged as a popular means to address decision-making problems affected by uncertainty. This includes single-stage and multi-stage problems involving real-valued and/or binary decisions and affected by exogenous (decision-independent) and/or endogenous (decision-dependent) uncertain parameters. Robust optimization techniques rely on duality theory potentially augmented with approximations to transform a (semi-)infinite optimization problem to a finite program, the robust counterpart. Whereas writing down the model for a robust optimization problem is usually a simple task, obtaining the robust counterpart requires expertise. To date, very few solutions are available that can facilitate the modeling and solution of such problems. This has been a major impediment to their being put to practical use. In this paper, we propose ROC++, an open-source C++ based platform for automatic robust optimization, applicable to a wide array of single-stage and multi-stage robust problems with both exogenous and endogenous uncertain parameters, that is easy to both use and extend. It also applies to certain classes of stochastic programs involving continuously distributed uncertain parameters and endogenous uncertainty. Our platform naturally extends existing off-the-shelf deterministic optimization platforms and offers ROPy, a Python interface in the form of a callable library, and the ROB file format for storing and sharing robust problems. We showcase the modeling power of ROC++ on several decision-making problems of practical interest. Our platform can help streamline the modeling and solution of stochastic and robust optimization problems for both researchers and practitioners. It comes with detailed documentation to facilitate its use and expansion. The latest version of ROC++ can be downloaded from https://sites.google.com/usc.edu/robust-opt-cpp/ . Summary of Contribution: The paper “ROC++: Robust Optimization in C++” proposes a new open-source C++ based platform for modeling, automatically reformulating, and solving robust optimization problems. ROC++ can address both single-stage and multi-stage problems involving exogenous and/or endogenous uncertain parameters and real- and/or binary-valued adaptive variables. The ROC++ modeling language is similar to the one provided for the deterministic case by state-of-the-art deterministic optimization solvers. ROC++ comes with detailed documentation to facilitate its use and expansion. It also offers ROPy, a Python interface in the form of a callable library. The latest version of ROC++ can be downloaded from https://sites.google.com/usc.edu/robust-opt-cpp/ . History: Accepted by Ted Ralphs, Area Editor for Software Tools. Funding: This material is based upon work supported by the National Science Foundation under Grant No. 1763108. This support is gratefully acknowledged. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplementary Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.1209 ) or is available from the IJOC GitHub software repository ( https://github.com/INFORMSJoC ) at ( https://dx.doi.org/10.5281/zenodo.6360996 ). 
    more » « less
  5. This paper introduces a major redesign of SimOpt, a testbed of simulation-optimization (SO) problems and solvers. The testbed promotes the empirical evaluation and comparison of solvers and aims to accelerate their development. Relative to previous versions of SimOpt, the redesign ports the code to an object-oriented architecture in Python; uses an implementation of the MRG32k3a random number generator that supports streams, substreams, and subsubstreams; supports the automated use of common random numbers for ease and efficiency; includes a powerful suite of plotting tools for visualizing experiment results; uses bootstrapping to obtain error estimates; accommodates the use of data farming to explore simulation models and optimization solvers as their input parameters vary; and provides a graphical user interface. The SimOpt source code is available on a GitHub repository under a permissive open-source license and as a Python package. History: Accepted by Ted Ralphs, Area Editor for Software Tools. Funding: This work was supported by the National Science Foundation [Grant CMMI-2035086]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1273 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0011 ) at ( http://dx.doi.org/10.5281/zenodo.7468744 ). 
    more » « less