Representing hydrologic connectivity of non‐floodplain wetlands (NFWs) to downstream waters in process‐based models is an emerging challenge relevant to many research, regulatory, and management activities. We review four case studies that utilize process‐based models developed to simulate NFW hydrology. Models range from a simple, lumped parameter model to a highly complex, fully distributed model. Across case studies, we highlight appropriate application of each model, emphasizing spatial scale, computational demands, process representation, and model limitations. We end with a synthesis of recommended “best modeling practices” to guide model application. These recommendations include: (1) clearly articulate modeling objectives, and revisit and adjust those objectives regularly; (2) develop a conceptualization of NFW connectivity using qualitative observations, empirical data, and process‐based modeling; (3) select a model to represent NFW connectivity by balancing both modeling objectives and available resources; (4) use innovative techniques and data sources to validate and calibrate NFW connectivity simulations; and (5) clearly articulate the limits of the resulting NFW connectivity representation. Our review and synthesis of these case studies highlights modeling approaches that incorporate NFW connectivity, demonstrates tradeoffs in model selection, and ultimately provides actionable guidance for future model application and development.
Selecting among competing statistical models is a core challenge in science. However, the many possible approaches and techniques for model selection, and the conflicting recommendations for their use, can be confusing. We contend that much confusion surrounding statistical model selection results from failing to first clearly specify the purpose of the analysis. We argue that there are three distinct goals for statistical modeling in ecology: data exploration, inference, and prediction. Once the modeling goal is clearly articulated, an appropriate model selection procedure is easier to identify. We review model selection approaches and highlight their strengths and weaknesses relative to each of the three modeling goals. We then present examples of modeling for exploration, inference, and prediction using a time series of butterfly population counts. These show how a model selection approach flows naturally from the modeling goal, leading to different models selected for different purposes, even with exactly the same data set. This review illustrates best practices for ecologists and should serve as a reminder that statistical recipes cannot substitute for critical thinking or for the use of independent data to test hypotheses and validate predictions.
more » « less- NSF-PAR ID:
- 10450796
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Ecology
- Volume:
- 102
- Issue:
- 6
- ISSN:
- 0012-9658
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract There is a growing need for flexible general frameworks that integrate individual-level data with external summary information for improved statistical inference. External information relevant for a risk prediction model may come in multiple forms, through regression coefficient estimates or predicted values of the outcome variable. Different external models may use different sets of predictors and the algorithm they used to predict the outcome Y given these predictors may or may not be known. The underlying populations corresponding to each external model may be different from each other and from the internal study population. Motivated by a prostate cancer risk prediction problem where novel biomarkers are measured only in the internal study, this paper proposes an imputation-based methodology, where the goal is to fit a target regression model with all available predictors in the internal study while utilizing summary information from external models that may have used only a subset of the predictors. The method allows for heterogeneity of covariate effects across the external populations. The proposed approach generates synthetic outcome data in each external population, uses stacked multiple imputation to create a long dataset with complete covariate information. The final analysis of the stacked imputed data is conducted by weighted regression. This flexible and unified approach can improve statistical efficiency of the estimated coefficients in the internal study, improve predictions by utilizing even partial information available from models that use a subset of the full set of covariates used in the internal study, and provide statistical inference for the external population with potentially different covariate effects from the internal population.
-
more » « less
Abstract Prediction of ionospheric state is a critical space weather problem. We expand on our previous research of medium‐range ionospheric forecasts and present new results on evaluating prediction capabilities of three physics‐based ionosphere‐thermosphere models (Thermosphere Ionosphere Electrodynamics General Circulation Model, TIE‐GCM; Coupled Thermosphere Ionosphere Plasmasphere Electrodynamics Model, CTIPe; and Global Ionosphere Thermosphere Model, GITM). The focus of our study is understanding how current modeling approaches may predict the global ionosphere for geomagnetic storms (as studied through 35 storms during 2000–2016). Prediction approach uses physics‐based modeling without any manual model adjustment, quality control, or selection of the results. Our goal is to understand to what extent current physics‐based modeling can be used in total electron content (TEC) prediction and explore uncertainties of these prediction efforts with multiday lead times. The ionosphere‐thermosphere model runs are driven by actual interplanetary conditions, whether those data come from real‐time measurements or predicted values themselves. These model runs were performed by the Community Coordinated Modeling Center (CCMC). Jet Propulsion Laboratory (JPL)‐produced global ionospheric maps (GIMs) were used to validate model TEC estimates. We utilize the True Skill Statistic (TSS) metric for the TEC prediction evaluation, noting that this is but one metric to assess predictive skill and that complete evaluations require combinations of such metrics. The meanings of contingency table elements for the prediction performance are analyzed in the context of ionosphere modeling. Prediction success is between about 0.2 and 0.5 for weak ionospheric disturbances and decreases for strong disturbances. We evaluate the prediction of TEC decreases and increases. Our results indicate that physics‐based modeling during storms shows promise in TEC prediction with multiday lead time.
-
more » « less
Abstract Vertical movements can expose individuals to rapid changes in physical and trophic environments—for aquatic fauna, dive profiles from biotelemetry data can be used to quantify and categorize vertical movements. Inferences on classes of vertical movement profiles typically rely on subjective summaries of parameters or statistical clustering techniques that utilize Euclidean matching of vertical movement profiles with vertical observation points. These approaches are prone to subjectivity, error, and bias. We used machine learning approaches on a large dataset of vertical time series (
N = 28,217 dives) for 31 post‐nesting leatherback turtles (Dermochelys coriacea ). We applied dynamic time warp (DTW) clustering to group vertical movement (dive) time series by their metrics (depth and duration) into an optimal number of clusters. We then identified environmental covariates associated with each cluster using a generalized additive mixed‐effects model (GAMM). A convolutional neural network (CNN) model, trained on standard dive shape types from the literature, was used to classify dives within each DTW cluster by their shape. Two clusters were identified with the DTW approach—these varied in their spatial and temporal distributions, with dependence on environmental covariates, sea surface temperature, bathymetry, sea surface height anomaly, and time‐lagged surface chlorophylla concentrations. CNN classification accuracy of the five standard dive profiles was 95%. Subsequent analyses revealed that the two clusters differed in their composition of standard dive shapes, with each cluster dominated by shapes indicative of distinct behaviors (pelagic foraging and exploration, respectively). The use of these two machine learning approaches allowed for discrete behaviors to be identified from vertical time series data, first by clustering vertical movements by their movement metrics (DTW) and second by classifying dive profiles within each cluster by their shapes (CNN). Statistical inference for the identified clusters found distinct relationships with environmental covariates, supporting hypotheses of vertical niche switching and vertically structured foraging behavior. This approach could be similarly applied to the time series of other animals utilizing the vertical dimension in their movements, including aerial, arboreal, and other aquatic species, to efficiently identify different movement behaviors and inform habitat models. -
more » « less
Abstract The rapid development of modeling techniques has brought many opportunities for data‐driven discovery and prediction. However, this also leads to the challenge of selecting the most appropriate model for any particular data task. Information criteria, such as the Akaike information criterion (AIC) and Bayesian information criterion (BIC), have been developed as a general class of model selection methods with profound connections with foundational thoughts in statistics and information theory. Many perspectives and theoretical justifications have been developed to understand when and how to use information criteria, which often depend on particular data circumstances. This review article will revisit information criteria by summarizing their key concepts, evaluation metrics, fundamental properties, interconnections, recent advancements, and common misconceptions to enrich the understanding of model selection in general.
This article is categorized under:
Data: Types and Structure > Traditional Statistical Data
Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods
Statistical and Graphical Methods of Data Analysis > Information Theoretic Methods
Statistical Models > Model Selection
