An official website of the United States government
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past observations, adaptive approaches adjust sampling constraints online as model parameter estimates are refined, continually maximizing expected information gained or variance reduced. We apply adaptive Bayesian inference to estimate transition rates of Markov chains, a common class of models for stochastic processes in nature. Unlike most previous studies, our sequential Bayesian optimal design is updated with each observation and can be simply extended beyond two-state models to birth–death processes and multistate models. By iteratively finding the best time to obtain each sample, our adaptive algorithm maximally reduces variance, resulting in lower overall error in ground truth parameter estimates across a wide range of Markov chain parameterizations and conformations.
more »
« less
Accounting for Location Measurement Error in Imaging Data With Application to Atomic Resolution Images of Crystalline Materials
Scientists use imaging to identify objects of interest and infer properties of these objects. The locations of these objects are often measured with error, which when ignored leads to biased parameter estimates and inflated variance. Current measurement error methods require an estimate or knowledge of the measurement error variance to correct these estimates, which may not be available. Instead, we create a spatial Bayesian hierarchical model that treats the locations as parameters, using the image itself to incorporate positional uncertainty. We lower the computational burden by approximating the likelihood using a noncontiguous block design around the object locations. We use this model to quantify the relationship between the intensity and displacement of hundreds of atom columns in crystal structures directly imaged via scanning transmission electron microscopy (STEM). Atomic displacements are related to important phenomena such as piezoelectricity, a property useful for engineering applications like ultrasound. Quantifying the sign and magnitude of this relationship will help materials scientists more precisely design materials with improved piezoelectricity. A simulation study confirms our method corrects bias in the estimate of the parameter of interest and drastically improves coverage in high noise scenarios compared to non-measurement error models.
more »
« less
- Award ID(s):
- 1633587
- PAR ID:
- 10295722
- Date Published:
- Journal Name:
- Technometrics
- ISSN:
- 0040-1706
- Page Range / eLocation ID:
- 1 to 11
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Nature is full of messy variation, which serves as the raw material for evolution. However, in comparative biology this variation is smoothed into averages. Overlooking this variation not only weakens our analyses but also risks selecting inaccurate models, generating false precision in parameter estimates, and creating artificial patterns. Furthermore, the complexity of uncertainty extends beyond traditional “measurement error,” encompassing various sources of intraspecific variance. To address this, we propose the term “tip fog” to describe the variance between the true species mean and what is recorded, without implying a specific mechanism. We show why accounting for tip fog remains critical by showing its impact on continuous comparative models and discrete comparative and diversification models. We rederive methods to estimate this variance and use simulations to assess its feasibility and importance in a comparative context. Our findings reveal that ignoring tip fog substantially affects the accuracy of rate estimates, with higher tip fog levels showing greater biases from the true rates, as well as affecting which models are chosen. The findings underscore the importance of model selection and the potential consequences of neglecting tip fog, providing insights for improving the accuracy of comparative methods in evolutionary biology.more » « less
-
PurposeTo improve the performance of neural networks for parameter estimation in quantitative MRI, in particular when the noise propagation varies throughout the space of biophysical parameters. Theory and MethodsA theoretically well‐founded loss function is proposed that normalizes the squared error of each estimate with respective Cramér–Rao bound (CRB)—a theoretical lower bound for the variance of an unbiased estimator. This avoids a dominance of hard‐to‐estimate parameters and areas in parameter space, which are often of little interest. The normalization with corresponding CRB balances the large errors of fundamentally more noisy estimates and the small errors of fundamentally less noisy estimates, allowing the network to better learn to estimate the latter. Further, proposed loss function provides an absolute evaluation metric for performance: A network has an average loss of 1 if it is a maximally efficient unbiased estimator, which can be considered the ideal performance. The performance gain with proposed loss function is demonstrated at the example of an eight‐parameter magnetization transfer model that is fitted to phantom and in vivo data. ResultsNetworks trained with proposed loss function perform close to optimal, that is, their loss converges to approximately 1, and their performance is superior to networks trained with the standard mean‐squared error (MSE). The proposed loss function reduces the bias of the estimates compared to the MSE loss, and improves the match of the noise variance to the CRB. This performance gain translates to in vivo maps that align better with the literature. ConclusionNormalizing the squared error with the CRB during the training of neural networks improves their performance in estimating biophysical parameters.more » « less
-
We review select mature geomorphic transport laws for use in temperate ridge and valley landscapes and compile parameter estimates for use in applications. This work is motivated by a case study of sensitivity analysis, calibration, validation, multimodel comparison, and prediction under uncertainty, which required bounding values for parameter ranges. Considered geomorphic transport formulae span hillslope sediment transport, soil production, and erosion by surface water. We compile or derive estimates for the parameters in these transport formulae. Additionally, we address a common challenge—connecting changes in precipitation distribution to changes in effective erodibility—by using a simple hydrologic model and a method to estimate precipitation distribution parameters using commonly available data. While some parameters are reasonably well constrained, others span orders of magnitude. Some, such as soil infiltration capacity, have a direct physical meaning but are challenging to measure on geologically relevant timescales. Through the process of compiling these ranges we identify common challenges in parameter determination. The issue of comparable units derives from considering an exponent as an empirically inferred coefficient rather than as an expression of a fundamental relationship. The issue of appropriate timescales derives from the mismatch between human measurement and geologic timescales. This contribution thus serves both as a practical compilation for applications and as a synthesis of outstanding challenges in parameter selection for geomorphic transport laws.more » « less
-
Rangel, Thiago F (Ed.)Species distribution models (SDMs) are frequently data-limited. In aquatic habitats, emerging environmental DNA (eDNA) sampling methods can be quicker and more cost-efficient than traditional count and capture surveys, but their utility for fitting SDMs is complicated by dilution, transport, and loss processes that modulate DNA concentrations and mix eDNA from different locations. Past models for estimating organism densities from measured species-specific eDNA concentrations have accounted for how these processes affect expected concentrations. We built off this previous work to construct a linear hierarchical model that also accounts for how they give rise to spatially correlated concentration errors. We applied our model to 60 simulated stream networks and three types of species niches in order to answer two questions: 1) what is the D-optimal sampling design, i.e. where should eDNA samples be positioned to most precisely estimate species–environment relationships? and 2) How does parameter estimation accuracy depend on the stream network’s topological and hydrologic properties? We found that correcting for eDNA dynamics was necessary to obtain consistent parameter estimates, and that relative to a heuristic benchmark design, optimizing sampling locations improved design efficiency by an average of 41.5%. Samples in the D-optimal design tended to be positioned near downstream ends of stream reaches high in the watershed, where eDNA concentration was high and mostly from homogeneous source areas, and they collectively spanned the full ranges of covariates. When measurement error was large, it was often optimal to collect replicate samples from high-information reaches. eDNA-based estimates of species–environment regression parameters were most precise in stream networks that had many reaches, large geographic size, slow flows, and/or high eDNA loss rates. Our study demonstrates the importance and viability of accounting for eDNA dilution, transport, and loss in order to optimize sampling designs and improve the accuracy of eDNA-based species distribution models.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1080/00401706.2021.1905070
