We describe the results of a randomized controlled trial of video-streaming algorithms for bitrate selection and network prediction. Over the last year, we have streamed 38.6 years of video to 63,508 users across the Internet. Sessions are randomized in blinded fashion among algorithms.
We found that in this real-world setting, it is difficult for sophisticated or machine-learned control schemes to outperform a "simple" scheme (buffer-based control), notwithstanding good performance in network emulators or simulators. We performed a statistical analysis and found that the heavy-tailed nature of network and user behavior, as well as the challenges of emulating diverse Internet paths during training, present obstacles for learned algorithms in this setting.
We then developed an ABR algorithm that robustly outperformed other schemes, by leveraging data from its deployment and limiting the scope of machine learning only to making predictions that can be checked soon after. The system uses supervised learning in situ, with data from the real deployment environment, to train a probabilistic predictor of upcoming chunk transmission times. This module then informs a classical control policy (model predictive control).
To support further investigation, we are publishing an archive of data and results each week, and will open our ongoing study to the community. We welcome other researchers to use this platform to develop and validate new algorithms for bitrate selection, network prediction, and congestion control.
more »
« less
On prediction of future insurance claims when the model is uncertain
Predictive modeling is arguably one of the most important tasks
actuaries face in their day-to-day work. In practice, actuaries
may have a number of reasonable models to consider, all of
which will provide different predictions. The most common
strategy is first to use some kind of model selection tool to select
a ``best model" and then to use that model to make predictions.
However, there is reason to be concerned about the use of the
classical distribution theory to develop predictions because this
theory ignores the selection effect. Since accuracy of predictions
is crucial to the insurer’s pricing and solvency, care is needed
to develop valid prediction methods. This paper investigates the
effects of model selection on the validity of classical prediction
tools and makes some recommendations for practitioners.
more »
« less
- Award ID(s):
- 1712940
- NSF-PAR ID:
- 10108899
- Date Published:
- Journal Name:
- Variance
- Volume:
- 12
- Issue:
- 1
- ISSN:
- 1940-6452
- Page Range / eLocation ID:
- 90-99
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract In this study we present AI Prediction of Equatorial Plasma Bubbles (APE), a machine learning model that can accurately predict the Ionospheric Bubble Index (IBI) on the Swarm spacecraft. IBI is a correlation (
R 2) between perturbations in plasma density and the magnetic field, whose source can be Equatorial Plasma Bubbles (EPBs). EPBs have been studied for a number of years, but their day‐to‐day variability has made predicting them a considerable challenge. We build an ensemble machine learning model to predict IBI. We use data from 2014 to 2022 at a resolution of 1s, and transform it from a time‐series into a 6‐dimensional space with a corresponding EPBR 2(0–1) acting as the label. APE performs well across all metrics, exhibiting a skill, association and root mean squared error score of 0.96, 0.98 and 0.08 respectively. The model performs best post‐sunset, in the American/Atlantic sector, around the equinoxes, and when solar activity is high. This is promising because EPBs are most likely to occur during these periods. Shapley values reveal that F10.7 is the most important feature in driving the predictions, whereas latitude is the least. The analysis also examines the relationship between the features, which reveals new insights into EPB climatology. Finally, the selection of the features means that APE could be expanded to forecasting EPBs following additional investigations into their onset. -
Accurate characterization of the mechanical properties of the human brain at both microscopic and macroscopic length scales is a critical requirement for modeling of traumatic brain injury and brain folding. To date, most experimental studies that employ classical tension/compression/shear tests report the mechanical properties of the brain averaged over both the gray and white matter within the macroscopic regions of interest. As a result, there is a missing correlation between the independent mechanical properties of the microscopic constituent elements and the composite bulk macroscopic mechanical properties of the tissue. This microstructural computational study aims to inversely predict the hyperelastic mechanical properties of the axonal fibers and their surrounding extracellular matrix (ECM) from the bulk tissue's mechanical properties. We develop a representative volume element (RVE) model of the bulk tissue consisting of axonal fibers and ECM with the embedded element technique. A multiobjective optimization technique is implemented to calibrate the model and establish the independent mechanical properties of axonal fibers and ECM based on seven previously reported experimental mechanical tests for bulk white matter tissue from the corpus callosum. The result of the study shows that the discrepancy between the reported values for the elastic behavior of white matter in literature stems from the anisotropy of the tissue at the microscale. The shear modulus of the axonal fiber is seven times larger than the ECM, with axonal fibers that also show greater nonlinearity, contrary to the common assumption that both components exhibit identical nonlinear characteristics. Statement of significance The reported mechanical properties of white matter microstructure used in traumatic brain injury or brain mechanics studies vary widely, in some cases by up to two orders of magnitude. Currently, the material parameters of the white matter microstructure are identified by a single loading mode or ultimately two modes of the bulk tissue. The presented material models only define the response of the bulk and homogenized white matter at a macroscopic scale and cannot explicitly capture the connection between the material properties of microstructure and bulk structure. To fill this knowledge gap, our study characterizes the hyperelastic material properties of axonal fibers and ECM using microscale computational modeling and multiobjective optimization. The hyperelastic material properties for axonal fibers and ECM presented in this study are more accurate than previously proposed because they have been optimized using seven or six loading modes of the bulk tissue, which were previously limited to only two of the seven possible loading modes. As such, the predicted values with high accuracy could be used in various computational modeling studies. The systematic characterization of the material properties of the human brain tissue at both macro- and microscales will lead to more accurate computational predictions, which will enable a better understanding of injury criteria, and has a positive impact on the improved development of smart protection systems, and more accurate prediction of brain development and disease progression.more » « less
-
more » « less
Abstract Despite the shared prediction that the width of a population's dietary niche expands as food becomes limiting, the Niche Variation Hypothesis (NVH) and Optimal Foraging Theory (OFT) offer contrasting views about how individuals alter diet selection when food is limited.
Classical OFT predicts that dietary preferences do not change as food becomes limiting, so individuals expand their diets as they compensate for a lack of preferred foods. In contrast, the NVH predicts that among‐individual variation in cognition, physiology or morphology create functional trade‐offs in foraging efficiency, thereby causing individuals to specialize on different subsets of food as food becomes limiting.
To evaluate (a) the predictions of the NVH and OFT and (b) evidence for physiological and cognitive‐based functional trade‐offs, we used DNA microsatellites and metabarcoding to quantify the diet, microbiome and genetic relatedness (a proxy for social learning) of 218 moose
Alces alces across six populations that varied in their degree of food limitation.Consistent with both the NVH and OFT, dietary niche breadth increased with food limitation. Increased diet breadth of individuals—rather than increased diet specialization—was strongly correlated with both food limitation and dietary niche breadth of populations, indicating that moose foraged in accordance with OFT. Diets were not constrained by inheritance of the microbiome or inheritance of diet selection, offering support for the little‐tested hypothesis that functional trade‐offs in food use (or lack thereof) determine whether populations adhere to the predictions of the NVH or OFT.
Our results indicate that both the absence of strong functional trade‐offs and the digestive physiology of ruminants provide contexts under which populations should forage in accordance with OFT rather than the NVH. Also, because dietary niche width increased with increased food limitation, OFT and the NVH provide theoretical support for the notion that plant–herbivore interaction networks are plastic rather than static, which has important implications for understanding interspecific niche partitioning. Lastly, because population‐level dietary niche breadth and calf recruitment are correlated, and because calf recruitment can be a proxy for food limitation, our work demonstrates how diet data can be employed to understand a populations' proximity to carrying capacity.
-
Neighborhood models have allowed us to test many hypotheses regarding the drivers of variation in tree growth, but require considerable computation due to the many empirically supported non-linear relationships they include. Regularized regression represents a far more efficient neighborhood modeling method, but it is unclear whether such an ecologically unrealistic model can provide accurate insights on tree growth. Rapid computation is becoming increasingly important as ecological datasets grow in size, and may be essential when using neighborhood models to predict tree growth beyond sample plots or into the future. We built a novel regularized regression model of tree growth and investigated whether it reached the same conclusions as a commonly used neighborhood model, regarding hypotheses of how tree growth is influenced by the species identity of neighboring trees. We also evaluated the ability of both models to interpolate the growth of trees not included in the model fitting dataset. Our regularized regression model replicated most of the classical model’s inferences in a fraction of the time without using high-performance computing resources. We found that both methods could interpolate out-of-sample tree growth, but the method making the most accurate predictions varied among focal species. Regularized regression is particularly efficient for comparing hypotheses because it automates the process of model selection and can handle correlated explanatory variables. This feature means that regularized regression could also be used to select among potential explanatory variables (e.g., climate variables) and thereby streamline the development of a classical neighborhood model. Both regularized regression and classical methods can interpolate out-of-sample tree growth, but future research must determine whether predictions can be extrapolated to trees experiencing novel conditions. Overall, we conclude that regularized regression methods can complement classical methods in the investigation of tree growth drivers and represent a valuable tool for advancing this field toward prediction.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
