An official website of the United States government
Abstract There has been a great deal of recent interest in the development of spatial prediction algorithms for very large datasets and/or prediction domains. These methods have primarily been developed in the spatial statistics community, but there has been growing interest in the machine learning community for such methods, primarily driven by the success of deep Gaussian process regression approaches and deep convolutional neural networks. These methods are often computationally expensive to train and implement and consequently, there has been a resurgence of interest in random projections and deep learning models based on random weights—so called reservoir computing methods. Here, we combine several of these ideas to develop the random ensemble deep spatial (REDS) approach to predict spatial data. The procedure uses random Fourier features as inputs to an extreme learning machine (a deep neural model with random weights), and with calibrated ensembles of outputs from this model based on different random weights, it provides a simple uncertainty quantification. The REDS method is demonstrated on simulated data and on a classic large satellite data set.
more »
« less
Dynamic Online Ensembles of Basis Expansions
Practical Bayesian learning often requires (1) online inference, (2) dynamic models, and (3) ensembling over multiple different models. Recent advances have shown how to use random feature approximations to achieve scalable, online ensembling of Gaussian processes with desirable theoretical properties and fruitful applications. One key to these methods’ success is the inclusion of a random walk on the model parameters, which makes models dynamic. We show that these methods can be generalized easily to any basis expansion model and that using alternative basis expansions, such as Hilbert space Gaussian processes, often results in better performance. To simplify the process of choosing a specific basis expansion, our method’s generality also allows the ensembling of several entirely different models, for example, a Gaussian process and polynomial regression. Finally, we propose a novel method to ensemble static and dynamic models together.
more »
« less
- Award ID(s):
- 2212506
- PAR ID:
- 10597631
- Publisher / Repository:
- The Journal of Machine Learning Research, Inc
- Date Published:
- Journal Name:
- Transactions on Machine Learning Research
- ISSN:
- 2835-8856.
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Hiemstra, D.; Moens, MF.; Mothe, J.; Perego, R.; Potthast, M.; Sebastiani, F. (Ed.)Our work aimed at experimentally assessing the benefits of model ensembling within the context of neural methods for passage re-ranking. Starting from relatively standard neural models, we use a previous technique named Fast Geometric Ensembling to generate multiple model instances from particular training schedules, then focusing or attention on different types of approaches for combining the results from the multiple model instances (e.g., averaging the ranking scores, using fusion methods from the IR literature, or using supervised learning-to-rank). Tests with the MS-MARCO dataset show that model ensembling can indeed benefit the ranking quality, particularly with supervised learning-to-rank although also with unsupervised rank aggregation.more » « less
-
null (Ed.)Across a wide variety of domains, artificial agents that can adapt and personalize to users have potential to improve and transform how social services are provided. Because of the need for personalized interaction data to drive this process, long-term (or longitudinal) interactions between users and agents, which unfold over a series of distinct interaction sessions, have attracted substantial research interest. In recognition of the expanded scope and structure of a long-term interaction, researchers are also adjusting the personalization models and algorithms used, orienting toward “continual learning” methods, which do not assume a stationary modeling target and explicitly account for the temporal context of training data. In parallel, researchers have also studied the effect of “multitask personalization,” an approach in which an agent interacts with users over multiple different tasks contexts throughout the course of a long-term interaction and learns personalized models of a user that are transferrable across these tasks. In this paper, we unite these two paradigms under the framework of “Lifelong Personalization,” analyzing the effect of multitask personalization applied to dynamic, non-stationary targets. We extend the multi-task personalization approach to the more complex and realistic scenario of modeling dynamic learners over time, focusing in particular on interactive scenarios in which the modeling agent plays an active role in teaching the student whose knowledge the agent is simultaneously attempting to model. Inspired by the way in which agents use active learning to select new training data based on domain context, we augment a Gaussian Process-based multitask personalization model with a mechanism to actively and continually manage its own training data, allowing a modeling agent to remove or reduce the weight of observed data from its training set, based on interactive context cues. We evaluate this method in a series of simulation experiments comparing different approaches to continual and multitask learning on simulated student data. We expect this method to substantially improve learning in Gaussian Process models in dynamic domains, establishing Gaussian Processes as another flexible modeling tool for Long-term Human-Robot Interaction (HRI) Studies.more » « less
-
Abstract Freeze nano 3D printing is a novel process that seamlessly integrates freeze casting and inkjet printing processes. It can fabricate flexible energy products with both macroscale and microscale features. These multi-scale features enable good mechanical and electrical properties with lightweight structures. However, the quality issues are among the biggest barriers that freeze nano printing, and other 3D printing processes, need to come through. In particular, the droplet solidification behavior is crucial for the product quality. The physical based heat transfer models are computationally inefficient for the online solidification time prediction during the printing process. In this paper, we integrate machine learning (i.e., tensor decomposition) methods and physical models to emulate the tensor responses of droplet solidification time from the physical based models. The tensor responses are factorized with joint tensor decomposition, and represented with low dimensional vectors. We then model these low dimensional vectors with Gaussian process models. We demonstrate the proposed framework for emulating the physical models of freeze nano 3D printing, which can help the future real-time process optimization.more » « less
-
Unlike traditional structural materials, soft solids can often sustain very large deformation before failure, and many exhibit nonlinear viscoelastic behavior. Modeling nonlinear viscoelasticity is a challenging problem for a number of reasons. In particular, a large number of material parameters are needed to capture material response and validation of models can be hindered by limited amounts of experimental data available. We have developed a Gaussian Process (GP) approach to determine the material parameters of a constitutive model describing the mechanical behavior of a soft, viscoelastic PVA hydrogel. A large number of stress histories generated by the constitutive model constitute the training sets. The low-rank representations of stress histories by Singular Value Decomposition (SVD) are taken to be random variables which can be modeled via Gaussian Processes with respect to the material parameters of the constitutive model. We obtain optimal material parameters by minimizing an objective function over the input set. We find that there are many good sets of parameters. Further the process reveals relationships between the model parameters. Results so far show that GP has great potential in fitting constitutive models.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
