More Like this

1. Neural Bootstrapper

   Shin, M. ; Cho, H. ; Min, H-S. ; Lim, S. ( January 2021 , Advances in neural information processing systems)

   Ranzato, M. ; Beygelzimer, A. ; Dauphin, Y. ; Liang, P. S. ; Wortman Vaughan, J. (Ed.)

   Bootstrapping has been a primary tool for ensemble and uncertainty quantification in machine learning and statistics. However, due to its nature of multiple training and resampling, bootstrapping deep neural networks is computationally burdensome; hence it has difficulties in practical application to the uncertainty estimation and related tasks. To overcome this computational bottleneck, we propose a novel approach called Neural Bootstrapper (NeuBoots), which learns to generate bootstrapped neural networks through single model training. NeuBoots injects the bootstrap weights into the high-level feature layers of the backbone network and outputs the bootstrapped predictions of the target, without additional parameters and the repetitive computations from scratch. We apply NeuBoots to various machine learning tasks related to uncertainty quantification, including prediction calibrations in image classification and semantic segmentation, active learning, and detection of out-of-distribution samples. Our empirical results show that NeuBoots outperforms other bagging based methods under a much lower computational cost without losing the validity of bootstrapping. 

   more » « less
2. Application of Convolutional Neural Network Algorithms for Advancing Sedentary and Activity Bout Classification

   https://doi.org/10.1123/jmpb.2020-0016  

   Nakandala, Supun ; Jankowska, Marta M. ; Tuz-Zahra, Fatima ; Bellettiere, John ; Carlson, Jordan A. ; LaCroix, Andrea Z. ; Hartman, Sheri J. ; Rosenberg, Dori E. ; Zou, Jingjing ; Kumar, Arun ; et al ( January 2020 , Journal for the Measurement of Physical Behaviour)

   null (Ed.)

   Background : Machine learning has been used for classification of physical behavior bouts from hip-worn accelerometers; however, this research has been limited due to the challenges of directly observing and coding human behavior “in the wild.” Deep learning algorithms, such as convolutional neural networks (CNNs), may offer better representation of data than other machine learning algorithms without the need for engineered features and may be better suited to dealing with free-living data. The purpose of this study was to develop a modeling pipeline for evaluation of a CNN model on a free-living data set and compare CNN inputs and results with the commonly used machine learning random forest and logistic regression algorithms. Method : Twenty-eight free-living women wore an ActiGraph GT3X+ accelerometer on their right hip for 7 days. A concurrently worn thigh-mounted activPAL device captured ground truth activity labels. The authors evaluated logistic regression, random forest, and CNN models for classifying sitting, standing, and stepping bouts. The authors also assessed the benefit of performing feature engineering for this task. Results : The CNN classifier performed best (average balanced accuracy for bout classification of sitting, standing, and stepping was 84%) compared with the other methods (56% for logistic regression and 76% for random forest), even without performing any feature engineering. Conclusion : Using the recent advancements in deep neural networks, the authors showed that a CNN model can outperform other methods even without feature engineering. This has important implications for both the model’s ability to deal with the complexity of free-living data and its potential transferability to new populations. 

   more » « less
3. A Semi-Supervised and Inductive Embedding Model for Churn Prediction of Large-Scale Mobile Games

   https://doi.org/10.1109/ICDM.2018.00043  

   Liu, Xi ; Xie, Muhe ; Wen, Xidao ; Chen, Rui ; Ge, Yong ; Duffield, Nick ; Wang, Na ( November 2018 , 2018 IEEE International Conference on Data Mining (ICDM 2018))

   Mobile gaming has emerged as a promising market with billion-dollar revenues. A variety of mobile game platforms and services have been developed around the world. One critical challenge for these platforms and services is to understand user churn behavior in mobile games. Accurate churn prediction will bene t many stakeholders such as game developers, advertisers, and platform operators. In this paper, we present the rst large- scale churn prediction solution for mobile games. In view of the common limitations of the state-of-the-art methods built upon traditional machine learning models, we devise a novel semi- supervised and inductive embedding model that jointly learns the prediction function and the embedding function for user- app relationships. We model these two functions by deep neural networks with a unique edge embedding technique that is able to capture both contextual information and relationship dynamics. We also design a novel attributed random walk technique that takes into consideration both topological adjacency and attribute similarities. To evaluate the performance of our solution, we collect real-world data from the Samsung Game Launcher platform that includes tens of thousands of games and hundreds of millions of user-app interactions. The experimental results with this data demonstrate the superiority of our proposed model against existing state-of-the-art methods. 

   more » « less
4. Deep Learning Approach to Link Weight Prediction

   Hou, Yuchen ; Holder, Lawrence B ( May 2017 , International Joint Conference on Neural Networks (IJCNN))

   Deep learning has been successful in various domains including image recognition, speech recognition and natural language processing. However, the research on its application in graph mining is still in an early stage. Here we present Model R, a neural network model created to provide a deep learning approach to link weight prediction problem. This model extracts knowledge of nodes from known links’ weights and uses this knowledge to predict unknown links’ weights. We demonstrate the power of Model R through experiments and compare it with stochastic block model and its derivatives. Model R shows that deep learning can be successfully applied to link weight prediction and it outperforms stochastic block model and its derivatives by up to 73% in terms of prediction accuracy. We anticipate this new approach to provide effective solutions to more graph mining tasks. 

   more » « less
5. A Machine Learning Tutorial for Operational Meteorology. Part II: Neural Networks and Deep Learning

   https://doi.org/10.1175/WAF-D-22-0187.1  

   Chase, Randy J. ; Harrison, David R. ; Lackmann, Gary M. ; McGovern, Amy ( August 2023 , Weather and Forecasting)

   Over the past decade the use of machine learning in meteorology has grown rapidly. Specifically neural networks and deep learning have been used at an unprecedented rate. To fill the dearth of resources covering neural networks with a meteorological lens, this paper discusses machine learning methods in a plain language format that is targeted to the operational meteorological community. This is the second paper in a pair that aim to serve as a machine learning resource for meteorologists. While the first paper focused on traditional machine learning methods (e.g., random forest), here a broad spectrum of neural networks and deep learning methods is discussed. Specifically, this paper covers perceptrons, artificial neural networks, convolutional neural networks, and U-networks. Like the Part I paper, this manuscript discusses the terms associated with neural networks and their training. Then the manuscript provides some intuition behind every method and concludes by showing each method used in a meteorological example of diagnosing thunderstorms from satellite images (e.g., lightning flashes). This paper is accompanied with an open-source code repository to allow readers to explore neural networks using either the dataset provided (which is used in the paper) or as a template for alternate datasets.

    

   more » « less