More Like this

1. Enabling Hyperparameter Tuning of Machine Learning Classifiers in Production

   https://doi.org/10.1109/CogMI52975.2021.00041  

   Sandha, Sandeep Singh ; Aggarwal, Mohit ; Saha, Swapnil Sayan ; Srivastava, Mani ( December 2021 , 2021 IEEE Third International Conference on Cognitive Machine Intelligence (CogMI))

   Machine learning (ML) classifiers are widely adopted in the learning-enabled components of intelligent Cyber-physical Systems (CPS) and tools used in designing integrated circuits. Due to the impact of the choice of hyperparameters on an ML classifier performance, hyperparameter tuning is a crucial step for application success. However, the practical adoption of existing hyperparameter tuning frameworks in production is hindered due to several factors such as inflexible architecture, limitations of search algorithms, software dependencies, or closed source nature. To enable state-of-the-art hyperparameter tuning in production, we propose the design of a lightweight library (1) having a flexible architecture facilitating usage on arbitrary systems, and (2) providing parallel optimization algorithms supporting mixed parameters (continuous, integer, and categorical), handling runtime failures, and allowing combined classifier selection and hyperparameter tuning (CASH). We present Mango, a black-box optimization library, to realize the proposed design. Mango is currently used in production at Arm for more than 25 months and is available open-source (https://github.com/ARM-software/mango). Our evaluation shows that Mango outperforms other black-box optimization libraries in tuning hyperparameters of ML classifiers having mixed param-eter search spaces. We discuss two use cases of Mango deployed in production at Arm, highlighting its flexible architecture and ease of adoption. The first use case trains ML classifiers on the Dask cluster using Mango to find bugs in Arm's integrated circuits designs. As a second use case, we introduce an AutoML framework deployed on the Kubernetes cluster using Mango. Finally, we present the third use-case of Mango in enabling neural architecture search (NAS) to transfer deep neural networks to TinyML platforms (microcontroller class devices) used by CPS/IoT applications. 

   more » « less
2. COMPARATIVE ANALYSIS OF HYPERPARAMETER TUNING IN 3D PRINTING

   Michael Ogunsanya, Salil Desai ( January 2022 , International Conference on Science, Technology, Engineering and Management)

   This paper presents hyperparameter tuning techniques for a deep learning predictive model with applications in additive manufacturing processes. Bioprinting is an additive manufacturing process which utilizes biomaterials, cells, and growth factors to build functional tissue constructs for biomedical applications. In this research, we evaluate the hyperparameter space using grid search technique to tune the perceptron deep learning hyperparameters for optimal prediction of additive manufacturing outcomes. Hyperparameter entities include number of neurons, learning rate, and number of epochs to run machine learning models. Five input parameters and three output variables were evaluated for a typical additive manufacturing process. A comparative analysis is conducted to demonstrate improved runtime and lower root mean squared error for additive manufacturing predictive models. The results from this research are extensible to several additive manufacturing processes including 3D bioprinting. 

   more » « less
3. Training Deep Networks without Learning Rates Through Coin Betting

   Orabona, Francesco ; Tommasi, Tatiana ( January 2017 , Advances in Neural Information Processing Systems 30)

   Deep learning methods achieve state-of-the-art performance in many application scenarios. Yet, these methods require a significant amount of hyperparameters tuning in order to achieve the best results. In particular, tuning the learning rates in the stochastic optimization process is still one of the main bottlenecks. In this paper, we propose a new stochastic gradient descent procedure for deep networks that does not require any learning rate setting. Contrary to previous methods, we do not adapt the learning rates nor we make use of the assumed curvature of the objective function. Instead, we reduce the optimization process to a game of betting on a coin and propose a learning rate free optimal algorithm for this scenario. Theoretical convergence is proven for convex and quasi-convex functions and empirical evidence shows the advantage of our algorithm over popular stochastic gradient algorithms. 

   more » « less
4. Auto-ASD-Network: A Technique Based on Deep Learning and Support Vector Machines for Diagnosing Autism Spectrum Disorder using fMRI Data

   https://doi.org/10.1145/3307339.3343482  

   Eslami, Taban ; Saeed, Fahad ( January 2019 , Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics)

   Quantitative analysis of brain disorders such as Autism Spectrum Disorder (ASD) is an ongoing field of research. Machine learning and deep learning techniques have been playing an important role in automating the diagnosis of brain disorders by extracting discriminative features from the brain data. In this study, we propose a model called Auto-ASD-Network in order to classify subjects with Autism disorder from healthy subjects using only fMRI data. Our model consists of a multilayer perceptron (MLP) with two hidden layers. We use an algorithm called SMOTE for performing data augmentation in order to generate artificial data and avoid overfitting, which helps increase the classification accuracy. We further investigate the discriminative power of features extracted using MLP by feeding them to an SVM classifier. In order to optimize the hyperparameters of SVM, we use a technique called Auto Tune Models (ATM) which searches over the hyperparameter space to find the best values of SVM hyperparameters. Our model achieves more than 70% classification accuracy for 4 fMRI datasets with the highest accuracy of 80%. It improves the performance of SVM by 26%, the stand-alone MLP by 16% and the state of the art method in ASD classification by 14%. The implemented code will be available as GPL license on GitHub portal of our lab (https://github.com/PCDS). 

   more » « less
5. Towards Hyperparameter-free Policy Selection for Offline Reinforcement Learning

   Zhang, Siyuan ; Jiang, Nan ( December 2021 , Advances in neural information processing systems)

   How to select between policies and value functions produced by different training algorithms in offline reinforcement learning (RL)—which is crucial for hyperparameter tuning—is an important open question. Existing approaches based on off-policy evaluation (OPE) often require additional function approximation and hence hyperparameters, creating a chicken-and-egg situation. In this paper, we design hyperparameter-free algorithms for policy selection based on BVFT [XJ21], a recent theoretical advance in value-function selection, and demonstrate their effectiveness in discrete-action benchmarks such as Atari. To address performance degradation due to poor critics in continuous-action domains, we further combine BVFT with OPE to get the best of both worlds, and obtain a hyperparameter-tuning method for Q-function based OPE with theoretical guarantees as a side product. 

   more » « less