More Like this

1. Sequential Bayesian Network Structure Learning

   https://doi.org/10.1109/IEEECONF56349.2022.10052016  

   Ekanayake, Sachini Piyoni; Zois, Daphney-Stavroula (October 2022, 2022 56th Asilomar Conference on Signals, Systems, and Computers)
2. A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

   Yucen, Lily L; Rudner, Tim G; Wilson, Andrew G (May 2024, International Conference on Learning Representations)

   Bayesian optimization is a highly efficient approach to optimizing objective functions which are expensive to query. These objectives are typically represented by Gaussian process (GP) surrogate models which are easy to optimize and support exact inference. While standard GP surrogates have been well-established in Bayesian optimization, Bayesian neural networks (BNNs) have recently become practical function approximators, with many benefits over standard GPs such as the ability to naturally handle non-stationarity and learn representations for high-dimensional data. In this paper, we study BNNs as alternatives to standard GP surrogates for optimization. We consider a variety of approximate inference procedures for finite-width BNNs, including high-quality Hamiltonian Monte Carlo, low-cost stochastic MCMC, and heuristics such as deep ensembles. We also consider infinite-width BNNs, linearized Laplace approximations, and partially stochastic models such as deep kernel learning. We evaluate this collection of surrogate models on diverse problems with varying dimensionality, number of objectives, non-stationarity, and discrete and continuous inputs. We find: (i) the ranking of methods is highly problem dependent, suggesting the need for tailored inductive biases; (ii) HMC is the most successful approximate inference procedure for fully stochastic BNNs; (iii) full stochasticity may be unnecessary as deep kernel learning is relatively competitive; (iv) deep ensembles perform relatively poorly; (v) infinite-width BNNs are particularly promising, especially in high dimensions. 

   more » « less
3. On Robust Trimming of Bayesian Network Classifiers

   Choi, Y.; Van den Broeck, G. (January 2018, Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI))
4. Adversarial Examples Detection With Bayesian Neural Network

   https://doi.org/10.1109/TETCI.2024.3372383  

   Li, Yao; Tang, Tongyi; Hsieh, Cho-Jui; Lee, Thomas_C M (March 2024, IEEE Transactions on Emerging Topics in Computational Intelligence)

   In this paper, we propose a new framework to detect adversarial examples motivated by the observations that random components can improve the smoothness of predictors and make it easier to simulate the output distribution of a deep neural network. With these observations, we propose a novel Bayesian adversarial example detector, short for BATER, to improve the performance of adversarial example detection. Specifically, we study the distributional difference of hidden layer output between natural and adversarial examples, and propose to use the randomness of the Bayesian neural network to simulate hidden layer output distribution and leverage the distribution dispersion to detect adversarial examples. The advantage of a Bayesian neural network is that the output is stochastic while a deep neural network without random components does not have such characteristics. Empirical results on several benchmark datasets against popular attacks show that the proposed BATER outperforms the state-of-the-art detectors in adversarial example detection. 

   more » « less
5. Partitioned hybrid learning of Bayesian network structures

   https://doi.org/10.1007/s10994-022-06145-4  

   Huang, Jireh; Zhou, Qing (March 2022, Machine Learning)

   Abstract We develop a novel hybrid method for Bayesian network structure learning called partitioned hybrid greedy search (pHGS), composed of three distinct yet compatible new algorithms: Partitioned PC (pPC) accelerates skeleton learning via a divide-and-conquer strategy,p-value adjacency thresholding (PATH) effectively accomplishes parameter tuning with a single execution, and hybrid greedy initialization (HGI) maximally utilizes constraint-based information to obtain a high-scoring and well-performing initial graph for greedy search. We establish structure learning consistency of our algorithms in the large-sample limit, and empirically validate our methods individually and collectively through extensive numerical comparisons. The combined merits of pPC and PATH achieve significant computational reductions compared to the PC algorithm without sacrificing the accuracy of estimated structures, and our generally applicable HGI strategy reliably improves the estimation structural accuracy of popular hybrid algorithms with negligible additional computational expense. Our empirical results demonstrate the competitive empirical performance of pHGS against many state-of-the-art structure learning algorithms. 

   more » « less