More Like this

1. Highly scalable maximum likelihood and conjugate Bayesian inference for ERGMs on graph sets with equivalent vertices

   https://doi.org/10.1371/journal.pone.0273039  

   Yin, Fan; Butts, Carter T. (August 2022, PLOS ONE)

   De Vico Fallani, Fabrizio (Ed.)

   The exponential family random graph modeling (ERGM) framework provides a highly flexible approach for the statistical analysis of networks (i.e., graphs). As ERGMs with dyadic dependence involve normalizing factors that are extremely costly to compute, practical strategies for ERGMs inference generally employ a variety of approximations or other workarounds. Markov Chain Monte Carlo maximum likelihood (MCMC MLE) provides a powerful tool to approximate the maximum likelihood estimator (MLE) of ERGM parameters, and is generally feasible for typical models on single networks with as many as a few thousand nodes. MCMC-based algorithms for Bayesian analysis are more expensive, and high-quality answers are challenging to obtain on large graphs. For both strategies, extension to the pooled case—in which we observe multiple networks from a common generative process—adds further computational cost, with both time and memory scaling linearly in the number of graphs. This becomes prohibitive for large networks, or cases in which large numbers of graph observations are available. Here, we exploit some basic properties of the discrete exponential families to develop an approach for ERGM inference in the pooled case that (where applicable) allows an arbitrarily large number of graph observations to be fit at no additional computational cost beyond preprocessing the data itself. Moreover, a variant of our approach can also be used to perform Bayesian inference under conjugate priors, again with no additional computational cost in the estimation phase. The latter can be employed either for single graph observations, or for observations from graph sets. As we show, the conjugate prior is easily specified, and is well-suited to applications such as regularization. Simulation studies show that the pooled method leads to estimates with good frequentist properties, and posterior estimates under the conjugate prior are well-behaved. We demonstrate the usefulness of our approach with applications to pooled analysis of brain functional connectivity networks and to replicated x-ray crystal structures of hen egg-white lysozyme. 

   more » « less
2. Adaptive Monte Carlo augmented with normalizing flows

   https://doi.org/10.1073/pnas.2109420119  

   Gabrié, Marylou; Rotskoff, Grant M.; Vanden-Eijnden, Eric (March 2022, Proceedings of the National Academy of Sciences)

   Many problems in the physical sciences, machine learning, and statistical inference necessitate sampling from a high-dimensional, multimodal probability distribution. Markov Chain Monte Carlo (MCMC) algorithms, the ubiquitous tool for this task, typically rely on random local updates to propagate configurations of a given system in a way that ensures that generated configurations will be distributed according to a target probability distribution asymptotically. In high-dimensional settings with multiple relevant metastable basins, local approaches require either immense computational effort or intricately designed importance sampling strategies to capture information about, for example, the relative populations of such basins. Here, we analyze an adaptive MCMC, which augments MCMC sampling with nonlocal transition kernels parameterized with generative models known as normalizing flows. We focus on a setting where there are no preexisting data, as is commonly the case for problems in which MCMC is used. Our method uses 1) an MCMC strategy that blends local moves obtained from any standard transition kernel with those from a generative model to accelerate the sampling and 2) the data generated this way to adapt the generative model and improve its efficacy in the MCMC algorithm. We provide a theoretical analysis of the convergence properties of this algorithm and investigate numerically its efficiency, in particular in terms of its propensity to equilibrate fast between metastable modes whose rough location is known a priori but respective probability weight is not. We show that our algorithm can sample effectively across large free energy barriers, providing dramatic accelerations relative to traditional MCMC algorithms. 

   more » « less
3. Big Data Aware Virtual Machine Placement in Cloud Data Centers

   https://doi.org/10.1145/3148055.3148057  

   Hall, Logan; Harris, Bryan; Tomes, Erica; Altiparmak, Nihat (December 2017, Proceedings of the Fourth IEEE/ACM International Conference on Big Data Computing, Applications and Technologies)

   While society continues to be transformed by insights from processing big data, the increasing rate at which this data is gathered is making processing in private clusters obsolete. A vast amount of big data already resides in the cloud, and cloud infrastructures provide a scalable platform for both the computational and I/O needs of big data processing applications. Virtualization is used as a base technology in the cloud; however, existing virtual machine placement techniques do not consider data replication and I/O bottlenecks of the infrastructure, yielding sub-optimal data retrieval times. This paper targets efficient big data processing in the cloud and proposes novel virtual machine placement techniques, which minimize data retrieval time by considering data replication, storage performance, and network bandwidth. We first present an integer-programming based optimal virtual machine placement algorithm and then propose two low cost data- and energy-aware virtual machine placement heuristics. Our proposed heuristics are compared with optimal and existing algorithms through extensive evaluation. Experimental results provide strong indications for the superiority of our proposed solutions in both performance and energy, and clearly outline the importance of big data aware virtual machine placement for efficient processing of large datasets in the cloud. 

   more » « less
4. Distributed Kriging-Bootstrapped DNN Model for Fast, Accurate Seizure Detection from EEG Signals

   Olokodana, Ibrahim; Mohanty, Saraju; Kougianos, Elias (January 2020, Proceedings of the 19th IEEE Computer Society Annual Symposium on VLSI (ISVLSI))

   The modeling of the brain as a three-dimensional spatial object, similar to a geographical landscape, has the paved way for the successful application of Kriging methods in solving the seizure detection problem with good performance but in cubic computational time complexity. The Deep Neural Network (DNN) has been widely used for seizure detection due to its effectiveness in classification tasks, although at the cost of a protracted training time. While Kriging exploits the spatial correlation between data locations, DNN relies on its capacity to learn intrinsic representations within the dataset from the basest unit parts. This paper presents a Distributed Kriging-Bootstrapped Deep Neural Network (DNN) model as a twofold solution for fast and accurate seizure detection using brain signals collected with the electroencephalogram (EEG) from healthy subjects and patients of epilepsy. The proposed model parallelizes the Kriging computation into different cores in a machine and then produces a strongly correlated, unified quasi-output data which serves as an input to the Deep Neural Network. Experimental results validate the proposed model as superior to conventional Kriging methods and DNN by training in 91% less time than the basic DNN and about three times as fast as the ordinary Kriging-Bootstrapped Deep Neural Network (DNN) model while maintaining good performance in terms of sensitivity, specificity and testing accuracy compared to other models and existing works. 

   more » « less
5. Energy-efficient, high-performance, highly-compressed deep neural network design using block-circulant matrices

   https://doi.org/10.1109/ICCAD.2017.8203813  

   Liao, Siyu; Li, Zhe; Lin, Xue; Qiu, Qinru; Wang, Yanzhi; Yuan, Bo (November 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD))

   Deep neural networks (DNNs) have emerged as the most powerful machine learning technique in numerous artificial intelligent applications. However, the large sizes of DNNs make themselves both computation and memory intensive, thereby limiting the hardware performance of dedicated DNN accelerators. In this paper, we propose a holistic framework for energy-efficient high-performance highly-compressed DNN hardware design. First, we propose block-circulant matrix-based DNN training and inference schemes, which theoretically guarantee Big-O complexity reduction in both computational cost (from O(n2) to O(n log n)) and storage requirement (from O(n2) to O(n)) of DNNs. Second, we dedicatedly optimize the hardware architecture, especially on the key fast Fourier transform (FFT) module, to improve the overall performance in terms of energy efficiency, computation performance and resource cost. Third, we propose a design flow to perform hardware-software co-optimization with the purpose of achieving good balance between test accuracy and hardware performance of DNNs. Based on the proposed design flow, two block-circulant matrix-based DNNs on two different datasets are implemented and evaluated on FPGA. The fixed-point quantization and the proposed block-circulant matrix-based inference scheme enables the network to achieve as high as 3.5 TOPS computation performance and 3.69 TOPS/W energy efficiency while the memory is saved by 108X ~ 116X with negligible accuracy degradation. 

   more » « less