More Like this

1. Block-Term Tensor Decomposition Via Constrained Matrix Factorization

   https://doi.org/10.1109/MLSP.2019.8918708  

   Fu, Xiao; Huang, Kejun (October 2019, 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP))

   In this paper we propose a quasi-Newton algorithm for the celebrated nonnegative matrix factorization (NMF) problem. The proposed algorithm falls into the general framework of Gauss-Newton and Levenberg-Marquardt methods. However, these methods were not able to handle constraints, which is present in NMF. One of the key contributions in this paper is to apply alternating direction method of multipliers (ADMM) to obtain the iterative update from this Gauss-Newton-like algorithm. Furthermore, we carefully study the structure of the Jacobian Gramian matrix given by the Gauss-Newton updates, and designed a way of exactly inverting the matrix with complexity $$\cO(mnk)$$, which is a significant reduction compared to the naive implementation of complexity $$\cO((m+n)^3k^3)$$. The resulting algorithm, which we call NLS-ADMM, enjoys fast convergence rate brought by the quasi-Newton algorithmic framework, while maintaining low per-iteration complexity similar to that of alternating algorithms. Numerical experiments on synthetic data confirms the efficiency of our proposed algorithm. 

   more » « less
2. Fast Node Communication ADMM-based Imaging Algorithm with a Compressive Reflector Antenna

   https://doi.org/10.1109/APUSNCURSINRSM.2018.8609046  

   Heredia-Juesas, Juan; Molaei, Ali; Tirado, Luis; Martinez-Lorenzo, Jose A. (July 2018, 2018 IEEE International Symposium on Antennas and Propagation & USNC/URSI National Radio Science Meeting)

   This paper presents a norm-1 regularized algorithm, based on the Alternating Direction Method of Multipliers(ADMM), in which the sensing matrix is divided by columns. This technique is based on sectioning the imaging domain into different regions and optimizing them in distributed computational nodes.The information shared among nodes is highly reduced compared to the consensus-based ADMM, when dividing the matrix by rows. The combination of the sectioning-based ADMM with the imaging capabilities of the recently proposed Compressive Reflector Antenna allows a distributed, real-time imaging with fast node communication. 

   more » « less
3. Distributed-Memory Parallel JointNMF

   https://doi.org/10.1145/3577193.3593733  

   Eswar, Srinivas; Cobb, Benjamin; Hayashi, Koby; Kannan, Ramakrishnan; Ballard, Grey; Vuduc, Richard; Park, Haesun (June 2023, Proceedings of the 37th International Conference on Supercomputing)

   Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60\% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods. 

   more » « less
4. An Enhanced SD-GS-AL Algorithm for Coordinating the Optimal Power and Traffic Flows With EVs

   https://doi.org/10.1109/TSG.2024.3358805  

   Sharma, Santosh; Li, Qifeng; Wei, Wei (July 2024, IEEE Transactions on Smart Grid)

   The electric power distribution network (PDN) and the transportation network (TN) are generally operated/coordinated by different entities. However, they are coupled through electric vehicle charging stations (EVCSs). This paper proposes to coordinate the operation of the two systems via a fully decentralized framework where the PDN and TN operators solve their own operation problems independently, with only limited information exchange. Nevertheless, the operation problems of both systems are generally mixed-integer programs (MIP), for which mature algorithms like the alternating direction method of multipliers (ADMM) may not guarantee convergence. This paper applies a novel distributed optimization algorithm called the SD-GS-AL method, which is a combination of the simplicial decomposition, gauss-seidel, and augmented Lagrangian, which can guarantee convergence and optimality for MIPs. However, the original SD-GS-AL may be computationally inefficient for solving a complex engineering problem like the PDN-TN coordinated optimization investigated in this paper. To improve the computational efficiency, an enhanced SD-GS-AL method is proposed by redesigning the inner loop of the algorithm, which can automatically and intelligently determine the iteration number of the inner loop. Simulations on the test cases show the efficiency and efficacy of the proposed framework and algorithm. 

   more » « less
5. Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation

   https://doi.org/10.1109/ASP-DAC47756.2020.9045658  

   Ma, Xiaolong; Yuan, Geng; Lin, Sheng; Ding, Caiwen; Yu, Fuxun; Liu, Tao; Wen, Wujie; Chen, Xiang; Wang, Yanzhi (January 2020, 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC))

   The memristor crossbar array has emerged as an intrinsically suitable matrix computation and low-power acceleration framework for DNN applications. Many techniques such as memristor-based weight pruning and memristor-based quantization have been studied. However, the high accuracy solution for the above techniques is still waiting for unraveling. In this paper, we propose a memristor-based DNN framework which combines both structured weight pruning and quantization by incorporating ADMM algorithm for better pruning and quantization performance. We also discover the non-optimality of the ADMM solution in weight pruning and the unused data path in a structured pruned model. We design a software-hardware co-optimization framework which contains the first proposed Network Purification and Unused Path Removal algorithms targeting on post-processing a structured pruned model after ADMM steps. By taking memristor hardware constraints into our whole framework, we achieve extreme high compression rate with minimum accuracy loss. For quantizing structured pruned model, our framework achieves nearly no accuracy loss after quantizing weights to 8-bit memristor weight representation. We share our models at anonymous link https://bit.ly/2VnMUy0. 

   more » « less