An official website of the United States government
Joint Nonnegative Matrix Factorization (JointNMF) is a hybrid method for mining information from datasets that contain both feature and connection information. We propose distributed-memory parallelizations of three algorithms for solving the JointNMF problem based on Alternating Nonnegative Least Squares, Projected Gradient Descent, and Projected Gauss-Newton. We extend well-known communication-avoiding algorithms using a single processor grid case to our coupled case on two processor grids. We demonstrate the scalability of the algorithms on up to 960 cores (40 nodes) with 60\% parallel efficiency. The more sophisticated Alternating Nonnegative Least Squares (ANLS) and Gauss-Newton variants outperform the first-order gradient descent method in reducing the objective on large-scale problems. We perform a topic modelling task on a large corpus of academic papers that consists of over 37 million paper abstracts and nearly a billion citation relationships, demonstrating the utility and scalability of the methods.
more »
« less
Graph Convolutional Networks for Power System State Estimation
Power system state estimation (PSSE) aims at finding the voltage magnitudes and angles at all generation and load buses, using meter readings and other available information. PSSE is often formulated as a nonconvex and nonlinear least-squares (NLS) cost function, which is traditionally solved by the Gauss-Newton method. However, Gauss-Newton iterations for minimizing nonconvex problems are sensitive to the initialization, and they can diverge. In this context, we advocate a deep neural network (DNN) based “trainable regularizer” to incorporate prior information for accurate and reliable state estimation. The resulting regularized NLS does not admit a neat closed form solution. To handle this, a novel end-to-end DNN is constructed subsequently by unrolling a Gauss-Newton-type solver which alternates between least-squares loss and the regularization term. Our DNN architecture can further offer a suite of advantages, e.g., accommodating network topology via graph neural networks based prior. Numerical tests using real load data on the IEEE 118-bus benchmark system showcase the improved estimation performance of the proposed scheme compared with state-of-the-art alternatives. Interestingly, our results suggest that a simple feed forward network based prior implicitly exploits the topology information hidden in data.
more »
« less
- Award ID(s):
- 1901134
- PAR ID:
- 10273951
- Date Published:
- Journal Name:
- Proceedings of IEEE Smartgridcom Conference
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this paper we propose a quasi-Newton algorithm for the celebrated nonnegative matrix factorization (NMF) problem. The proposed algorithm falls into the general framework of Gauss-Newton and Levenberg-Marquardt methods. However, these methods were not able to handle constraints, which is present in NMF. One of the key contributions in this paper is to apply alternating direction method of multipliers (ADMM) to obtain the iterative update from this Gauss-Newton-like algorithm. Furthermore, we carefully study the structure of the Jacobian Gramian matrix given by the Gauss-Newton updates, and designed a way of exactly inverting the matrix with complexity $$\cO(mnk)$$, which is a significant reduction compared to the naive implementation of complexity $$\cO((m+n)^3k^3)$$. The resulting algorithm, which we call NLS-ADMM, enjoys fast convergence rate brought by the quasi-Newton algorithmic framework, while maintaining low per-iteration complexity similar to that of alternating algorithms. Numerical experiments on synthetic data confirms the efficiency of our proposed algorithm. \end{abstract}more » « less
-
We present efficient algorithms to learn the pa- rameters governing the dynamics of networked agents, given equilibrium steady state data. A key feature of our methods is the ability to learn without seeing the dynamics, using only the steady states. A key to the efficiency of our approach is the use of mean-field approximations to tune the parameters within a nonlinear least squares (NLS) framework. Our results on real networks demonstrate the accuracy of our approach in two ways. Using the learned parameters, we can: (i) Recover more accurate estimates of the true steady states when the observed steady states are noisy. (ii) Predict evolution to new equilibrium steady states after perturbations to the network topology.more » « less
-
Abstract A parameter identification inverse problem in the form of nonlinear least squares is considered.In the lack of stability, the frozen iteratively regularized Gauss–Newton (FIRGN) algorithm is proposed and its convergence is justified under what we call a generalized normal solvability condition.The penalty term is constructed based on a semi-norm generated by a linear operator yielding a greater flexibility in the use of qualitative and quantitative a priori information available for each particular model.Unlike previously known theoretical results on the FIRGN method, our convergence analysis does not rely on any nonlinearity conditions and it is applicable to a large class of nonlinear operators.In our study, we leverage the nature of ill-posedness in order to establish convergence in the noise-free case.For noise contaminated data, we show that, at least theoretically, the process does not require a stopping rule and is no longer semi-convergent.Numerical simulations for a parameter estimation problem in epidemiology illustrate the efficiency of the algorithm.more » « less
-
This study evaluates the performance of multiple machine learning (ML) algorithms and electrical resistivity (ER) arrays for inversion with comparison to a conventional Gauss-Newton numerical inversion method. Four different ML models and four arrays were used for the estimation of only six variables for locating and characterizing hypothetical subsurface targets. The combination of dipole-dipole with Multilayer Perceptron Neural Network (MLP-NN) had the highest accuracy. Evaluation showed that both MLP-NN and Gauss-Newton methods performed well for estimating the matrix resistivity while target resistivity accuracy was lower, and MLP-NN produced sharper contrast at target boundaries for the field and hypothetical data. Both methods exhibited comparable target characterization performance, whereas MLP-NN had increased accuracy compared to Gauss-Newton in prediction of target width and height, which was attributed to numerical smoothing present in the Gauss-Newton approach. MLP-NN was also applied to a field dataset acquired at U.S. DOE Hanford site.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
