An official website of the United States government
Abstract A state‐space representation of water quality (WQ) dynamics describing disinfectant (e.g., chlorine) transport dynamics in drinking water distribution networks has been recently proposed. Such representation is a byproduct of space‐ and time‐discretization of the partial differential equations modeling transport dynamics. This results in a large state‐space dimension even for small networks with tens of nodes. Although such a state‐space model provides a model‐driven approach to predict WQ dynamics, incorporating it into model‐based control algorithms or state estimators for large networks is challenging and at times intractable. To that end, this paper investigates model order reduction (MOR) methods for WQ dynamics with the objective of performing post‐reduction feedback control. The presented investigation focuses on reducing state‐dimension by orders of magnitude, the stability of the MOR methods, and the application of these methods to model predictive control.
more »
« less
Network Restructuring Control for Conic Invariance with Application to Neural Networks
Recent advances in the study of artificial and biological neural networks support the power of dynamic representations-computing with information stored as nontrivial limit-sets rather than fixed-point attractors. Understanding and manipulating these computations in nonlinear networks requires a theory of control for abstract objective functions. Towards this end, we consider two properties of limit-sets: their topological dimension and orientation (covariance) in phase space and combine these abstract properties into a single well-defined objective: conic control-invariant sets in the derivative space (i.e., the vector field). Real-world applications, such as neural-medicine, constrain which control laws are feasible with less-invasive controllers being preferable. To this end, we derive a feedback control-law for conic invariance which corresponds to constrained restructuring of the network connections as might occur with pharmacological intervention (as opposed to a physically separate control unit). We demonstrate the ease and efficacy of the technique in controlling the orientation and dimension of limit sets in high-dimensional neural networks.
more »
« less
- Award ID(s):
- 1653589
- PAR ID:
- 10107627
- Date Published:
- Journal Name:
- Conference on Decision and Control
- Page Range / eLocation ID:
- 2704 to 2709
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
ABSTRACT Studies of cosmology, galaxy evolution, and astronomical transients with current and next-generation wide-field imaging surveys like the Rubin Observatory Legacy Survey of Space and Time are all critically dependent on estimates of photometric redshifts. Capsule networks are a new type of neural network architecture that is better suited for identifying morphological features of the input images than traditional convolutional neural networks. We use a deep capsule network trained on ugriz images, spectroscopic redshifts, and Galaxy Zoo spiral/elliptical classifications of ∼400 000 Sloan Digital Sky Survey galaxies to do photometric redshift estimation. We achieve a photometric redshift prediction accuracy and a fraction of catastrophic outliers that are comparable to or better than current methods for SDSS main galaxy sample-like data sets (r ≤ 17.8 and zspec ≤ 0.4) while requiring less data and fewer trainable parameters. Furthermore, the decision-making of our capsule network is much more easily interpretable as capsules act as a low-dimensional encoding of the image. When the capsules are projected on a two-dimensional manifold, they form a single redshift sequence with the fraction of spirals in a region exhibiting a gradient roughly perpendicular to the redshift sequence. We perturb encodings of real galaxy images in this low-dimensional space to create synthetic galaxy images that demonstrate the image properties (e.g. size, orientation, and surface brightness) encoded by each dimension. We also measure correlations between galaxy properties (e.g. magnitudes, colours, and stellar mass) and each capsule dimension. We publicly release our code, estimated redshifts, and additional catalogues at https://biprateep.github.io/encapZulate-1.more » « less
-
Exploding and vanishing gradient are both major problems often faced when an artificial neural network is trained with gradient descent. Inspired by the ubiquity and robustness of nonlinear oscillations in biological neural systems, we investigate the properties of their artificial counterpart, the stable limit cycle neural networks. Using a continuous time dynamical system interpretation of neural networks and backpropagation, we show that stable limit cycle neural networks have non-exploding gradients, and at least one effective non-vanishing gradient dimension. We conjecture that limit cycles can support the learning of long temporal dependence in both biological and artificial neural networks.more » « less
-
Many machine learning applications involve learning a latent representation of data, which is often high-dimensional and difficult to directly interpret. In this work, we propose “moment pooling,” a natural extension of deep sets networks which drastically decreases the latent space dimensionality of these networks while maintaining or even improving performance. Moment pooling generalizes the summation in deep sets to arbitrary multivariate moments, which enables the model to achieve a much higher effective latent dimensionality for a fixed learned latent space dimension. We demonstrate moment pooling on the collider physics task of quark/gluon jet classification by extending energy flow networks (EFNs) to moment EFNs. We find that moment EFNs with latent dimensions as small as 1 perform similarly to ordinary EFNs with higher latent dimension. This small latent dimension allows for the internal representation to be directly visualized and interpreted, which in turn enables the learned internal jet representation to be extracted in closed form. Published by the American Physical Society2024more » « less
-
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problems with only noisy objective function evaluations. Toward this, our main contribution is to propose estimators of the Riemannian gradient and Hessian from noisy objective function evaluations, based on a Riemannian version of the Gaussian smoothing technique. The proposed estimators overcome the difficulty of nonlinearity of the manifold constraint and issues that arise in using Euclidean Gaussian smoothing techniques when the function is defined only over the manifold. We use the proposed estimators to solve Riemannian optimization problems in the following settings for the objective function: (i) stochastic and gradient-Lipschitz (in both nonconvex and geodesic convex settings), (ii) sum of gradient-Lipschitz and nonsmooth functions, and (iii) Hessian-Lipschitz. For these settings, we analyze the oracle complexity of our algorithms to obtain appropriately defined notions of ϵ-stationary point or ϵ-approximate local minimizer. Notably, our complexities are independent of the dimension of the ambient Euclidean space and depend only on the intrinsic dimension of the manifold under consideration. We demonstrate the applicability of our algorithms by simulation results and real-world applications on black-box stiffness control for robotics and black-box attacks to neural networks.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/CDC.2018.8618729
