skip to main content

Explore Research Products in the NSF-PAR

Title: Hierarchically Modular Dynamical Neural Network Relaxing in a Warped Space: Basic Model and its Characteristics
We propose a hierarchically modular, dynamical neural network model whose architecture minimizes a specifically designed energy function and defines its temporal characteristics. The model has an internal and an external space that are connected with a layered internetwork that consists of a pair of forward and backward subnets composed of static neurons (with an instantaneous time-course). Dynamical neurons with large time constants in the internal space determine the overall time-course. The model offers a framework in which state variables in the network relax in a warped space, due to the cooperation between dynamic and static neurons. We assume that the system operates in either a learning or an association mode, depending on the presence or absence of feedback paths and input ports. In the learning mode, synaptic weights in the internetwork are modified by strong inputs corresponding to repetitive neuronal bursting, which represents sinusoidal or quasi-sinusoidal waves in the short-term average density of nerve impulses or in the membrane potential. A two-dimensional mapping relationship can be formed by employing signals with different frequencies based on the same mechanism as Lissajous curves. In the association mode, the speed of convergence to a goal point greatly varies with the mapping relationship of the previously trained internetwork, and owing to this property, the convergence trajectory in the two-dimensional model with the non-linear mapping internetwork cannot go straight but instead must curve. We further introduce a constrained association mode with a given target trajectory and elucidate that in the internal space, an output trajectory is generated, which is mapped from the external space according to the inverse of the mapping relationship of the forward subnet.  more » « less
Award ID(s):
1835202
NSF-PAR ID:
10394581
Author(s) / Creator(s):
Date Published:
Journal Name:
arXivorg
Volume:
arXiv:2211.11346
ISSN:
2331-8422
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ( , Emergent Behavior in System of Systems Engineering)
    Rainey, Larry B. ; Holland, O. Thomas (Ed.)
    Biological neural networks offer some of the most striking and complex examples of emergence ever observed in natural or man-made systems. Individually, the behavior of a single neuron is rather simple, yet these basic building blocks are connected through synapses to form neural networks, which are capable of sophisticated capabilities such as pattern recognition and navigation. Lower-level functionality provided by a given network is combined with other networks to produce more sophisticated capabilities. These capabilities manifest emergently at two vastly different, yet interconnected time scales. At the time scale of neural dynamics, neural networks are responsible for turning noisy external stimuli and internal signals into signals capable of supporting complex computations. A key component in this process is the structure of the network, which itself forms emergently over much longer time scales based on the outputs of its constituent neurons, a process called learning. The analysis and interpretation of the behaviors of these interconnected dynamical systems of neurons should account for the network structure and the collective behavior of the network. The field of graph signal processing (GSP) combines signal processing with network science to study signals defined on irregular network structures. Here, we show that GSP can be a valuable tool in the analysis of emergence in biological neural networks. Beyond any purely scientific pursuits, understanding the emergence in biological neural networks directly impacts the design of more effective artificial neural networks for general machine learning and artificial intelligence tasks across domains, and motivates additional design motifs for novel emergent systems of systems. 
    more » « less
  2. ; ( , Communications on Pure and Applied Mathematics)
    Abstract

    Koopman operators are infinite‐dimensional operators that globally linearize nonlinear dynamical systems, making their spectral information valuable for understanding dynamics. However, Koopman operators can have continuous spectra and infinite‐dimensional invariant subspaces, making computing their spectral information a considerable challenge. This paper describes data‐driven algorithms with rigorous convergence guarantees for computing spectral information of Koopman operators from trajectory data. We introduce residual dynamic mode decomposition (ResDMD), which provides the first scheme for computing the spectra and pseudospectra of general Koopman operators from snapshot data without spectral pollution. Using the resolvent operator and ResDMD, we compute smoothed approximations of spectral measures associated with general measure‐preserving dynamical systems. We prove explicit convergence theorems for our algorithms (including for general systems that are not measure‐preserving), which can achieve high‐order convergence even for chaotic systems when computing the density of the continuous spectrum and the discrete spectrum. Since our algorithms have error control, ResDMD allows aposteri verification of spectral quantities, Koopman mode decompositions, and learned dictionaries. We demonstrate our algorithms on the tent map, circle rotations, Gauss iterated map, nonlinear pendulum, double pendulum, and Lorenz system. Finally, we provide kernelized variants of our algorithms for dynamical systems with a high‐dimensional state space. This allows us to compute the spectral measure associated with the dynamics of a protein molecule with a 20,046‐dimensional state space and compute nonlinear Koopman modes with error bounds for turbulent flow past aerofoils with Reynolds number >105that has a 295,122‐dimensional state space.

     
    more » « less
  3. ; ; ( , Frontiers in Water)
    Introduction Integrated water management (IWM) involves a range of policies, actions, and organizational processes that go beyond traditional hydrology to consider multifaceted aspects of complex water resource systems. Due to its transdisciplinary nature, IWM comprises input from diverse stakeholders, each with unique perceptions, values, and experiences. However, stakeholders from differing backgrounds may disagree on best practices and collective paths forward. As such, successful IWM must address key governance principles (e.g., information flow, collective decision-making, and power relations) across social and institutional scales. Here, we sought to demonstrate how network structure impacts shared decision-making within IWM. Methods We explored a case study in Houston, Texas, USA, where decision-making stakeholders from various sectors and levels of governance engaged in a participatory modeling workshop to improve adoption of nature-based solutions (NBS) through IWM. The stakeholders used fuzzy cognitive mapping (FCM) to define an IWM model comprising multifaceted elements and their interrelationships, which influenced the adoption of NBS in Houston. We applied grounded theory and inductive reasoning to categorize tacit belief schemas regarding how stakeholders viewed themselves within the management system. We then used FCM-based modeling to explore how unique NBS policies would translate into more (or less) NBS adoption. Finally, we calculated specific network metrics (e.g., density, hierarchy, and centrality indices) to better understand the structure of human-water relations embedded within the IWM model. We compared the tacit assumptions about stakeholder roles in IWM against the quantitative degrees of influence and collectivism embedded within the stakeholder-defined model. Results and discussion Our findings revealed a mismatch between stakeholders' external belief statements about IWM and their internal assumptions through cognitive mapping and participatory modeling. The case study network was characterized by a limited degree of internal coordination (low density index), high democratic potential (low hierarchy index), and high-efficiency management opportunities (high centrality index), which transcended across socio-institutional scales. These findings contrasted with several of the belief schemas described by stakeholders during the group workshop. We describe how ongoing partnership with the stakeholders resulted in an opportunity for adaptive learning, where the NBS planning paradigm began to shift toward trans-scale collaboration aimed at high-leverage management opportunities. We emphasize how network analytics allowed us to better understand the extent to which key governance principles drove the behavior of the IWM model, which we leveraged to form deeper stakeholder partnerships by identifying hidden opportunities for governance transformation. 
    more » « less
  4. ; ; ; ; ; ; ( , Journal of Climate)
    null (Ed.)
    Abstract The Southern Hemisphere summertime eddy-driven jet and storm tracks have shifted poleward over the recent few decades. In previous studies, explanations have mainly stressed the influence of external forcing in driving this trend. Here we examine the role of internal tropical SST variability in controlling the austral summer jet’s poleward migration, with a focus on interdecadal time scales. The role of external forcing and internal variability are isolated by using a hierarchy of Community Earth System Model version 1 (CESM1) simulations, including the pre-industrial control, large ensemble, and pacemaker runs. Model simulations suggest that in the early twenty-first century, both external forcing and internal tropical Pacific SST variability are important in driving a positive southern annular mode (SAM) phase and a poleward migration of the eddy-driven jet. Tropical Pacific SST variability, associated with the negative phase of the interdecadal Pacific oscillation (IPO), acts to shift the jet poleward over the southern Indian and southwestern Pacific Oceans and intensify the jet in the southeastern Pacific basin, while external forcing drives a significant poleward jet shift in the South Atlantic basin. In response to both external forcing and decadal Pacific SST variability, the transient eddy momentum flux convergence belt in the middle latitudes experiences a poleward migration due to the enhanced meridional temperature gradient, leading to a zonally symmetric southward migration of the eddy-driven jet. This mechanism distinguishes the influence of the IPO on the midlatitude circulation from the dynamical impact of ENSO, with the latter mainly promoting the subtropical wave-breaking critical latitude poleward and pushing the midlatitude jet to higher latitudes. 
    more » « less
  5. ; ; ; ; ( , 4th Annual Conference on Learning for Dynamics and Control)
    Firoozi, R. ; Mehr, N. ; Yel, E. ; Antonova, R. ; Bohg, J. ; Schwager, M. ; Kochenderfer, M. (Ed.)
    Effective inclusion of physics-based knowledge into deep neural network models of dynamical sys- tems can greatly improve data efficiency and generalization. Such a priori knowledge might arise from physical principles (e.g., conservation laws) or from the system’s design (e.g., the Jacobian matrix of a robot), even if large portions of the system dynamics remain unknown. We develop a framework to learn dynamics models from trajectory data while incorporating a priori system knowledge as inductive bias. More specifically, the proposed framework uses physics-based side information to inform the structure of the neural network itself, and to place constraints on the values of the outputs and the internal states of the model. It represents the system’s vector field as a composition of known and unknown functions, the latter of which are parametrized by neural networks. The physics-informed constraints are enforced via the augmented Lagrangian method during the model’s training. We experimentally demonstrate the benefits of the proposed approach on a variety of dynamical systems – including a benchmark suite of robotics environments featur- ing large state spaces, non-linear dynamics, external forces, contact forces, and control inputs. By exploiting a priori system knowledge during training, the proposed approach learns to predict the system dynamics two orders of magnitude more accurately than a baseline approach that does not include prior knowledge, given the same training dataset. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email