Search for: All records

Scientific Machine Learning (SciML) is a new multidisciplinary methodology that combines the data-driven machine learning models and the principle-based computational models to improve the simulations of scientific phenomenon and uncover new scientific rules from existing measurements. This article reveals the experience of using the SciML method to discover the nonlinear dynamics that may be hard to model or be unknown in the real-world scenario. The SciML method solves the traditional principle-based differential equations by integrating a neural network to accurately model the nonlinear dynamics while respecting the scientific constraints and principles. The paper discusses the latest SciML models and apply them to the oscillator simulations and experiment. Besides better capacity to simulate, and match with the observation, the results also demonstrate a successful discovery of the hidden physics in the pendulum dynamics using SciML. 

Scientific Machine Learning (SciML) is a new multidisciplinary methodology that combines the data-driven machine learning models and the principle-based computational models to improve the simulations of scientific phenomenon and uncover new scientific rules from existing measurements. This article reveals the experience of using the SciML method to discover the nonlinear dynamics that may be hard to model or be unknown in the real-world scenario. The SciML method solves the traditional principle-based differential equations by integrating a neural network to accurately model the nonlinear dynamics while respecting the scientific constraints and principles. The paper discusses the latest SciML models and apply them to the oscillator simulations and experiment. Besides better capacity to simulate, and match with the observation, the results also demonstrate a successful discovery of the hidden physics in the pendulum dynamics using SciML. 

The nonlinearity of activation functions used in deep learning models is crucial for the success of predictive models. Several simple nonlinear functions, including Rectified Linear Unit (ReLU) and Leaky-ReLU (L-ReLU) are commonly used in neural networks to impose the nonlinearity. In practice, these functions remarkably enhance the model accuracy. However, there is limited insight into the effects of nonlinearity in neural networks on their performance. Here, we investigate the performance of neural network models as a function of nonlinearity using ReLU and L-ReLU activation functions in the context of different model architectures and data domains. We use entropy as a measurement of the randomness, to quantify the effects of nonlinearity in different architecture shapes on the performance of neural networks. We show that the ReLU nonliearity is a better choice for activation function mostly when the network has sufficient number of parameters. However, we found that the image classification models with transfer learning seem to perform well with L-ReLU in fully connected layers. We show that the entropy of hidden layer outputs in neural networks can fairly represent the fluctuations in information loss as a function of nonlinearity. Furthermore, we investigate the entropy profile of shallow neural networks as a way of representing their hidden layer dynamics. 

Previously, symmetry of network models has been proposed to account for interocular grouping during binocular rivalry. Here, we construct and analyze generalized rivalry network models with different types of symmetry (based on different kinds of excitatory coupling) to derive predictions of possible perceptual states in 12 experiments with four retinal locations. Percepts in binocular rivalry involving more than three locations have not been empirically investigated due to the difficulty in reporting simultaneous percepts at multiple locations. Here, we develop a novel reporting procedure in which the stimulus disappears when the subject is cued to report the simultaneously perceived colors in all four retinal locations. This procedure ensures that simultaneous rather than sequential percepts are reported. The procedure was applied in 12 experiments with six binocular rivalry stimulus configurations, all consisting of dichoptic displays of red and green squares at four locations. We call configurations with an even or odd number of red squares even or odd configurations, respectively. In experiments using even stimulus configurations, we found that even percepts were more frequently observed than odd percepts, whereas in experiments using odd stimulus configurations even and odd percepts were observed with equal probability. The generalized rivalry network models in which couplings depend on stimulus features and spatial configurations was in better agreement with the empirical results. We conclude that the excitatory coupling strength in the horizontal and vertical configurations are different and the coupling strengths between the same color and between different colors are different. NEW & NOTEWORTHY Wilson network models of interocular groupings during binocular rivalry are constructed by considering features that indicate equal coupling strengths. Network symmetries, based on equal couplings, predict percepts. For a four-location rivalry experiment with red or green squares at each location, we analyze different possible Wilson networks. In our experiments we develop a novel reporting procedure and show that networks in which stimulus features and spatial configurations are distinguished best agree with experiments. 

The respiratory rhythm generator is spectacular in its ability to support a wide range of activities and adapt to changing environmental conditions, yet its operating mechanisms remain elusive. We show how selective control of inspiration and expiration times can be achieved in a new representation of the neural system (called a Boolean network). The new framework enables us to predict the behavior of neural networks based on properties of neurons, not their values. Hence, it reveals the logic behind the neural mechanisms that control the breathing pattern. Our network mimics many features seen in the respiratory network such as the transition from a 3-phase to 2-phase to 1-phase rhythm, providing novel insights and new testable predictions.