Search for: All records

Abstract Imagine sitting at your desk, looking at objects on it. You do not know their exact distances from your eye in meters, but you can immediately reach out and touch them. Instead of an externally defined unit, your sense of distance is tied to your action’s embodiment. In contrast, conventional robotics relies on precise calibration to external units, with which vision and control processes communicate. We introduceEmbodied Visuomotor Representation, a methodology for inferring distance in a unit implied by action. With it a robot without knowledge of its size, environmental scale, or strength can quickly learn to touch and clear obstacles within seconds of operation. Likewise, in simulation, an agent without knowledge of its mass or strength can successfully jump across a gap of unknown size after a few test oscillations. These behaviors mirror natural strategies observed in bees and gerbils, which also lack calibration in an external unit. 

Abstract A neuromorphic simultaneous localization and mapping (SLAM) system shows potential for more efficient implementation than its traditional counterpart. At the mean time a neuromorphic model of spatial encoding neurons in silicon could provide insights on the functionality and dynamic between each group of cells. Especially when realistic factors including variations and imperfections on the neural movement encoding are presented to challenge the existing hypothetical models for localization. We demonstrate a mixed-mode implementation for spatial encoding neurons including theta cells, egocentric place cells, and the typical allocentric place cells. Together, they form a biologically plausible network that could reproduce the localization functionality of place cells observed in rodents. The system consists of a theta chip with 128 theta cell units and an FPGA implementing 4 networks for egocentric place cells formation that provides the capability for tracking on a 11 by 11 place cell grid. Experimental results validate the robustness of our model when suffering from as much as 18% deviation, induced by parameter variations in analog circuits, from the mathematical model of theta cells. We provide a model for implementing dynamic neuromorphic SLAM systems for dynamic-scale mapping of cluttered environments, even when subject to significant errors in sensory measurements and real-time analog computation. We also suggest a robust approach for the network topology of spatial cells that can mitigate neural non-uniformity and provides a hypothesis for the function of grid cells and the existence of egocentric place cells. 

Abstract Plasticity and homeostatic mechanisms allow neural networks to maintain proper function while responding to physiological challenges. Despite previous work investigating morphological and synaptic effects of brain-derived neurotrophic factor (BDNF), the most prevalent growth factor in the central nervous system, how exposure to BDNF manifests at the network level remains unknown. Here we report that BDNF treatment affects rodent hippocampal network dynamics during development and recovery from glutamate-induced excitotoxicity in culture. Importantly, these effects are not obvious when traditional activity metrics are used, so we delve more deeply into network organization, functional analyses, and in silico simulations. We demonstrate that BDNF partially restores homeostasis by promoting recovery of weak and medium connections after injury. Imaging and computational analyses suggest these effects are caused by changes to inhibitory neurons and connections. From our in silico simulations, we find that BDNF remodels the network by indirectly strengthening weak excitatory synapses after injury. Ultimately, our findings may explain the difficulties encountered in preclinical and clinical trials with BDNF and also offer information for future trials to consider. 

Abstract In recent years, there has been a growing demand for miniaturization, low power consumption, quick treatments, and non-invasive clinical strategies in the healthcare industry. To meet these demands, healthcare professionals are seeking new technological paradigms that can improve diagnostic accuracy while ensuring patient compliance. Neuromorphic engineering, which uses neural models in hardware and software to replicate brain-like behaviors, can help usher in a new era of medicine by delivering low power, low latency, small footprint, and high bandwidth solutions. This paper provides an overview of recent neuromorphic advancements in medicine, including medical imaging and cancer diagnosis, processing of biosignals for diagnosis, and biomedical interfaces, such as motor, cognitive, and perception prostheses. For each section, we provide examples of how brain-inspired models can successfully compete with conventional artificial intelligence algorithms, demonstrating the potential of neuromorphic engineering to meet demands and improve patient outcomes. Lastly, we discuss current struggles in fitting neuromorphic hardware with non-neuromorphic technologies and propose potential solutions for future bottlenecks in hardware compatibility. 

Abstract Measures of functional connectivity have played a central role in advancing our understanding of how information is transmitted and processed within the brain. Traditionally, these studies have focused on identifying redundant functional connectivity, which involves determining when activity is similar across different sites or neurons. However, recent research has highlighted the importance of also identifying synergistic connectivity—that is, connectivity that gives rise to information not contained in either site or neuron alone. Here, we measured redundant and synergistic functional connectivity between neurons in the mouse primary auditory cortex during a sound discrimination task. Specifically, we measured directed functional connectivity between neurons simultaneously recorded with calcium imaging. We used Granger Causality as a functional connectivity measure. We then used Partial Information Decomposition to quantify the amount of redundant and synergistic information about the presented sound that is carried by functionally connected or functionally unconnected pairs of neurons. We found that functionally connected pairs present proportionally more redundant information and proportionally less synergistic information about sound than unconnected pairs, suggesting that their functional connectivity is primarily redundant. Further, synergy and redundancy coexisted both when mice made correct or incorrect perceptual discriminations. However, redundancy was much higher (both in absolute terms and in proportion to the total information available in neuron pairs) in correct behavioural choices compared to incorrect ones, whereas synergy was higher in absolute terms but lower in relative terms in correct than in incorrect behavioural choices. Moreover, the proportion of redundancy reliably predicted perceptual discriminations, with the proportion of synergy adding no extra predictive power. These results suggest a crucial contribution of redundancy to correct perceptual discriminations, possibly due to the advantage it offers for information propagation, and also suggest a role of synergy in enhancing information level during correct discriminations. 

Abstract Global optical flow estimation is the foundation stone for obtaining odometry which is used to enable aerial robot navigation. However, such a method has to be of low latency and high robustness whilst also respecting the size, weight, area and power (SWAP) constraints of the robot. A combination of cameras coupled with inertial measurement units (IMUs) has proven to be the best combination in order to obtain such low latency odometry on resource‐constrained aerial robots. Recently, deep learning approaches for visual inertial fusion have gained momentum due to their high accuracy and robustness. However, an equally noteworthy benefit for robotics of these techniques are their inherent scalability (adaptation to different sized aerial robots) and unification (same method works on different sized aerial robots). To this end, we present a deep learning approach called PRGFlow for obtaining global optical flow and then loosely fuse it with an IMU for full 6‐DoF (Degrees of Freedom) relative pose estimation (which is then integrated to obtain odometry). The network is evaluated on the MSCOCO dataset and the dead‐reckoned odometry on multiple real‐flight trajectories without any fine‐tuning or re‐training. A detailed benchmark comparing different network architectures and loss functions to enable scalability is also presented. It is shown that the method outperforms classical feature matching methods by 2 under noisy data. The supplementary material and code can be found athttp://prg.cs.umd.edu/PRGFlow. 

Event-based motion field estimation is an important task. However, current optical flow methods face challenges: learning-based approaches, often frame-based and relying on CNNs, lack cross-domain transferability, while model-based methods, though more robust, are less accurate. To address the limitations of optical flow estimation, recent works have focused on normal flow, which can be more reliably measured in regions with limited texture or strong edges. However, existing normal flow estimators are predominantly model-based and suffer from high errors. In this paper, we propose a novel supervised point-based method for normal flow estimation that overcomes the limitations of existing event learning-based approaches. Using a local point cloud encoder, our method directly estimates per-event normal flow from raw events, offering multiple unique advantages: 1) It produces temporally and spatially sharp predictions. 2) It supports more diverse data augmentation, such as random rotation, to improve robustness across various domains. 3) It naturally supports uncertainty quantification via ensemble inference, which benefits downstream tasks. 4) It enables training and inference on undistorted data in normalized camera coordinates, improving transferability across cameras. Extensive experiments demonstrate that our method achieves better and more consistent performance than state-of-the-art methods when transferred across different datasets. Leveraging this transferability, we train our model on the union of datasets and release it for public use. Finally, we introduce an egomotion solver based on a maximum-margin problem that uses normal flow and IMU to achieve strong performance in challenging scenarios. Codes are available at github.com/dhyuan99/VecKM flow. 

Analyzing the functional connectivity of the brain is an enormous challenge, as deciphering functional connectivity requires knowledge of functional responses and connections. One promising strategy is analyzing the spatial pattern of activity correlations across cell populations. In the primary auditory cortex (A1), cells respond to different sound features. On the large scale, there exists a tonotopic map, which is fractured at the small scale, raising the question of whether functional connections are spatially ordered or disordered. To test whether functional connectivity on a local and a global scale is also disordered, we first designed a robust statistical model to estimate parameters and test for the significance of the estimated correlation maps. We developed an inference method that allows efficient model fitting and statistical testing to project the correlation maps to 2D space. We then performed in vivo two-photon calcium imaging in layer 2/3 of A1 with pure tones (PT) or a combination of two tones (TT; harmonically related or not). We found that the spatial patterns of signal correlations (SCs) depend on the type of sound stimuli that were presented. The functional 2D maps of PT-driven SCs are more restricted to local neurons than TT signal correlations which showed more global textures. 2D SC patterns for harmonic stimuli showed spatially distinct relationships. TT SCs revealed spatially precise functional connectivity between harmonically related neurons. Thus, even though the frequency preference of neighboring neurons in A1 is functionally diverse, the functional connection pattern of these neurons is functionally precise and harmonically related. 

Video Frame Interpolation aims to recover realistic missing frames between observed frames, generating a highframe- rate video from a low-frame-rate video. However, without additional guidance, the large motion between frames makes this problem ill-posed. Event-based Video Frame Interpolation (EVFI) addresses this challenge by using sparse, high-temporal-resolution event measurements as motion guidance. This guidance allows EVFI methods to significantly outperform frame-only methods. However, to date, EVFI methods have relied on a limited set of paired eventframe training data, severely limiting their performance and generalization capabilities. In this work, we overcome the limited data challenge by adapting pre-trained video diffusion models trained on internet-scale datasets to EVFI. We experimentally validate our approach on real-world EVFI datasets, including a new one that we introduce. Our method outperforms existing methods and generalizes across cameras far better than existing approaches.