An official website of the United States government
Abstract The heterogeneity of brain imaging methods in neuroscience provides rich data that cannot be captured by a single technique, and our interpretations benefit from approaches that enable easy comparison both within and across different data types. For example, comparing brain-wide neural dynamics across experiments and aligning such data to anatomical resources, such as gene expression patterns or connectomes, requires precise alignment to a common set of anatomical coordinates. However, this is challenging because registeringin vivofunctional imaging data toex vivoreference atlases requires accommodating differences in imaging modality, microscope specification, and sample preparation. We overcome these challenges inDrosophilaby building anin vivoreference atlas from multiphoton-imaged brains, called the Functional Drosophila Atlas (FDA). We then develop a two-step pipeline, BrIdge For Registering Over Statistical Templates (BIFROST), for transforming neural imaging data into this common space and for importingex vivoresources such as connectomes. Using genetically labeled cell types as ground truth, we demonstrate registration with a precision of less than 10 microns. Overall, BIFROST provides a pipeline for registering functional imaging datasets in the fly, both within and across experiments. SignificanceLarge-scale functional imaging experiments inDrosophilahave given us new insights into neural activity in various sensory and behavioral contexts. However, precisely registering volumetric images from different studies has proven challenging, limiting quantitative comparisons of data across experiments. Here, we address this limitation by developing BIFROST, a registration pipeline robust to differences across experimental setups and datasets. We benchmark this pipeline by genetically labeling cell types in the fly brain and demonstrate sub-10 micron registration precision, both across specimens and across laboratories. We further demonstrate accurate registration betweenin-vivobrain volumes and ultrastructural connectomes, enabling direct structure-function comparisons in future experiments.
more »
« less
Decentralized Motion Inference and Registration of Neuropixel Data
Multi-electrode arrays such as "Neuropixels" probes enable the study of neuronal voltage signals at high temporal and single-cell spatial resolution. However, in vivo recordings from these devices often experience some shifting of the probe (due e.g. to animal movement), resulting in poorly localized voltage readings that in turn can corrupt estimates of neural activity. We introduce a new registration method to partially correct for this motion. In contrast to previous template-based registration methods, the proposed approach is decentralized, estimating shifts of the data recorded in multiple timebins with respect to one another, and then extracting a global registration estimate from the resulting estimated shift matrix. We find that the resulting decentralized registration is more robust and accurate than previous template-based approaches applied to both simulated and real data, but nonetheless some significant non-stationarity in the recovered neural activity remains that should be accounted for by downstream processing pipelines. Open source code is available at https://github.com/evarol/NeuropixelsRegistration.
more »
« less
- Award ID(s):
- 1912194
- PAR ID:
- 10297238
- Date Published:
- Journal Name:
- ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Page Range / eLocation ID:
- 1085 to 1089
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Deformable image registration (DIR) is an active research topic in biomedical imaging. There is a growing interest in developing DIR methods based on deep learning (DL). A traditional DL approach to DIR is based on training a convolutional neural network (CNN) to estimate the registration field between two input images. While conceptually simple, this approach comes with a limitation that it exclusively relies on a pre-trained CNN without explicitly enforcing fidelity between the registered image and the reference. We present plug-and-play image registration network (PIRATE) as a new DIR method that addresses this issue by integrating an explicit data-fidelity penalty and a CNN prior. PIRATE pre-trains a CNN denoiser on the registration field and "plugs" it into an iterative method as a regularizer. We additionally present PIRATE+ that fine-tunes the CNN prior in PIRATE using deep equilibrium models (DEQ). PIRATE+ interprets the fixed-point iteration of PIRATE as a network with effectively infinite layers and then trains the resulting network end-to-end, enabling it to learn more task-specific information and boosting its performance. Our numerical results on OASIS and CANDI datasets show that our methods achieve state-of-the-art performance on DIR.more » « less
-
In the realm of neuroscience, mapping the three-dimensional (3D) neural circuitry and architecture of the brain is important for advancing our understanding of neural circuit organization and function. This study presents a novel pipeline that transforms mouse brain samples into detailed 3D brain models using a collaborative data analytics platform called “Texera.” The user-friendly Texera platform allows for effective interdisciplinary collaboration between team members in neuroscience, computer vision, and data processing. Our pipeline utilizes the tile images from a serial two-photon tomography/TissueCyte system, then stitches tile images into brain section images, and constructs 3D whole-brain image datasets. The resulting 3D data supports downstream analyses, including 3D whole-brain registration, atlas-based segmentation, cell counting, and high-resolution volumetric visualization. Using this platform, we implemented specialized optimization methods and obtained significant performance enhancement in workflow operations. We expect the neuroscience community can adopt our approach for large-scale image-based data processing and analysis.more » « less
-
Multi-robot cooperative control has been extensively studied using model-based distributed control methods. However, such control methods rely on sensing and perception modules in a sequential pipeline design, and the separation of perception and controls may cause processing latencies and compounding errors that affect control performance. End-to-end learning overcomes this limitation by implementing direct learning from onboard sensing data, with control commands output to the robots. Challenges exist in end-to-end learning for multi-robot cooperative control, and previous results are not scalable. We propose in this article a novel decentralized cooperative control method for multi-robot formations using deep neural networks, in which inter-robot communication is modeled by a graph neural network (GNN). Our method takes LiDAR sensor data as input, and the control policy is learned from demonstrations that are provided by an expert controller for decentralized formation control. Although it is trained with a fixed number of robots, the learned control policy is scalable. Evaluation in a robot simulator demonstrates the triangular formation behavior of multi-robot teams of different sizes under the learned control policy.more » « less
-
null (Ed.)Neuroimaging data typically undergoes several preprocessing steps before further analysis and mining can be done. Affine image registration is one of the important tasks during preprocessing. Recently, several image registration methods which are based on Convolutional Neural Networks have been proposed. However, due to the high computational and memory requirements of CNNs, these methods cannot be used in real-time for large neuroimaging data like fMRI. In this paper, we propose a Dual-Attention Recurrent Network (DRN) which uses a hard attention mechanism to allow the model to focus on small, but task-relevant, parts of the input image – thus reducing computational and memory costs. Furthermore, DRN naturally supports inhomogeneity between the raw input image (e.g., functional MRI) and the image we want to align it to (e.g., anatomical MRI) so it can be applied to harder registration tasks such as fMRI coregistration and normalization. Extensive experiments on two different datasets demonstrate that DRN significantly reduces the computational and memory costs compared with other neural network-based methods without sacrificing the quality of image registrationmore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICASSP39728.2021.9414145
