Abstract Decoding sensory stimuli from neural activity can provide insight into how the nervous system might interpret the physical environment, and facilitates the development of brain-machine interfaces. Nevertheless, the neural decoding problem remains a significant open challenge. Here, we present an efficient nonlinear decoding approach for inferring natural scene stimuli from the spiking activities of retinal ganglion cells (RGCs). Our approach uses neural networks to improve on existing decoders in both accuracy and scalability. Trained and validated on real retinal spike data from more than 1000 simultaneously recorded macaque RGC units, the decoder demonstrates the necessity of nonlinear computations formore »
Sparse Coding Enables the Reconstruction of High-Fidelity Images and Video from Retinal Spike Trains
The optic nerve transmits visual information to the brain as trains
of discrete events, a low-power, low-bandwidth communication
channel also exploited by silicon retina cameras. Extracting highfidelity
visual input from retinal event trains is thus a key challenge
for both computational neuroscience and neuromorphic engineering.
Here, we investigate whether sparse coding can enable the
reconstruction of high-fidelity images and video from retinal event
trains. Our approach is analogous to compressive sensing, in which
only a random subset of pixels are transmitted and the missing
information is estimated via inference. We employed a variant of
the Locally Competitive Algorithm to infer sparse representations
from retinal event trains, using a dictionary of convolutional features
optimized via stochastic gradient descent and trained in an
unsupervised manner using a local Hebbian learning rule with
momentum.
We used an anatomically realistic retinal model with stochastic
graded release from cones and bipolar cells to encode thumbnail
images as spike trains arising from ON and OFF retinal ganglion
cells. The spikes from each model ganglion cell were summed over a
32 msec time window, yielding a noisy rate-coded image. Analogous
to how the primary visual cortex is postulated to infer features
from noisy spike trains arising from the optic nerve, we inferred
a higher-fidelity sparse reconstruction from the noisy rate-coded
image using a convolutional dictionary trained on the original
CIFAR10 database. more »
- Award ID(s):
- 1734980
- Publication Date:
- NSF-PAR ID:
- 10075673
- Journal Name:
- ICONS '18 Proceedings of the International Conference on Neuromorphic Systems Article No. 8
- Page Range or eLocation-ID:
- 1 to 5
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Bioengineering systems have transformed scientific knowledge of cellular behaviors in the nervous system (NS) and pioneered innovative, regenerative therapies to treat adult neural disorders. Microscale systems with characteristic lengths of single to hundreds of microns have examined the development and specialized behaviors of numerous neuromuscular and neurosensory components of the NS. The visual system is comprised of the eye sensory organ and its connecting pathways to the visual cortex. Significant vision loss arises from dysfunction in the retina, the photosensitive tissue at the eye posterior that achieves phototransduction of light to form images in the brain. Retinal regenerative medicine hasmore »
-
Convolutional neural networks (CNNs), a class of deep learning models, have experienced recent success in modeling sensory cortices and retinal circuits through optimizing performance on machine learning tasks, otherwise known as task optimization. Previous research has shown task-optimized CNNs to be capable of providing explanations as to why the retina efficiently encodes natural stimuli and how certain retinal cell types are involved in efficient encoding. In our work, we sought to use task-optimized CNNs as a means of explaining computational mechanisms responsible for motion-selective retinal circuits. We designed a biologically constrained CNN and optimized its performance on a motion-classification task.more »
-
Telecystoscopy can lower the barrier to access critical urologic diagnostics for patients around the world. A major challenge for robotic control of flexible cystoscopes and intuitive teleoperation is the pose estimation of the scope tip. We propose a novel real-time camera localization method using video recordings from a prior cystoscopy and 3D bladder reconstruction to estimate cystoscope pose within the bladder during follow-up telecystoscopy. We map prior video frames into a low-dimensional space as a dictionary so that a new image can be likewise mapped to efficiently retrieve its nearest neighbor among the dictionary images. The cystoscope pose is thenmore »
-
The choroid provides the majority of blood flow to the ocular tissues and structures that facilitate the processes of retinal metabolism responsible for vision. Specifically, the choriocapillaris provides a structural network of small blood vessels that supplies the retinal ganglion cells and deep ocular tissues. Similar to retinal nerve fiber layer thickness, choroidal thickness (CT) has been suggested to represent a quantifiable health biomarker for choroidal tissues. Glaucoma is a disease with vascular contributions in its onset and progression. Despite its importance in maintaining ocular structure and vascular functionality, clinical assessments of choroidal tissues have been historically challenged by themore »
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Publisher's Version of Record
- https://doi.org/10.1145/3229884.3229892
