skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Sparse Coding Enables the Reconstruction of High-Fidelity Images and Video from Retinal Spike Trains
The optic nerve transmits visual information to the brain as trains of discrete events, a low-power, low-bandwidth communication channel also exploited by silicon retina cameras. Extracting highfidelity visual input from retinal event trains is thus a key challenge for both computational neuroscience and neuromorphic engineering. Here, we investigate whether sparse coding can enable the reconstruction of high-fidelity images and video from retinal event trains. Our approach is analogous to compressive sensing, in which only a random subset of pixels are transmitted and the missing information is estimated via inference. We employed a variant of the Locally Competitive Algorithm to infer sparse representations from retinal event trains, using a dictionary of convolutional features optimized via stochastic gradient descent and trained in an unsupervised manner using a local Hebbian learning rule with momentum. We used an anatomically realistic retinal model with stochastic graded release from cones and bipolar cells to encode thumbnail images as spike trains arising from ON and OFF retinal ganglion cells. The spikes from each model ganglion cell were summed over a 32 msec time window, yielding a noisy rate-coded image. Analogous to how the primary visual cortex is postulated to infer features from noisy spike trains arising from the optic nerve, we inferred a higher-fidelity sparse reconstruction from the noisy rate-coded image using a convolutional dictionary trained on the original CIFAR10 database. To investigate whether a similar approachworks on non-stochastic data, we demonstrate that the same procedure can be used to reconstruct high-frequency video from the asynchronous events arising from a silicon retina camera moving through a laboratory environment.  more » « less
Award ID(s):
1734980
PAR ID:
10075673
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
ICONS '18 Proceedings of the International Conference on Neuromorphic Systems Article No. 8
Page Range / eLocation ID:
1 to 5
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Primary open-angle glaucoma (POAG) is an optic neuropathy characterized by irreversible retinal ganglion cell damage and visual field loss. The global POAG prevalence is estimated to be 3.05%, and near term is expected to significantly rise, especially within aging Asian populations. Primary angle-closure glaucoma disproportionately affects Asians, with up to four times greater prevalence of normal-tension glaucoma reported compared with high-tension glaucoma. Estimates for overall POAG prevalence in Asian populations vary, with Chinese and Indian populations representing the majority of future cases. Structural characteristics associated with glaucoma progression including the optic nerve head, retina, and cornea are distinct in Asians, serving as intermediates between African and European descent populations. Patterns in IOP suggest some similarities between races, with a significant inverse relationship between age and IOP only in Asian populations. Genetic differences have been suggested to play a role in these differences, however, a clear genetic pattern is yet to be established. POAG pathogenesis differs between Asians and other ethnicities, and it may differ within the broad classification of the Asian race. Greater awareness and further research are needed to improve treatment plans and outcomes for the increasingly high prevalence of normal tension glaucoma within aging Asian populations. 
    more » « less
  2. Everything that the brain sees must first be encoded by the retina, which maintains a reliable representation of the visual world in many different, complex natural scenes while also adapting to stimulus changes. This study quantifies whether and how the brain selectively encodes stimulus features about scene identity in complex naturalistic environments. While a wealth of previous work has dug into the static and dynamic features of the population code in retinal ganglion cells, less is known about how populations form both flexible and reliable encoding in natural moving scenes. We record from the larval salamander retina responding to five different natural movies, over many repeats, and use these data to characterize the population code in terms of single-cell fluctuations in rate and pairwise couplings between cells. Decomposing the population code into independent and cell-cell interactions reveals how broad scene structure is encoded in the retinal output. while the single-cell activity adapts to different stimuli, the population structure captured in the sparse, strong couplings is consistent across natural movies as well as synthetic stimuli. We show that these interactions contribute to encoding scene identity. We also demonstrate that this structure likely arises in part from shared bipolar cell input as well as from gap junctions between retinal ganglion cells and amacrine cells. 
    more » « less
  3. Purpose To investigate relationships between blood pressure and the thickness of single retinal layers in the macula. Methods Participants of the population-based Beijing Eye Study, free of retinal or optic nerve disease, underwent medical and ophthalmological examinations including optical coherence tomographic examination of the macula. Applying a multiple-surface segmentation solution, we automatically segmented the retina into its various layers. Results The study included 2237 participants (mean age 61.8±8.4 years, range 50–93 years). Mean thicknesses of the retinal nerve fibre layer (RNFL), ganglion cell layer (GCL), inner plexiform layer, inner nuclear layer (INL), outer plexiform layer, outer nuclear layer/external limiting membrane, ellipsoid zone, photoreceptor outer segments (POS) and retinal pigment epithelium–Bruch membrane were 31.1±2.3 µm, 39.7±3.5 µm, 38.4±3.3 µm, 34.8±2.0 µm, 28.1±3.0 µm, 79.2±7.3 µm, 22.9±0.6 µm, 19.2±3.3 µm and 20.7±1.4 µm, respectively. In multivariable analysis, higher systolic blood pressure (SBP) and diastolic blood pressure (DBP) were associated with thinner GCL and thicker INL, after adjusting for age, sex and axial length (all p<0.0056). Higher SBP was additionally associated with thinner POS and higher DBP with thinner RNFL. For an elevation of SBP/DBP by 10 mm Hg, the RNFL, GCL, INL and POS changed by 2.0, 3.0, 1.5 and 2.0 µm, respectively. Conclusions Thickness of RNFL, GCL and POS was inversely and INL thickness was positively associated with higher blood pressure, while the thickness of the other retinal layers was not significantly correlated with blood pressure. The findings may be helpful for refinement of the morphometric detection of retinal diseases. 
    more » « less
  4. Abstract Vision is a complex sensory system that requires coordination among cellular and morphological traits, and it remains unclear how functional relationships among traits interact with ecological selective pressures to shape the evolution of vision. Many species have specialized high visual acuity regions in the retina defined by patterns of ganglion cell density, which may evolve in response to ecological traits. For example, ganglion cell density can increase radially towards the center of the retina to form an area centralis, which is thought to improve acuity towards the center of the visual field in predators. Another example is the horizontal streak, where ganglion cells are dense in a horizontal pattern across the retina, which is thought to be beneficial in horizon-dominated habitats. At the morphological level, many have proposed that predation selects for high orbit convergence angles, or forward-facing eyes. We tested these hypotheses in a phylogenetic framework across eutherian mammals and found support for the association between the horizontal streak and horizon-dominated habitats. However, we did not find a significant association between orbit convergence and predation. We also tested if retinal specializations evolve in response to orbit convergence angles. We found that horizontal streaks were associated with side-facing eyes, potentially facilitating panoramic vision. Previous studies observed that some species with side-facing eyes have an area centralis shifted towards the temporal side of the retina, such that the high acuity region would project forward, but this relationship had not been tested quantitatively. We found that the temporal distance of the area centralis from the center of the retina was inversely correlated with orbit convergence, as predicted. Our work shows a strong relationship between orbit convergence and retinal specializations. We find support that both visual ecology and functional interactions among traits play important roles in the evolution of ocular traits across mammals. 
    more » « less
  5. Bioengineering systems have transformed scientific knowledge of cellular behaviors in the nervous system (NS) and pioneered innovative, regenerative therapies to treat adult neural disorders. Microscale systems with characteristic lengths of single to hundreds of microns have examined the development and specialized behaviors of numerous neuromuscular and neurosensory components of the NS. The visual system is comprised of the eye sensory organ and its connecting pathways to the visual cortex. Significant vision loss arises from dysfunction in the retina, the photosensitive tissue at the eye posterior that achieves phototransduction of light to form images in the brain. Retinal regenerative medicine has embraced microfluidic technologies to manipulate stem-like cells for transplantation therapies, where de/differentiated cells are introduced within adult tissue to replace dysfunctional or damaged neurons. Microfluidic systems coupled with stem cell biology and biomaterials have produced exciting advances to restore vision. The current article reviews contemporary microfluidic technologies and microfluidics-enhanced bioassays, developed to interrogate cellular responses to adult retinal cues. The focus is on applications of microfluidics and microscale assays within mammalian sensory retina, or neuro retina, comprised of five types of retinal neurons (photoreceptors, horizontal, bipolar, amacrine, retinal ganglion) and one neuroglia (Müller), but excludes the non-sensory, retinal pigmented epithelium. 
    more » « less