Search for: All records

The neural plausibility of backpropagation has long been disputed, primarily for its use of non-local weight transport — the biologically dubious requirement that one neuron instantaneously measure the synaptic weights of another. Until recently, attempts to create local learning rules that avoid weight transport have typically failed in the large-scale learning scenarios where backpropagation shines, e.g. ImageNet categorization with deep convolutional networks. Here, we investigate a recently proposed local learning rule that yields competitive performance with backpropagation and find that it is highly sensitive to metaparameter choices, requiring laborious tuning that does not transfer across network architecture. Our analysis indicates the underlying mathematical reason for this instability, allowing us to identify a more robust local learning rule that better transfers without metaparameter tuning. Nonetheless, we find a performance and stability gap between this local rule and backpropagation that widens with increasing model depth. We then investigate several non-local learning rules that relax the need for instantaneous weight transport into a more biologically-plausible "weight estimation" process, showing that these rules match state-of-the-art performance on deep networks and operate effectively in the presence of noisy updates. Taken together, our results suggest two routes towards the discovery of neural implementations for credit assignmentmore »without weight symmetry: further improvement of local rules so that they perform consistently across architectures and the identification of biological implementations for non-local learning mechanisms.« less

Abstract The number of long gamma-ray bursts (GRBs) known to have occurred in the distant Universe (z > 5) is small (∼15); however, these events provide a powerful way of probing star formation at the onset of galaxy evolution. In this paper, we present the case for GRB 100205A being a largely overlooked high-redshift event. While initially noted as a high-z candidate, this event and its host galaxy have not been explored in detail. By combining optical and near-infrared Gemini afterglow imaging (at t < 1.3 d since burst) with deep late-time limits on host emission from the Hubble Space Telescope, we show that the most likely scenario is that GRB 100205A arose in the range 4 < z < 8. GRB 100205A is an example of a burst whose afterglow, even at ∼1 h post burst, could only be identified by 8-m class IR observations, and suggests that such observations of all optically dark bursts may be necessary to significantly enhance the number of high-redshift GRBs known.