skip to main content
Explore Scholarly Publications and Datasets in the NSF-PAR

  • Home
  • Search Results
  • Page 1 of 6

Search for: All records

Creators/Authors contains: "Tanaka, H."

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. ; ; ; ( , Advances in neural information processing systems)
    Pruning the parameters of deep neural networks has generated intense interest due to potential savings in time, memory and energy both during training and at test time. Recent works have identified, through an expensive sequence of training and pruning cycles, the existence of winning lottery tickets or sparse trainable subnetworks at initialization. This raises a foundational question: can we identify highly sparse trainable subnetworks at initialization, without ever training, or indeed without ever looking at the data? We provide an affirmative answer to this question through theory driven algorithm design. We first mathematically formulate and experimentally verify a conservation lawmore »that explains why existing gradient-based pruning algorithms at initialization suffer from layer-collapse, the premature pruning of an entire layer rendering a network untrainable. This theory also elucidates how layer-collapse can be entirely avoided, motivating a novel pruning algorithm Iterative Synaptic Flow Pruning (SynFlow). This algorithm can be interpreted as preserving the total flow of synaptic strengths through the network at initialization subject to a sparsity constraint. Notably, this algorithm makes no reference to the training data and consistently competes with or outperforms existing state-of-the-art pruning algorithms at initialization over a range of models (VGG and ResNet), datasets (CIFAR-10/100 and Tiny ImageNet), and sparsity constraints (up to 99.99 percent). Thus our data-agnostic pruning algorithm challenges the existing paradigm that, at initialization, data must be used to quantify which synapses are important.« less
    Accepted Manuscript Full Text Available
  2. ; ; ; ; ; ( , Advances in neural information processing systems)
    Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess themore »degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.« less
    Accepted Manuscript Full Text Available
  3. ; ; ; ; ; ; ; ; ; ; et al ( , Physics Reports)
    Full Text Available 
  4. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review B)
    Full Text Available 
  5. ; ; ; ; ; ; ; ; ; ; et al ( , Journal of Physics G: Nuclear and Particle Physics)
    Free, publicly-accessible full text available April 29, 2022
  6. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Full Text Available 
  7. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review Letters)
    Full Text Available 
  8. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Free, publicly-accessible full text available July 1, 2022
  9. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Free, publicly-accessible full text available July 1, 2022
  10. ; ; ; ; ; ; ; ; ; ; et al ( , The European Physical Journal C)
    Abstract The Deep Underground Neutrino Experiment (DUNE), a 40-kton underground liquid argon time projection chamber experiment, will be sensitive to the electron-neutrino flavor component of the burst of neutrinos expected from the next Galactic core-collapse supernova. Such an observation will bring unique insight into the astrophysics of core collapse as well as into the properties of neutrinos. The general capabilities of DUNE for neutrino detection in the relevant few- to few-tens-of-MeV neutrino energy range will be described. As an example, DUNE’s ability to constrain the $$\nu _e$$ ν e spectral parameters of the neutrino burst will be considered.
    Free, publicly-accessible full text available May 1, 2022