skip to main content
Explore Scholarly Publications and Datasets in the NSF-PAR

  • Home
  • Search Results
  • Page 1 of 8

Search for: All records

Creators/Authors contains: "Tanaka, H."

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. ; ; ; ( , Advances in neural information processing systems)
    Pruning the parameters of deep neural networks has generated intense interest due to potential savings in time, memory and energy both during training and at test time. Recent works have identified, through an expensive sequence of training and pruning cycles, the existence of winning lottery tickets or sparse trainable subnetworks at initialization. This raises a foundational question: can we identify highly sparse trainable subnetworks at initialization, without ever training, or indeed without ever looking at the data? We provide an affirmative answer to this question through theory driven algorithm design. We first mathematically formulate and experimentally verify a conservation lawmore »that explains why existing gradient-based pruning algorithms at initialization suffer from layer-collapse, the premature pruning of an entire layer rendering a network untrainable. This theory also elucidates how layer-collapse can be entirely avoided, motivating a novel pruning algorithm Iterative Synaptic Flow Pruning (SynFlow). This algorithm can be interpreted as preserving the total flow of synaptic strengths through the network at initialization subject to a sparsity constraint. Notably, this algorithm makes no reference to the training data and consistently competes with or outperforms existing state-of-the-art pruning algorithms at initialization over a range of models (VGG and ResNet), datasets (CIFAR-10/100 and Tiny ImageNet), and sparsity constraints (up to 99.99 percent). Thus our data-agnostic pruning algorithm challenges the existing paradigm that, at initialization, data must be used to quantify which synapses are important.« less
    Accepted Manuscript Full Text Available
  2. ; ; ; ; ; ( , Advances in neural information processing systems)
    Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess themore »degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.« less
    Accepted Manuscript Full Text Available
  3. ; ; ; ; ; ; ; ; ; ; et al ( , Astroparticle Physics)
    Free, publicly-accessible full text available June 1, 2023
  4. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Free, publicly-accessible full text available December 1, 2022
  5. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Free, publicly-accessible full text available March 1, 2023
  6. ; ; ; ; ; ; ; ; ; ; et al ( , Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment)
    Free, publicly-accessible full text available March 1, 2023
  7. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review B)
    Full Text Available 
  8. ; ; ; ; ; ; ; ; ; ; et al ( , Physics Reports)
    Full Text Available 
  9. ; ; ; ; ; ; ; ; ; ; et al ( , Physical Review D)
    Free, publicly-accessible full text available April 1, 2023
  10. ; ; ; ; ; ; ; ; ; ; et al ( , The Astrophysical Journal)
    Abstract We search for gravitational-wave signals associated with gamma-ray bursts (GRBs) detected by the Fermi and Swift satellites during the second half of the third observing run of Advanced LIGO and Advanced Virgo (2019 November 1 15:00 UTC–2020 March 27 17:00 UTC). We conduct two independent searches: a generic gravitational-wave transients search to analyze 86 GRBs and an analysis to target binary mergers with at least one neutron star as short GRB progenitors for 17 events. We find no significant evidence for gravitational-wave signals associated with any of these GRBs. A weighted binomial test of the combined results finds nomore »evidence for subthreshold gravitational-wave signals associated with this GRB ensemble either. We use several source types and signal morphologies during the searches, resulting in lower bounds on the estimated distance to each GRB. Finally, we constrain the population of low-luminosity short GRBs using results from the first to the third observing runs of Advanced LIGO and Advanced Virgo. The resulting population is in accordance with the local binary neutron star merger rate.« less
    Free, publicly-accessible full text available April 1, 2023