Pruning and quantization are core techniques used to reduce the inference costs of deep neural networks. Among the state-of-the-art pruning techniques, magnitude-based pruning algorithms have demonstrated consistent success in the reduction of both weight and feature map complexity. However, we find that existing measures of neuron (or channel) importance estimation used for such pruning procedures have at least one of two limitations: (1) failure to consider the interdependence between successive layers; and/or (2) performing the estimation in a parametric setting or by using distributional assumptions on the feature maps. In this work, we demonstrate that the importance rankings of the output neurons of a given layer strongly depend on the sparsity level of the preceding layer, and therefore, naïvely estimating neuron importance to drive magnitude-based pruning will lead to suboptimal performance. Informed by this observation, we propose a purely data-driven nonparametric, magnitude-based channel pruning strategy that works in a greedy manner based on the activations of the previous sparsified layer. We demonstrate that our proposed method works effectively in combination with statistics-based quantization techniques to generate low precision structured subnetworks that can be efficiently accelerated by hardware platforms such as GPUs and FPGAs. Using our proposed algorithms, we demonstrate increased performance per memory footprint over existing solutions across a range of discriminative and generative networks.
more »
« less
Deep Spectrum Cartography Using Quantized Measurements
Spectrum cartography (SC) techniques craft multi-domain (e.g., space and frequency) radio maps from limited measurements, which is an ill-posed inverse problem. Recent works used low-dimensional priors such as a low tensor rank structure and a deep generative model to assist radio map estimation---with provable guarantees.
However, a premise of these approaches is that the sensors are able to send real-valued feedback to a fusion center for SC---yet practical communication systems often use (heavy) quantization for signaling. This work puts forth a limited feedback-based SC framework.
Similar to a prior work, a generative adversarial network (GAN)-based deep prior is used in our framework for fending against heavy shadowing.
However, instead of using real-valued feedback, a random quantization strategy is adopted and a maximum likelihood estimation (MLE) criterion is proposed.
Analysis shows that the MLE provably recovers the radio map, under reasonable conditions.
Simulations are conducted to showcase the effectiveness of the proposed approach.
more »
« less
- Award ID(s):
- 2210004
- NSF-PAR ID:
- 10515731
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 978-1-7281-6327-7
- Page Range / eLocation ID:
- 1 to 5
- Format(s):
- Medium: X
- Location:
- Rhodes Island, Greece
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper focuses on downlink channel state information (CSI) acquisition. A frequency division duplex (FDD) of massive MIMO system is considered. In such systems, the base station (BS) obtains the downlink CSI from the mobile users' feedback. A key consideration is to reduce the feedback overhead while ensuring that the BS accurately recovers the downlink CSI. Existing approaches often resort to dictionary-based or tensor/matrix decomposition techniques, which either exhibit unsatisfactory accuracy or induce heavy computational load at the mobile end. To circumvent these challenges, this work formulates the limited channel feedback problem as a quantized and compressed matrix recovery problem. The formulation presents a computationally challenging maximum likelihood estimation (MLE) problem. An ADMM algorithm leveraging existing harmonic retrieval tools is proposed to effectively tackle the optimization problem. Simulations show that the proposed method attains promising channel estimation accuracy, using a much smaller amount of feedback bits relative to existing methods.more » « less
-
We present a novel generative Session-Based Recommendation (SBR) framework, called VAriational SEssion-based Recommendation (VASER) – a non-linear probabilistic methodology allowing Bayesian inference for flexible parameter estimation of sequential recommendations. Instead of directly applying extended Variational AutoEncoders (VAE) to SBR, the proposed method introduces normalizing flows to estimate the probabilistic posterior, which is more effective than the agnostic presumed prior approximation used in existing deep generative recommendation approaches. VASER explores soft attention mechanism to upweight the important clicks in a session. We empirically demonstrate that the proposed model significantly outperforms several state-of-the-art baselines, including the recently-proposed RNN/VAE-based approaches on real-world datasets.more » « less
-
null (Ed.)Spectrum cartography (SC) aims at estimating the multi-aspect (e.g., space, frequency, and time) interference level caused by multiple emitters from limited measurements. Early SC approaches rely on model assumptions about the radio map, e.g., sparsity and smoothness, which may be grossly violated under critical scenarios, e.g., in the presence of severe shadowing. More recent data-driven methods train deep generative networks to distill parsimonious representations of complex scenarios, in order to enhance performance of SC. The challenge is that the state space of this learning problem is extremely large—induced by different combinations of key problem constituents, e.g., the number of emitters, the emitters’ carrier frequencies, and the emitter locations. Learning over such a huge space can be costly in terms of sample complexity and training time; it also frequently leads to generalization problems. Our method integrates the favorable traits of model and data-driven approaches, which substantially ‘shrinks’ the state space. Specifically, the proposed learning paradigm only needs to learn a generative model for the radio map of a single emitter (as opposed to numerous combinations of multiple emitters), leveraging a nonnegative matrix factorization (NMF)-based emitter disaggregation process. Numerical evidence shows that the proposed method outperforms state-of-the-art purely model-driven and purely data-driven approachesmore » « less
-
Current cellular systems use pilot-aided statistical channel state information (S-CSI) estimation and limited feedback schemes to aid in link adaptation and scheduling decisions. However, in the presence of pulsed radar signals, pilot-aided S-CSI is inaccurate since interference statistics on pilot and nonpilot resources can be different. Moreover, the channel will be bimodal as a result of the periodic interference. In this paper, we propose a max-min heuristic to estimate the post-equalizer SINR in the case of non-pilot pulsed radar interference, and characterize its distribution as a function of noise variance and interference power. We observe that the proposed heuristic incurs low computational complexity, and is robust beyond a certain SINR threshold for different modulation schemes, especially for QPSK. This enables us to develop a comprehensive semi-blind framework to estimate the wideband SINR metric that is commonly used for S-CSI quantization in 3GPP Long-Term Evolution (LTE) and New Radio (NR) networks. Finally, we propose dual CSI feedback for practical radar-cellular spectrum sharing, to enable accurate CSI acquisition in the bimodal channel. We demonstrate significant improvements in throughput, block error rate and retransmission-induced latency for LTE-Advanced Pro when compared to conventional pilot-aided S-CSI estimation and limited feedback schemes.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ICASSP49357.2023.10096058
