An official website of the United States government
We study the design of a goal-oriented sampling and scheduling strategy through a channel with highly variable two-way random delay, which can exhibit memory (e.g., Delay and Disruption Tolerant Networks). The objective of the communication is to optimize the performance of remote inference, where an inference algorithm (e.g., a trained neural network) on the receiver side predicts a time-varying target signal using the data samples transmitted by a sensor. Previous formulations to this problem either assumed a channel with IID transmission delay, neglecting feedback delay or considered the monotonic relation that the performance only gets worse as the input information ages. We show how, with delayed feedback, one can effectively exploit the knowledge about delay memory through an index-based threshold policy. This policy minimizes the expected time-average inference error that can be monotone or non-monotone in age. The index function is expressed in terms of the Age of Information (AoI) on the receiver side and a parameter regarding the distribution of subsequent transmission delay, both of which can readily be tracked.
more »
« less
Learning and Communications Co-Design for Remote Inference Systems: Feature Length Selection and Transmission Scheduling
In this paper, we consider a remote inference system, where a neural network is used to infer a time-varying target (e.g., robot movement), based on features (e.g., video clips) that are progressively received from a sensing node (e.g., a camera). Each feature is a temporal sequence of sensory data. The inference error is determined by (i) the timeliness and (ii) the sequence length of the feature, where we use Age of Information (AoI) as a metric for timeliness. While a longer feature can typically provide better inference performance, it often requires more channel resources for sending the feature. To minimize the time-averaged inference error, we study a learning and communication co-design problem that jointly optimizes feature length selection and transmission scheduling. When there is a single sensor-predictor pair and a single channel, we develop low-complexity optimal co-designs for both the cases of time-invariant and time-variant feature length. When there are multiple sensor-predictor pairs and multiple channels, the co-design problem becomes a restless multi-arm multi-action bandit problem that is PSPACE-hard. For this setting, we design a low-complexity algorithm to solve the problem. Trace-driven evaluations demonstrate the potential of these co-designs to reduce inference error by up to 10000 times.
more »
« less
- PAR ID:
- 10495999
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Journal on Selected Areas in Information Theory
- Volume:
- 4
- ISSN:
- 2641-8770
- Page Range / eLocation ID:
- 524 to 538
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We study adversarially robust transfer learning, wherein, given labeled data on multiple (source) tasks, the goal is to train a model with small robust error on a previously unseen (target) task. In particular, we consider a multi-task representation learning (MTRL) setting, i.e., we assume that the source and target tasks admit a simple (linear) predictor on top of a shared representation (e.g., the final hidden layer of a deep neural network). In this general setting, we provide rates on the excess adversarial (transfer) risk for Lipschitz losses and smooth nonnegative losses. These rates show that learning a representation using adversarial training on diverse tasks helps protect against inference-time attacks in data-scarce environments. Additionally, we provide novel rates for the single-task setting.more » « less
-
The forecasting of significant societal events such as civil unrest and economic crisis is an interesting and challenging problem which requires both timeliness, precision, and comprehensiveness. Significant societal events are influenced and indicated jointly by multiple aspects of a society, including its economics, politics, and culture. Traditional forecasting methods based on a single data source find it hard to cover all these aspects comprehensively, thus limiting model performance. Multi-source event forecasting has proven promising but still suffers from several challenges, including (1) geographical hierarchies in multi-source data features, (2) hierarchical missing values, (3) characterization of structured feature sparsity, and (4) difficulty in model’s online update with incomplete multiple sources. This article proposes a novel feature learning model that concurrently addresses all the above challenges. Specifically, given multi-source data from different geographical levels, we design a new forecasting model by characterizing the lower-level features’ dependence on higher-level features. To handle the correlations amidst structured feature sets and deal with missing values among the coupled features, we propose a novel feature learning model based on an N th-order strong hierarchy and fused-overlapping group Lasso. An efficient algorithm is developed to optimize model parameters and ensure global optima. More importantly, to enable the model update in real time, the online learning algorithm is formulated and active set techniques are leveraged to resolve the crucial challenge when new patterns of missing features appear in real time. Extensive experiments on 10 datasets in different domains demonstrate the effectiveness and efficiency of the proposed models.more » « less
-
This article investigates a robust receiver scheme for a single carrier, multiple-input–multiple-output (MIMO) underwater acoustic (UWA) communications, which uses the sparse Bayesian learning algorithm for iterative channel estimation embedded in Turbo equalization (TEQ). We derive a block-wise sparse Bayesian learning framework modeling the spatial correlation of the MIMO UWA channels, where a more robust expectation–maximization algorithm is proposed for updating the joint estimates of channel impulse response, residual noise, and channel covariance matrix. By exploiting the spatially correlated sparsity of MIMO UWA channels and the second-order a priori channel statistics from the training sequence, the proposed Bayesian channel estimator enjoys not only relatively low complexity but also more stable control of the hyperparameters that determine the channel sparsity and recovery accuracy. Moreover, this article proposes a low complexity space-time soft decision feedback equalizer (ST-SDFE) with successive soft interference cancellation. Evaluated by the undersea 2008 Surface Processes and Acoustic Communications Experiment, the improved sparse Bayesian learning channel estimation algorithm outperforms the conventional Bayesian algorithms in terms of the robustness and complexity, while enjoying better estimation accuracy than the orthogonal matching pursuit and the improved proportionate normalized least mean squares algorithms. We have also verified that the proposed ST-SDFE TEQ significantly outperforms the low-complexity minimum mean square error TEQ in terms of the bit error rate and error propagation.more » « less
-
We consider the problem of coded distributed computing where a large linear computational job, such as a matrix multiplication, is divided into $$k$$ smaller tasks, encoded using an $(n,k)$ linear code, and performed over $$n$$ distributed nodes. The goal is to reduce the average execution time of the computational job. We provide a connection between the problem of characterizing the average execution time of a coded distributed computing system and the problem of analyzing the error probability of codes of length $$n$$ used over erasure channels. Accordingly, we present closed-form expressions for the execution time using binary random linear codes and the best execution time any linear-coded distributed computing system can achieve. It is also shown that there exist good binary linear codes that attain, asymptotically, the best performance any linear code, not necessarily binary, can achieve. We also investigate the performance of coded distributed computing systems using polar and Reed-Muller (RM) codes that can benefit from low-complexity decoding, and superior performance, respectively, as well as explicit constructions. The proposed framework in this paper can enable efficient designs of distributed computing systems given the rich literature in the channel coding theory.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1109/JSAIT.2023.3322620
