Search for: All records

This paper develops bounds for learning lossless source coding under the PAC (probably approximately correct) framework. The paper considers iid sources with online learning: first the coder learns the data structure from training sequences. When presented with a test sequence for compression, it continues to learn from/adapt to the test sequence. The results show, not unsurprisingly, that there is little gain from online learning when the training sequence length is much longer than the test sequence length. But if the test sequence length is longer than the training sequence, there is a significant gain. Coders for online learning has a somewhat surprising structure: the training sequence is used to estimate a confidence interval for the distribution, and the coding distribution is found through a prior distribution over this interval. 

In a traditional distributed storage system, a source can be restored perfectly when a certain subset of servers is contacted. The coding is independent of the contents of the source. This paper considers instead a lossy source coding version of this problem where the more servers that are contacted, the higher the quality of the restored source. An example could be video stored on distributed storage. In information theory, this is called the multiple description problem, where the distortion depends on the number of descriptions received. The problem considered in this paper is how to restore the system operation when one of the servers fail and a new server replaces it, that is, repair. The requirement is that the distortions in the restored system should be no more than in the original system. The question is how many extra bits are needed for repair. We find an achievable rate and show that this is optimal in certain cases. One conclusion is that it is necessary to design the multiple description codes with repair in mind; just using an existing multiple description code results in unnecessary high repair rates.