More Like this

1. (f,Γ)-Divergences: Interpolating between f-Divergences and Integral Probability Metrics

   Birrell, Jeremiah ; Dupuis, Paul ; Katsoulakis, Markos ; Pantazis, Yannis ; Rey-Bellet, Luc ( January 2022 , Journal of machine learning research)
2. Optimized Quantum F-Divergences

   https://doi.org/10.1109/ISIT.2018.8437925  

   Wilde, Mark M. ( June 2018 , Proceedings of the 2018 IEEE International Symposium on Information Theory)

   null (Ed.)

   The quantum relative entropy is a measure of the distinguishability of two quantum states, and it is a unifying concept in quantum information theory: many information measures such as entropy, conditional entropy, mutual information, and entanglement measures can be realized from it. As such, there has been broad interest in generalizing the notion to further understand its most basic properties, one of which is the data processing inequality. The quantum f-divergence of Petz is one generalization of the quantum relative entropy, and it also leads to other relative entropies, such as the Petz--Renyi relative entropies. In this contribution, I introduce the optimized quantum f-divergence as a related generalization of quantum relative entropy. I prove that it satisfies the data processing inequality, and the method of proof relies upon the operator Jensen inequality, similar to Petz's original approach. Interestingly, the sandwiched Renyi relative entropies are particular examples of the optimized f-divergence. Thus, one benefit of this approach is that there is now a single, unified approach for establishing the data processing inequality for both the Petz--Renyi and sandwiched Renyi relative entropies, for the full range of parameters for which it is known to hold. 

   more » « less
3. Optimized quantum f -divergences and data processing

   https://doi.org/10.1088/1751-8121/aad5a1  

   Wilde, Mark M ( September 2018 , Journal of Physics A: Mathematical and Theoretical)
4. A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via f-Divergences

   https://doi.org/10.1109/ISIT44484.2020.9174015  

   Asoodeh, Shahab ; Liao, Jiachun ; Calmon, Flavio P. ; Kosut, Oliver ; Sankar, Lalitha ( June 2020 , Proc. of the 2020 IEEE International Symposium on Information Theory (ISIT))

   null (Ed.)
5. Non-asymptotic Performance Guarantees for Neural Estimation of f-divergences

   Sreekumar, Sreejith ; Zhang, Zhengxin ; Goldfeld, Ziv ( January 2022 , Proceedings of Machine Learning Research)

   Statistical distances (SDs), which quantify the dissimilarity between probability distributions, are central to machine learning and statistics. A modern method for estimating such distances from data relies on parametrizing a variational form by a neural network (NN) and optimizing it. These estimators are abundantly used in practice, but corresponding performance guarantees are partial and call for further exploration. In particular, there seems to be a fundamental tradeoff between the two sources of error involved: approximation and estimation. While the former needs the NN class to be rich and expressive, the latter relies on controlling complexity. This paper explores this tradeoff by means of non-asymptotic error bounds, focusing on three popular choices of SDs—Kullback-Leibler divergence, chi-squared divergence, and squared Hellinger distance. Our analysis relies on non-asymptotic function approximation theorems and tools from empirical process theory. Numerical results validating the theory are also provided. 

   more » « less