More Like this

1. Privacy-Preserving Prediction

   Dwork, C; Feldman, V (June 2018, Proceedings of Machine Learning Research)

   Bubeck, S; Perchet, V; Rigollet, P (Ed.)

   Ensuring differential privacy of models learned from sensitive user data is an important goal that has been studied extensively in recent years. It is now known that for some basic learning problems, especially those involving high-dimensional data, producing an accurate private model requires much more data than learning without privacy. At the same time, in many applications it is not necessary to expose the model itself. Instead users may be allowed to query the prediction model on their inputs only through an appropriate interface. Here we formulate the problem of ensuring privacy of individual predictions and investigate the overheads required to achieve it in several standard models of classification and regression. We first describe a simple baseline approach based on training several models on disjoint subsets of data and using standard private aggregation techniques to predict. We show that this approach has nearly optimal sample complexity for (realizable) PAC learning of any class of Boolean functions. At the same time, without strong assumptions on the data distribution, the aggregation step introduces a substantial overhead. We demonstrate that this overhead can be avoided for the well-studied class of thresholds on a line and for a number of standard settings of convex regression. The analysis of our algorithm for learning thresholds relies crucially on strong generalization guarantees that we establish for all differentially private prediction algorithms. 

   more » « less
2. Structure-preserving GANs

   Birrell, J.; Katsoulakis, M.; Rey-Bellet, L.; Zhu, W. (January 2022, Proceedings of Machine Learning Research)

   Generative adversarial networks (GANs), a class of distribution-learning methods based on a two-player game between a generator and a discriminator, can generally be formulated as a minmax problem based on the variational representation of a divergence between the unknown and the generated distributions. We introduce structure-preserving GANs as a data-efficient framework for learning distributions with additional structure such as group symmetry, by developing new variational representations for divergences. Our theory shows that we can reduce the discriminator space to its projection on the invariant discriminator space, using the conditional expectation with respect to the sigma-algebra associated to the underlying structure. In addition, we prove that the discriminator space reduction must be accompanied by a careful design of structured generators, as flawed designs may easily lead to a catastrophic “mode collapse” of the learned distribution. We contextualize our framework by building symmetry-preserving GANs for distributions with intrinsic group symmetry, and demonstrate that both players, namely the equivariant generator and invariant discriminator, play important but distinct roles in the learning process. Empirical experiments and ablation studies across a broad range of data sets, including real-world medical imaging, validate our theory, and show our proposed methods achieve significantly improved sample fidelity and diversity—almost an order of magnitude measured in Frechet Inception Distance—especially in the small data regime 

   more » « less
3. Structure-preserving GANs

   Birell, Jeremiah; Katsoulakis, Markos A.; Rey-Bellet, Luc; Zhu, Wei (July 2022, Proceedings of Machine Learning Research)
4. Structure-preserving GANs

   Birrell, Jeremiah; Katsoulakis, Markos; Rey-Bellet, Luc; Zhu, Wei (January 2022, ICML 2022)
5. Privacy-Preserving Database Fingerprinting

   https://doi.org/10.14722/ndss.2023.24693  

   Ji, Tianxi; Ayday, Erman; Yilmaz, Emre; Li, Ming; Li, Pan (January 2023, NDDS Symposium)