Organized surveillance, especially by governments poses a major
challenge to individual privacy, due to the resources governments
have at their disposal, and the possibility of overreach. Given the
impact of invasive monitoring, in most democratic countries, government
surveillance is, in theory, monitored and subject to public
oversight to guard against violations. In practice, there is a difficult
fine balance between safeguarding individual’s privacy rights
and not diluting the efficacy of national security investigations, as
exemplified by reports on government surveillance programs that
have caused public controversy, and have been challenged by civil
and privacy rights organizations.
Surveillance is generally conducted through a mechanism where
federal agencies obtain a warrant from a federal or state judge
(e.g., the US FISA court, Supreme Court in Canada) to subpoena
a company or service-provider (e.g., Google, Microsoft) for their
customers’ data. The courts provide annual statistics on the requests
(accepted, rejected), while the companies provide annual
transparency reports for public auditing. However, in practice, the
statistical information provided by the courts and companies is at a
very high level, generic, is released after-the-fact, and is inadequate
for auditing the operations. Often this is attributed to the lack of
scalable mechanisms for reporting and transparent auditing.
In this paper, we present SAMPL, a novel auditing framework
which leverages cryptographic mechanisms, such as zero knowledge
proofs, Pedersen commitments, Merkle trees, and public ledgers
to create a scalable mechanism for auditing electronic surveillance
processes involving multiple actors. SAMPL is the first framework
that can identify the actors (e.g., agencies and companies) that
violate the purview of the court orders. We experimentally demonstrate
the scalability for SAMPL for handling concurrent monitoring
processes without undermining their secrecy and auditability.
more »
« less
Chorus: a Programming Framework for Building Scalable Differential Privacy Mechanisms
Differential privacy is fast becoming the gold standard in enabling statistical analysis of data while protecting the privacy of individuals. However, practical use of differential privacy still lags behind research progress because research prototypes cannot satisfy the scalability requirements of production deployments. To address this challenge, we present Chorus, a framework for building scalable differential privacy mechanisms which is based on cooperation between the mechanism itself and a high-performance production database management system (DBMS). We demonstrate the use of Chorus to build the first highly scalable implementations of complex mechanisms like Weighted PINQ, MWEM, and the matrix mechanism. We report on our experience deploying Chorus at Uber, and evaluate its scalability on real-world queries.
more »
« less
- Award ID(s):
- 1730628
- NSF-PAR ID:
- 10221256
- Date Published:
- Journal Name:
- 2020 IEEE European Symposium on Security and Privacy (EuroS&P)
- Page Range / eLocation ID:
- 535 to 551
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples. By leveraging the sequential composition theory in DP, we randomize both input and latent spaces to strengthen our certified robustness bounds. To address the trade-off among model utility, privacy loss, and robustness, we design an original adversarial objective function, based on the post-processing property in DP, to tighten the sensitivity of our model. A new stochastic batch training is proposed to apply our mechanism on large DNNs and datasets, by bypassing the vanilla iterative batch-by-batch training in DP DNNs. An end-to-end theoretical analysis and evaluations show that our mechanism notably improves the robustness and scalability of DP DNNs.more » « less
-
null (Ed.)In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples. By leveraging the sequential composition theory in DP, we randomize both input and latent spaces to strengthen our certified robustness bounds. To address the trade-off among model utility, privacy loss, and robustness, we design an original adversarial objective function, based on the post-processing property in DP, to tighten the sensitivity of our model. A new stochastic batch training is proposed to apply our mechanism on large DNNs and datasets, by bypassing the vanilla iterative batch-by-batch training in DP DNNs. An end-to-end theoretical analysis and evaluations show that our mechanism notably improves the robustness and scalability of DP DNNs.more » « less
-
There is great demand for scalable, secure, and efficient privacy-preserving machine learning models that can be trained over distributed data. While deep learning models typically achieve the best results in a centralized non-secure setting, different models can excel when privacy and communication constraints are imposed. Instead, tree-based approaches such as XGBoost have attracted much attention for their high performance and ease of use; in particular, they often achieve state-of-the-art results on tabular data. Consequently, several recent works have focused on translating Gradient Boosted Decision Tree (GBDT) models like XGBoost into federated settings, via cryptographic mechanisms such as Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC). However, these do not always provide formal privacy guarantees, or consider the full range of hyperparameters and implementation settings. In this work, we implement the GBDT model under Differential Privacy (DP). We propose a general framework that captures and extends existing approaches for differentially private decision trees. Our framework of methods is tailored to the federated setting, and we show that with a careful choice of techniques it is possible to achieve very high utility while maintaining strong levels of privacy.more » « less
-
more » « less
To quantify trade-offs between increasing demand for open data sharing and concerns about sensitive information disclosure, statistical data privacy (SDP) methodology analyzes data release mechanisms that sanitize outputs based on confidential data. Two dominant frameworks exist: statistical disclosure control (SDC) and the more recent differential privacy (DP). Despite framing differences, both SDC and DP share the same statistical problems at their core. For inference problems, either we may design optimal release mechanisms and associated estimators that satisfy bounds on disclosure risk measures, or we may adjust existing sanitized output to create new statistically valid and optimal estimators. Regardless of design or adjustment, in evaluating risk and utility, valid statistical inferences from mechanism outputs require uncertainty quantification that accounts for the effect of the sanitization mechanism that introduces bias and/or variance. In this review, we discuss the statistical foundations common to both SDC and DP, highlight major developments in SDP, and present exciting open research problems in private inference.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/EuroSP48549.2020.00041
