%AXu, Depeng%ADu, Wei%AWu, Xintao%BJournal Name: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
%D2021%I
%JJournal Name: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
%K
%MOSTI ID: 10321611
%PMedium: X
%TRemoving Disparate Impact on Model Accuracy in Differentially Private Stochastic Gradient Descent
%XIn differentially private stochastic gradient descent (DPSGD), gradient
clipping and random noise addition disproportionately affect
underrepresented and complex classes and subgroups. As a consequence,
DPSGD has disparate impact: the accuracy of a model
trained using DPSGD tends to decrease more on these classes and
subgroups vs. the original, non-private model. If the original model
is unfair in the sense that its accuracy is not the same across all
subgroups, DPSGD exacerbates this unfairness. In this work, we
study the inequality in utility loss due to differential privacy, which
compares the changes in prediction accuracy w.r.t. each group between
the private model and the non-private model. We analyze
the cost of privacy w.r.t. each group and explain how the group
sample size along with other factors is related to the privacy impact
on group accuracy. Furthermore, we propose a modified DPSGD
algorithm, called DPSGD-F, to achieve differential privacy, equal
costs of differential privacy, and good utility. DPSGD-F adaptively
adjusts the contribution of samples in a group depending on the
group clipping bias such that differential privacy has no disparate
impact on group accuracy. Our experimental evaluation shows the
effectiveness of our removal algorithm on achieving equal costs of
differential privacy with satisfactory utility.
%0Journal Article
Country unknown/Code not availablehttps://doi.org/10.1145/3447548.3467268OSTI-MSA