Abstract: To address the issues of utility degradation in deep learning models under differential privacy protection and the gap between theoretical and actual privacy protection effectiveness, this paper proposes a deep learning differential privacy protection scheme based on adaptive clipping. The scheme optimizes the process through a fourstep mechanism: firstly, gradient adaptive clipping controls the gradient magnitude during training by dynamically adjusting the gradient clipping threshold, thereby enabling the control of the magnitude of noise added subsequently; secondly, group label selection identifies the group with the smallest gradient as the privacypreserving object, and more accurate privacy loss can be obtained by training this group; thirdly, optimized privacy loss calculation combines the gaussian mechanism based on subsampling to reduce the computational overhead of model privacy loss calculation; finally, optimized gradient adaptive descent realizes the adaptive descent of gradients by adjusting the conditional smoothing parameter, thus improving the usability of the model. Experiments were conducted on the VGG architecture using the MNIST, CIFAR10, and MedicalMNIST datasets. The results show that the model accuracy rates after training with this scheme are 81.08%, 72.30%, and 67.91% respectively, representing improvements of 15.60%, 10.60%, and 9.71% compared to the traditional DPSGD, and 0.63%, 2.50%, and 4.40% over the widely used Nadam algorithm in recent years. The model training efficiency has been improved by 35.5% and 39.4%, respectively.

Key words: deep learning, differential privacy, gradient adaptive clipping, label selection, smoothed loss function

摘要： 针对深度学习模型在差分隐私保护下存在效用下降以及理论隐私保护效果与实际效果偏差的问题，提出基于自适应裁剪的深度学习差分隐私保护方案.方案通过4步机制实现优化：梯度自适应裁剪自适应调整梯度裁剪阈值控制训练时梯度大小，实现对后续添加噪声大小的控制；分组标签选择，通过选取梯度最小的组作为隐私保护对象，训练该分组能够得到更准确的隐私损失；优化的隐私损失计算通过结合基于子采样的高斯机制，减小模型隐私损失计算的开销；优化的梯度自适应下降，通过调整条件平滑参数实现梯度的自适应下降，提升模型的可用性.在VGG(visual geometry group)架构下训练MNIST，CIFAR10，MedicalMNIST数据集，结果表明采用该方案训练后模型准确率分别为81.08%，72.30%，67.91%，相比传统的差分隐私随机梯度下降(differential privacy stochastic gradient descent, DPSGD)方法分别提升15.60%，10.60%，9.71%，相比近年常用的Nadam算法分别提升0.63%，2.50%，4.40%，模型训练效率分别提升35.5%，39.4%.

关键词: 深度学习, 差分隐私, 梯度自适应裁剪, 标签选择, 平滑损失函数