Abstract: Deep learning models enhance performance by memorizing training data, but this also poses a risk of training data leakage. Differential privacy, as a mainstream privacy protection method, effectively mitigates this risk. However, existing differentially private data synthesis approaches suffer from slow model convergence and low data usability. To address these issues, we propose the TVDPSGDLM_D framework. This approach introduces TVDPSGD, a thresholdvalidated differentially private optimization algorithm that incorporates a validation mechanism to filter gradient directions during differentially private model training. By discarding ineffective updates, this approach accelerates model convergence. TVDPSGDLM embeds TVDPSGD into a language generation model to synthesize labeled text datasets that maintain statistical similarity to the original data. Additionally, a pretrained classifier is used to filter the generated text, removing samples where the classification results do not match the assigned labels, thereby improving the quality of the synthetic dataset. Experimental results on public datasets demonstrate that the proposed method preserves data privacy while achieving a classification accuracy of 89.4% on the processed synthetic dataset.

Key words: differential privacy, gradient direction filtering, contrastive filtering, text synthesis, conditional control code

摘要： 深度学习模型通过记忆训练数据提升性能的同时，存在训练数据泄露的风险，差分隐私作为一种主流的隐私保护方法能够有效降低此风险.目前基于差分隐私的数据合成方案存在模型收敛速度较慢、数据可用性较低等问题.为了解决这些问题，提出TVDPSGDLM_D方案，通过阈值验证的差分隐私优化算法TVDPSGD，采用验证机制筛选梯度方向优化差分隐私模型训练的过程，摒弃无效更新提高模型收敛速度；将TVDPSGD作为优化算法嵌入语言生成模型中，得到TVDPSGDLM，生成与原始数据统计分布相似的、带标注的合成初始文本数据集；并利用预训练分类器筛选初始文本，去除分类结果与合成文本标签不一致的样本，提高合成数据的质量，获得满足差分隐私的公开数据集.在公开数据集上的实验结果显示，使用该方案合成并处理后的数据集实现了对数据隐私性的保护，分类准确率可达89.4%.

关键词: 差分隐私, 梯度方向筛选, 对比筛选, 文本合成, 条件控制码