一种基于差分隐私的可追踪深度学习分类器

摘要/Abstract

摘要： 随着深度学习在各个领域的广泛应用，数据收集和训练过程中产生的隐私泄漏问题已成为阻碍人工智能进一步发展的原因之一。目前已有很多研究将深度学习与同态加密或者差分隐私等技术结合以实现对深度学习中的隐私保护。本文从另一个角度尝试解决这个问题，即在一定程度上保证训练数据集的隐私性的基础上，实现对训练数据的计算节点的可追踪性。为此我们提出了一种基于差分隐私的可追踪深度学习分类器，它结合差分隐私和数字指纹技术，在为训练数据集提供隐私保护的同时保证在出现非法传播的训练模型或者数据集时，能根据其中的指纹信息定位到问题训练节点。我们设计的分类器既能保证安全判定分类功能，又能保证指纹的不可感知性、鲁棒性、可信度和可行性等基本特征。从后续的公式推导、理论分析和在真实数据的仿真结果表明，该方案能够满足深度学习中对隐私信息的安全可追踪性的需求。

关键词: 深度学习, 分类器, 差分隐私, 数字指纹, 隐私保护, 可追踪性

Abstract: With the application of deep learning in various fields, privacy leakage in data collection and training has become one of the reasons hindering the further development of artificial intelligence. At present, many studies have combined deep learning with homomorphic encryption or differential privacy technologies to achieve privacy protection in deep learning. Our paper tries to solve this problem from another perspective, that is, to achieve traceability of computing nodes of training data on the basis of guaranteeing some degree of privacy of training data set. Therefore, we propose a trackable deep learning classifier based on differential privacy. It combines differential privacy and digital fingerprint technologies to provide privacy protection for training data sets and ensure that the problem training nodes can be located according to the fingerprint information in training models or data sets that are illegally transmitted. We design the classifier can ensure safety decision classification function, and can guarantee the robustness of fingerprint is perceptual basic characteristics such as reliability and feasibility. From the subsequent formulas derived the theoretical analysis and simulation results on real data show that the scheme can satisfy the deep learning of privacy information safety traceability requirements.

Key words: Deep Learning, Classifier, Differential Privacy, Digital Fingerprinting, Privacy Protection, Traceability

胡韵, 刘嘉驹, 李春国, . 一种基于差分隐私的可追踪深度学习分类器[J]. 信息安全研究, 2022, 8(3): 277-.

参考文献

[1]. Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006,313(5786): 504-507.

[2]. Wang S, Cao J, Yu P. Deep learning for spatiotemporal data mining: A survey[J]. IEEE Transactions on Knowledge and Data Engineering. 2020, 99(2):1-21.

[3]. Young T, Hazarika D, Poria S, et al. Recent trends in deep learning based natural language processing[J]. IEEE Computational Intelligence Magazine, 2018, 13(3):55-75.

[4]. Quintanilla E, Rawat Y S, Sakryukin A, et al. Adversarial learning for personalized tag recommendation[J]. IEEE Transactions on Multimedia, 2021,23(1): 1083–1094.

[5]. Zhang Y, Yao J, Guan H. Intelligent cloud resource management with deep reinforcement learning[J]. IEEE Cloud Computing, 2018, 4(6):60-69.

[6]. Kannan A, Kurach K, Ravi S, et al. Smart reply: Automated response suggestion for email[C] // Proc of the ACM SIGKDD Conference on Knowledge Discovery and Data mining. Washington:ACM, 2016.

[7]. Alipanahi B, Delong A, Weirauch MT, et al. Predicting the sequence specificities of DNA and RNA-binding proteins by deep learning[J]. Nature biotechnology, 2015;33(8):831-839.

[8]. Jain P, Gyanchandani M, Khare N. Differential privacy: its technological prescriptive using big data[J]. Journal of Big Data, 2018, 5(1):15.

[9]. Fredrikson M, Jha S, Ristenpart T. Model inversion attacks that exploit confidence information and basic countermeasures[C] //Proc of the 22nd ACM SIGSAC Conference on Computer and Communications Security. Colorado: ACM, 2015: 1322–1333.

[10]. Phan N H, Wang Y, Wu X, et al. Differential privacy preservation for deep auto-encoders: an application of human behavior prediction [C] //Proc of the 30th AAAI Conference on Artificial Intelligence (AAAI-16). Arizona :AAAI Press, 2016.

[11]. Abadi M, Chu A, Goodfellow I J, et al. Deep learning with differential privacy[C] //Proc of the 2016 ACM SIGSAC Conference on Computer and Communications Security. Vienna, Austria: ACM, 2016:308-318.

[12]. Shokri R, Stronati M, Song C, et al. Membership inference attacks against machine learning Models[C] // Proc of 2017 IEEE Symposium on Security and Privacy (SP), Security and Privacy (SP). CA: IEEE, 2017:3-8.

[13]. 彭祯方，邢国强，陈兴跃. 人工智能在网络安全领域的应用及技术综述[J].信息安全研究，2022, 8(2): 110-116.

[14]. 蒋凯元，多方安全计算研究综述[J]. 信息安全研究，2021，7(12): 1161-1165.

[15]. Orlandi C, Piva A, Barni M, Oblivious neural network computing via homomorphic encryption[J]. Eurasip Journal on Information Security, 2007, 1(2007):1-11.

[16]. Dowlin N, Gilad-Bachrach R, Laine K, et al. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy[R]. USA:Microsoft Research, 2016:76-78.

[17]. Hesamifard E, Takabi H, Ghasemi M, et al. Privacy-preserving machine learning in cloud[C] //Proc of the 2017 ACM Cloud Computing Security Workshop. Scottsdale, Arizona, USA. 2017: 39-43.

[18]. Xie P, Bilenko M, Finley T, et al. Crypto-Nets: Neural networks over encrypted data[J/OL]. Computer Science, 2014, https://search.ebscohost.com/login.aspx?direct=true&db=edsarx&AN=edsarx.1412.6181&lang=zh-cn&site=eds-live.

[19]. Dwork C. Differential privacy[C] // Proc of the 33rd international conference on Automata, Languages and Programming Volume Part II., Berlin, Heidelberg: Springer, 2006.

[20]. Zhu T, Li G, Zhou W, et al. Differential privacy and applications[M]. Cham: Springer, 2017.

[21]. Xie L, Lin K, Wang S, et al. Differentially private generative adversarial network[J/OL]. Axiv preprint arXiv, 2018, https://search.ebscohost.com/login.aspx?direct=true&db=edsarx&AN=edsarx.1802.06739&lang=zh-cn&site=eds-live.

[22]. 毛典辉, 李子沁, 蔡强等. 基于DCGAN反馈的深度差分隐私保护方法[J]. 北京工业大学学报, 2018, 44(6): 870- 877.

[23]. Papernot N, Abadi M, Erlingsson L, et al. Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data[J/OL]. Axiv preprint arXiv, 2016, https: // search. ebscohost.com/login.aspx?direct=true&db=edsarx&AN=edsarx.1610.05755&lang=zh-cn&site=eds-live.

[24]. Missier P, Bryans J, Gamble C. ProvAbs: model, policy, and tooling for abstracting Prov graphs[C] //Proc of the 5th International Provenance and Annotation Workshop. Berlin: Springer, 2014: 3-15.

[25]. Boneh D, Shaw J. Collusion-secure fingerprinting for digital data[J]. IEEE Transactions on Information Theory, 1998, 44(5): 1897-1905.

[26]. Chor B, Fiat A, Naor M. Tracing traitors[C]// Proc of CRYPTO’94 : 14th Annual International Cryptology Conference, California, USA: Computer Science , 1994: 257-270.

[27]. Rodríguez J J, Kuncheva L I, Alonso C J. Rotation forest: A new classifier ensemble method[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(10): 1619-1630.

[28]. Haasdonk B. Feature space interpretation of SVM with indefinite kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(4): 482-492.

[29]. Zeiler M D, Fergus R. Visualizing and understanding convolutional neural networks[C] // Proc of 12th European Conference on Computer Vision. Firenze, Italy: Springer International Publishing, 2013.

[30]. Mitra S, Pal SK, Mitra P, et al. Data mining in soft computing framework: A survey[J]. IEEE Transactions on Neural Networks, 2002, 13(1): 3-14

[31]. Tao W, Li Z, Yan Y, et al. A survey of fuzzy decision treeclassifier methodology[J]. Springer Berlin Heidelberg, 2007,40(-), 959-968.

[32]. Dwork C, Roth A. The algorithmic foundations of differential privacy[J]. Foundations and Trends in Theoretical Computer ence, 2013, 9(3-4): 1-27, 29-117, 119-141, 143-191, 193-219, 221-235, 237-259, 261-267, 269-277, 279-286, a1-a2.

[33]. McSherry F, Talwar K. Mechanism design via differential privacy[C] //Proc of 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07). RI, USA: IEEE, 2007: 94-103.

[34]. McSherry F. Privacy integrated queries: An extensible platform for privacy-preserving data analysis[J]. Communications of the ACM, 2010, 53(9): 89-97.

[35]. Zhang L, Wei D. Dual DCT-DWT-SVD digital watermarking algorithm based on particle swarm optimization[J]. Multimedia Tools and Applications, 2019, 78(19): 28003-28023.

[36]. Singh A K. Improved hybrid algorithm for robust and imperceptible multiple watermarking using digital images[J]. Multimedia Tools and Applications, 2017, 76(6): 8881-8900.

[37]. Chang, C.H, Zhang, et al. A Blind Dynamic Fingerprinting Technique for Sequential Circuit Intellectual Property Protection[J]. IEEE Transactions on Computer Aided Design of Integrated Circuits & Systems A Publication of the IEEE Circuits & Systems Society, 2014, 33(1): 76-89.

[38]. 马英，一种基于隐私计算的数据共享模型研究[J].信息安全研究，2022，8(2): 122-128.