信息安全研究 ›› 2021, Vol. 7 ›› Issue (4): 294-309.
• 学术论文 • 下一篇
徐金才1,2 任民1,2 李琦2 孙哲南2
出版日期:
2021-04-05
发布日期:
2021-04-14
通讯作者:
孙哲南
作者简介:
徐金才,硕士研究生,主要研究方向为对抗样本的攻击和防御.
13527449440@163.com
任民,博士研究生,主要研究方向为虹膜识别的泛化研究。
min.ren@cripac.ia.ac.cn
李琦,副研究员,主要研究方向为为人脸特征识别与安全,人脸属性编辑,人脸深度伪造。
qli@nlpr.ia.ac.cn
孙哲南,研究员,主要研究方向为人脸特征识别与安全,计算机视觉.
znsun@nlpr.ia.ac.cn
Online:
2021-04-05
Published:
2021-04-14
摘要: 计算机性能的提高和深度学习的出现,使得人工智能技术实现了广泛的应用。深度学习模型的安全性问题受到了广泛的关注。对抗样本的存在是深度学习应用场景的主要威胁之一,限制了诸如人脸识别、自动驾驶等隐私安全性要求较高的应用场景。深度学习模型除了需要神经网络有良好的性能外,还需要它有足够的鲁棒性。令人担心的是,深度神经网络是否可以稳定可靠有效的应用在现实世界中?如果我们对深度神经网络的认知仅仅停留在一个黑盒模型,对于输入有良好的输出效果,那很难放心的将它应用在现实中。论文介绍了对抗样本存在的原因,分类归纳了对抗攻击和对抗防御的算法。同时使用MNIST、CIFAR-10、ImageNet数据集对相关代表性的方法进行了实验验证,最后对这一领域的发展趋势进行了展望。
徐金才 任民 李琦 孙哲南. 图像对抗样本的安全性研究概述[J]. 信息安全研究, 2021, 7(4): 294-309.
[1]Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105 [2]He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2016: 770-778 [3]Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]// Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2017: 4700-4708 [4]Tian Y, Pei K, Jana S, et al. Deeptest: Automated testing of deep-neural-network-driven autonomous cars[C]//Proc of the 40th International Conf on Software Engineering. New Work: ACM, 2018: 303-314 [5]段广晗,马春光,宋蕾,等.深度学习中对抗样本的构造及防御研究[J].网络与信息安全学报, 2020,6(2):1-11 [6]张嘉楠,王逸翔,刘博,等.深度学习的对抗攻击方法综述[J].网络空间安全,2019,10(7):87-96 [7]张嘉楠,赵镇东,宣晶,等.深度学习对抗样本的防御方法综述[J].网络空间安全,2019,10(8):93-101 [8]Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014 [9]Szegedy C, Zaremba W, Sutskever I, et al. Intriguing properties of neural networks[C/OL]//Proc of the 2nd Int Conf on Learning Representations.2014[2021-1-29]. https://arxiv.org/pdf/1312.6199 [10]王伟,董晶,何子文,等. 视觉对抗样本生成技术概述[J].信息安全学报,2020,5(2):39-48 [11]蒋凯,易平. 关于对抗样本恢复的研究 [J]. 通信技术,2018,51(12): 2946-2952 [12]McDaniel P,Papernot N,Celik ZB. Machine Learning in adversarial settings[J].IEEE Security&Privacy, 2016, 14(3):68- 72 [13]Meng D, Chen H. Magnet: A two-pronged defense against adversarial examples[C]//Proc of the 2017 ACM SIGSAC Conf on Computer and Communications Security. New Work:ACM, 2017: 135-147 [14]Narayanan H, Mitter S. Sample complexity of testing the manifold hypothesis[C/OL]//Proc of the 23rd International Conference on Neural Information Processing Systems-Volume 2. 2010: 1786-1794.[2021-1-29] https://www.researchgate.net/publication/221619959_Sample_Complexity_of_Testing_the_Manifold_Hypothesis [15]Lawrence N D. A unifying probabilistic perspective for spectral dimensionality reduction: Insights and new models[J/OL]. Journal of Machine Learning Research, 2012, 13.[2021-1-29] https://arxiv.org/search/?query=A+unifying+probabilistic+perspective+for+spectral+dimensionality+reduction%3A+Insights+and+new+models&searchtype=all&abstracts=show&order=-announced_date_first&size=50 [16]Goodfellow I, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[C/OL]//Proc of the 3rd Int Conf on Learning Representations.2 015[2021-1-29]. https://arxiv.org/pdf/1412.6572 [17]Papernot N, McDaniel P, Goodfellow I. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016 [18]Geirhos R, Rubisch P, Michaelis C, et al. ImageNet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness[J]. arXiv preprint arXiv:1811.12231, 2018 [19]kurffzhou, 深度学习系统为什么容易受到对抗样本的欺骗?https://zhuanlan.zhihu.com/p/89665397 [20]Ilyas A, Santurkar S, Tsipras D, et al. Adversarial examples are not bugs, they are features[J]. arXiv preprint arXiv:1905.02175, 2019 [21]Duan R, Ma X, Wang Y, et al. Adversarial camouflage: Hiding physical-world attacks with natural styles[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE , 2020: 1000-1008 [22]Miyato T , Dai A M , Goodfellow I . Adversarial training methods for semi-supervised text classification[C/OL]// International Conference on Learning Representations.2016. [2021-1-29].https://www.researchgate.net/publication/303521296_Adversarial_Training_Methods_for_Semi-Supervised_Text_Classification [23]Kurakin A, Goodfellow I, Bengio S. Adversarial machine learning at scale [C]// Proc of the 5th International Conference on Learning Representations. Piscataway, NJ: IEEE, 2017: 1–17 [24]Dong Y, Liao F, Pang T, et al. Boosting adversarial attacks with momentum[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2018: 9185-9193 [25]Polyak B T. Some methods of speeding up the convergence of iteration methods[J].Ussr computational mathematics and mathematical physics, 1964, 4(5): 1-17 [26]Li M, Deng C, Li T, et al. Towards transferable targeted attack[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2020: 641-649 [27]Xie C, Zhang Z, Zhou Y, et al. Improving transferability of adversarial examples with input diversity[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2019: 2730-2739 [28]Moosavi-Dezfooli S M , Fawzi A , Frossard P .Deepfool:a simple and accurate method to fool deep neural networks[C]// Proc of Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2016:2574-2582 [29]Carlini N, Wagner D. Towards evaluating the robustness of neural networks[C]//Proc of Symp on Security and Privacy(SP). Piscataway, NJ:IEEE, 2017:39-57 [30]Papernot N, McDaniel P, Jha S, et al. The limitations of deep learning in adversarial settings[C]//Proc of European symposium on Security and Privacy (EuroS&P).Piscataway, NJ:IEEE, 2016: 372-387 [31]Moosavi-Dezfooli S M, Fawzi A, Fawzi O, et al. Universal adversarial perturbations[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2017: 1765-1773 [32]Athalye A, Carlini N, Wagner D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples[C/OL]//Proc of the 35th International Conference on Machine Learning. 2018: 436-448.[2021-1-29] https://arxiv.org/pdf/1802.00420 [33]Zhang X, Trmal J, Povey D, et al. Improving deep neural network acoustic models using generalized maxout networks[C]//Proc of Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2014: 215-219 [34]Chen P Y, Zhang H, Sharma Y, et al. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models[C]//Proc of the 10th ACM Workshop on Artificial Intelligence and Security. New Work: ACM , 2017: 15-26 [35]Ru B, Cobb A, Blaas A, et al. Bayesopt adversarial attack[C/OL]//Int Conf on Learning Representations, 2019[2021-1-29]. https://scholar.google.com.sg/scholar?hl=zh-CN&as_sdt=0%2C5&as_vis=1&q=Bayesopt+adversarial+attack&btnG= [36]Meunier L, Atif J, Teytaud O. Yet another but more efficient black-box adversarial attack: tiling and evolution strategies[J]. arXiv preprint arXiv:1910.02244, 2019 [37]Du J, Zhang H, Zhou J T, et al. Query-efficient meta attack to deep neural networks[J]. arXiv preprint arXiv:1906.02398, 2019 [38]Narodytska N, Kasiviswanathan S P. Simple black-box adversarial perturbations for deep networks[J]. arXiv preprint arXiv:1612.06299, 2016 [39]Athalye A, Engstrom L, Ilyas A, et al. Synthesizing robust adversarial examples[C/OL]//International conference on machine learning. PMLR, 2018: 284-293[2021-1-29]. https://arxiv.org/pdf/1707.07397 [40]McLaren K. XIII—The development of the cie 1976 (L* a* b*) uniform colour space and colour‐difference formula[J]. Journal of the Society of Dyers and Colourists, 1976, 92(9): 338-341 [41]Brendel W, Rauber J, Bethge M. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models[J]. arXiv preprint arXiv:1712.04248, 2017 [42]Xiao C , Li B , Zhu J Y , et al. Generating adversarial examples with Adversarial Networks[C/OL]//Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18.2018[2021-1-29]. https://www.researchgate.net/publication/322328780_Generating_adversarial_examples_with_adversarial_networks [43]Xiao C, Zhu J Y, Li B, et al. Spatially transformed adversarial examples[J]. arXiv preprint arXiv:1801.02612, 2018 [44]Papernot N, McDaniel P, Goodfellow I, et al. Practical black-box attacks against machine learning[C]//Proc of the Asia Conf on Computer and Communications Security. New Work: ACM, 2017: 506-519 [45]Shi Y C, Wang SY, Han Y H. Curls & Whey: Boosting Black-Box Adversarial Attacks[C]//Proc of the Conf on Computer Vision and Pattern Recognition (CVPR).Piscataway, NJ: IEEE, 2019: 6519-6527 [46]Xie C, Zhang Z, Zhou Y, et al. Improving transferability of adversarial examples with input diversity[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 2730-2739 [47]Yanpei Liu, Xinyun Chen, Chang Liu, et al. Delving into transferable adversarial examples and black-box attacks[C/OL].5th International Conference on Learning Representations, ICLR 2017 -Conference Track Proceedings, 2019: 1-7.[2021-1-29]https://arxiv.org/pdf/1611.02770 [48]张文翔.基于批量梯度的对抗样本生成方法的研究[D].武汉:华中科技大学,2019 [49]Sharif M, Bhagavatula S, Bauer L, et al. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition[C]//Proc of the Sigsac Conf on Computer and Communications Security. New Work: ACM , 2016: 1528-1540 [50]Eykholt K, Evtimov I, Fernandes E, et al. Robust physical- world attacks on deep learning visual classification[C]//Proc of the Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 1625-1634 [51]Brown T B, Mané D, Roy A, et al. Adversarial patch[J]. arXiv preprint arXiv:1712.09665, 2017 [52]Dziugaite G K, Ghahramani Z, Roy D M. A study of the effect of jpg compression on adversarial images[J].arXiv preprint arXiv:1608.00853, 2016 [53]Das N, Shanbhogue M, Chen S T, et al. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression[J]. arXiv preprint arXiv:1705.02900, 2017 [54]Liu Z, Liu Q, Liu T, et al. Feature distillation: Dnn-oriented jpeg compression against adversarial examples[C]//Proc of Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2019: 860-868 [55]Jia X, Wei X, Cao X, et al. Comdefend: An efficient image compression model to defend adversarial examples[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 6084-6092 [56]Raff E, Sylvester J, Forsyth S, et al. Barrage of random transforms for adversarially robust defense[C]//Proc of the Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 6528-6537 [57]Guo C, Rana M, Cisse M, et al. Countering adversarial images using input transformations[J].arXiv preprint arXiv:1711.00117, 2017 [58]Prakash A, Moran N, Garber S, et al. Deflecting adversarial attacks with pixel deflection[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 8571-8580 [59]Chang S G, Yu B, Vetterli M. Adaptive wavelet thresholding for image denoising and compression[J]. IEEE Trans on Image Processing, 2000, 9(9): 1532-1546 [60]Sun B, Tsai N, Liu F, et al. Adversarial defense by stratified convolutional sparse coding[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 11447-11456 [61]Osadchy M , Hernandez-Castro J , Gibson S , et al. No bot expects the deep captcha! Introducing immutable adversarial examples, with applications to captcha generation[J].IEEE Trans on Information Forensics and Security, 2019, 12(11):2640-2653 [62]Liao F, Liang M, Dong Y, et al. Defense against adversarial attacks using high-level representation guided denoiser[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 1778-1787 [63]Ronneberger O , Fischer P , Brox T . U-Net: Convolutional networks for biomedical image segmentation[C]// Int Conf on Medical Image Computing and Computer-Assisted Intervention. New work :Springer, 2015 [64]Mustafa A, Khan S H, Hayat M, et al.Image super-resolution as a defense against adversarial attacks[J]. IEEE Trans on Image Processing, 2019, 29(10): 1711-1724 [65]Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proc of the Conf on Computer Vision and Pattern Recognition Workshops(CVPR). Piscataway, NJ: IEEE , 2017: 136-144 [66]Yang Y Y, Rashtchian C, Zhang H, et al. A closer look at accuracy vs. robustness[J/OL].Advances in Neural Information Processing Systems, 2020[2021-1-29]. https://arxiv.org/pdf/2003.02460.pdf [67]Kurakin A, Goodfellow I, Bengio S. Adversarial examples in the physical world[C/OL]//Proc of the 5th Int Conf on Learning Representations.2017[2021-1-29]. https://arxiv.org/pdf/1607.02533 [68]Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C/OL]//International conference on machine learning. PMLR, 2015: 448-456[2021-1-29]. https://arxiv.org/pdf/1502.03167 [69]Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint arXiv:1706.06083, 2017. [70]Sharma Y, Ding G W, Brubaker M. On the effectiveness of low frequency perturbations[J]. arXiv preprint arXiv:1903.00073, 2019 [71]刘野,黄贤英,刘文星,等.基于自适应噪声添加的防御对抗样本的算法[J/OL].计算机应用研究.[2021-1-29] https://doi.org/10.19734/j.issn.1001-3695.2020.03.0055 [72]Gu S, Rigazio L. Towards deep neural network architectures robust to adversarial examples[C/OL].ICLR (Workshop). 2015.[2021-1-29].https://arxiv.org/pdf/1412.5068.pdf [73]Ross A, Doshi-Velez F. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park,CA:AAAI, 2018, 32(1) [74]刘嘉阳.针对图像分类的对抗样本防御方法研究[D].合肥:中国科学技术大学,2020 [75]Papernot N,McDaniel P,Wu X,et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]//Proc of Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2016: 582-597 [76]Hinton G , Vinyals O , Dean J . Distilling the knowledge in a neural network[J]. Computer ence, 2015, 14(7):38-39 [77]Lee H, Han S, Lee J. Generative adversarial trainer: Defense to adversarial perturbations with gan[J].arXiv preprint arXiv:1705.03387, 2017 [78]Samangouei P, Kabkab M, Chellappa R. Defense-gan: Protecting classifiers against adversarial attacks using generative models[J]. arXiv preprint arXiv:1805.06605, 2018 [79]Song Y, Kim T, Nowozin S, et al. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples[J]. arXiv preprint arXiv:1710.10766, 2017 [80]Xu W, Evans D, Qi Y. Feature squeezing:Detecting adversarial examples in deep neural networks[J]. arXiv preprint arXiv:1704.01155, 2017 [81]王晓鹏,罗威,秦克,等.一种针对快速梯度下降对抗攻 击的防御方法[J/OL].计算机工程.[2021-1-29]. https://doi.org/10.19678/j.issn.1000-3428.0059367 [82]陈岳峰,毛潇峰,李裕宏,等.AI安全-对抗样本技术综述与应用[J].信息安全研究,2019,5(11):1000-1007 |
[1] | 冯科 阮树骅 陈兴蜀 王海舟 王文贤 蒋术语. 基于联合模型的网络舆情事件检测方法 [J]. 信息安全研究, 2021, 7(3): 207-214. |
[2] | 魏国富 石英村. 人工智能数据安全治理与技术发展概述?[J]. 信息安全研究, 2021, 7(2): 110-119. |
[3] | 黄陈辰. 人工智能时代侵犯著作权罪的智能化倾向与刑法应对[J]. 信息安全研究, 2020, 6(9): 0-0. |
[4] | 邱波. 滥用爬虫技术的刑事风险与刑法应对[J]. 信息安全研究, 2020, 6(9): 0-0. |
[5] | 郭小波 李景华 蒋才平 田青 赵洋. 网络身份管理体系发展展望[J]. 信息安全研究, 2020, 6(7): 602-607. |
[6] | 张行. 人工智能在手写签字鉴定应用中的研究[J]. 信息安全研究, 2020, 6(7): 622-633. |
[7] | 高威 萧子豪 朱益灵. DeepFake技术背后的安全问题:机遇与挑战[J]. 信息安全研究, 2020, 6(7): 634-644. |
[8] | 高一骄. AI+生物识别技术对可信身份认证的挑战?[J]. 信息安全研究, 2020, 6(7): 645-651. |
[9] | 肖喜生 彭凯飞 龙春 魏金侠 赵静. 基于人工智能的安全态势预测技术研究综述[J]. 信息安全研究, 2020, 6(6): 0-0. |
[10] | 赵宇航 马修军. 一种高效率的多智能体协作学习通信机制[J]. 信息安全研究, 2020, 6(4): 345-349. |
[11] | 陈阳. 著作权法下“换脸技术”的法律约束缺位与规制路径[J]. 信息安全研究, 2020, 6(12): 1109-1117. |
[12] | 雷惊鹏. 基于云计算和深度学习的协议监测系统设计[J]. 信息安全研究, 2020, 6(12): 1127-1132. |
[13] | 周琳娜 吕欣一. 基于GAN图像生成的信息隐藏技术综述[J]. 信息安全研究, 2019, 5(9): 771-777. |
[14] | 李创丰 李云龙 孙伟. 基于CNN和朴素贝叶斯方法的安卓恶意 应用检测算法[J]. 信息安全研究, 2019, 5(6): 470-476. |
[15] | 李敏 杨阳 王钤 孟博 李凌寒 白入文 杜虹. 基于智能视频分析的人流量态势感知方法研究[J]. 信息安全研究, 2019, 5(6): 488-494. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||