Summary of The Security Of Image Adversarial Samples

Abstract

Abstract: The improvement of computer performance and the emergence of deep learning have made artificial intelligence technology widely used. More people are paying their attention to the security of deep learning models. The existence of adversarial samples is one of the main threats of deep learning models, which limits their application scenarios such as face recognition systems and self-driving that require high privacy security. Despite the high performance, deep learning models are also required to be sufficiently robust. However, one might be concerned with whether deep neural networks can be applied in real-world applications stably, reliably and effectively? If our understanding of deep neural networks is only a black box model and only requires it to produce a satisfying output given an input, then it would be difficult to safely apply it in reality. Research on adversarial examples is also a hot spot. In this paper, we explain why adversarial examples exist and summarize some algorithms for both adversarial attack and defense. Meanwhile, we conduct experimental verification of several representative methods on MNIST, CIFAR-10, and ImageNet. In the end, we discuss the outlook and trends of this field.

Key words: Adversarial Samples, Adversarial Attack, Adversarial Defense, Privacy Security, Artificial Intelligence, Deep Learning

摘要： 计算机性能的提高和深度学习的出现，使得人工智能技术实现了广泛的应用。深度学习模型的安全性问题受到了广泛的关注。对抗样本的存在是深度学习应用场景的主要威胁之一，限制了诸如人脸识别、自动驾驶等隐私安全性要求较高的应用场景。深度学习模型除了需要神经网络有良好的性能外，还需要它有足够的鲁棒性。令人担心的是，深度神经网络是否可以稳定可靠有效的应用在现实世界中？如果我们对深度神经网络的认知仅仅停留在一个黑盒模型，对于输入有良好的输出效果，那很难放心的将它应用在现实中。论文介绍了对抗样本存在的原因，分类归纳了对抗攻击和对抗防御的算法。同时使用MNIST、CIFAR-10、ImageNet数据集对相关代表性的方法进行了实验验证，最后对这一领域的发展趋势进行了展望。

关键词: 对抗样本, 对抗攻击, 对抗防御, 隐私安全, 人工智能, 深度学习

徐金才任民李琦孙哲南. 图像对抗样本的安全性研究概述[J]. 信息安全研究, 2021, 7(4): 294-309.

References

[1]Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems, 2012, 25: 1097-1105
[2]He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2016: 770-778
[3]Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]// Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2017: 4700-4708
[4]Tian Y, Pei K, Jana S, et al. Deeptest: Automated testing of deep-neural-network-driven autonomous cars[C]//Proc of the 40th International Conf on Software Engineering. New Work: ACM, 2018: 303-314
[5]段广晗,马春光，宋蕾，等.深度学习中对抗样本的构造及防御研究[J].网络与信息安全学报, 2020,6(2):1-11
[6]张嘉楠，王逸翔，刘博，等.深度学习的对抗攻击方法综述[J].网络空间安全，2019,10(7):87-96
[7]张嘉楠，赵镇东，宣晶，等.深度学习对抗样本的防御方法综述[J].网络空间安全，2019,10(8):93-101
[8]Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014
[9]Szegedy C, Zaremba W, Sutskever I, et al. Intriguing properties of neural networks[C/OL]//Proc of the 2nd Int Conf on Learning Representations.2014[2021-1-29].
https://arxiv.org/pdf/1312.6199
[10]王伟，董晶，何子文，等. 视觉对抗样本生成技术概述[J].信息安全学报，2020,5(2):39-48
[11]蒋凯，易平. 关于对抗样本恢复的研究 [J]. 通信技术，2018，51(12): 2946-2952
[12]McDaniel P,Papernot N,Celik ZB. Machine Learning in adversarial settings[J].IEEE Security&Privacy, 2016, 14(3):68- 72
[13]Meng D, Chen H. Magnet: A two-pronged defense against adversarial examples[C]//Proc of the 2017 ACM SIGSAC Conf on Computer and Communications Security. New Work:ACM, 2017: 135-147
[14]Narayanan H, Mitter S. Sample complexity of testing the manifold hypothesis[C/OL]//Proc of the 23rd International Conference on Neural Information Processing Systems-Volume 2. 2010: 1786-1794.[2021-1-29]
https://www.researchgate.net/publication/221619959_Sample_Complexity_of_Testing_the_Manifold_Hypothesis
[15]Lawrence N D. A unifying probabilistic perspective for spectral dimensionality reduction: Insights and new models[J/OL]. Journal of Machine Learning Research, 2012, 13.[2021-1-29]
https://arxiv.org/search/?query=A+unifying+probabilistic+perspective+for+spectral+dimensionality+reduction%3A+Insights+and+new+models&searchtype=all&abstracts=show&order=-announced_date_first&size=50
[16]Goodfellow I, Shlens J, Szegedy C. Explaining and harnessing adversarial examples[C/OL]//Proc of the 3rd Int Conf on Learning Representations.2 015[2021-1-29].
https://arxiv.org/pdf/1412.6572
[17]Papernot N, McDaniel P, Goodfellow I. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples[J]. arXiv preprint arXiv:1605.07277, 2016
[18]Geirhos R, Rubisch P, Michaelis C, et al. ImageNet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness[J]. arXiv preprint arXiv:1811.12231, 2018
[19]kurffzhou, 深度学习系统为什么容易受到对抗样本的欺骗？https://zhuanlan.zhihu.com/p/89665397
[20]Ilyas A, Santurkar S, Tsipras D, et al. Adversarial examples are not bugs, they are features[J]. arXiv preprint arXiv:1905.02175, 2019
[21]Duan R, Ma X, Wang Y, et al. Adversarial camouflage: Hiding physical-world attacks with natural styles[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE , 2020: 1000-1008
[22]Miyato T , Dai A M , Goodfellow I . Adversarial training methods for semi-supervised text classification[C/OL]// International Conference on Learning Representations.2016. [2021-1-29].https://www.researchgate.net/publication/303521296_Adversarial_Training_Methods_for_Semi-Supervised_Text_Classification
[23]Kurakin A, Goodfellow I, Bengio S. Adversarial machine learning at scale [C]// Proc of the 5th International Conference on Learning Representations. Piscataway, NJ: IEEE, 2017: 1–17
[24]Dong Y, Liao F, Pang T, et al. Boosting adversarial attacks with momentum[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2018: 9185-9193
[25]Polyak B T. Some methods of speeding up the convergence of iteration methods[J].Ussr computational mathematics and mathematical physics, 1964, 4(5): 1-17
[26]Li M, Deng C, Li T, et al. Towards transferable targeted attack[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2020: 641-649
[27]Xie C, Zhang Z, Zhou Y, et al. Improving transferability of adversarial examples with input diversity[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2019: 2730-2739
[28]Moosavi-Dezfooli S M , Fawzi A , Frossard P .Deepfool:a simple and accurate method to fool deep neural networks[C]// Proc of Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2016:2574-2582
[29]Carlini N, Wagner D. Towards evaluating the robustness of neural networks[C]//Proc of Symp on Security and Privacy(SP). Piscataway, NJ:IEEE, 2017:39-57
[30]Papernot N, McDaniel P, Jha S, et al. The limitations of deep learning in adversarial settings[C]//Proc of European symposium on Security and Privacy (EuroS&P).Piscataway, NJ:IEEE, 2016: 372-387
[31]Moosavi-Dezfooli S M, Fawzi A, Fawzi O, et al. Universal adversarial perturbations[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ:IEEE, 2017: 1765-1773
[32]Athalye A, Carlini N, Wagner D. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples[C/OL]//Proc of the 35th International Conference on Machine Learning. 2018: 436-448.[2021-1-29]
https://arxiv.org/pdf/1802.00420
[33]Zhang X, Trmal J, Povey D, et al. Improving deep neural network acoustic models using generalized maxout networks[C]//Proc of Int Conf on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2014: 215-219
[34]Chen P Y, Zhang H, Sharma Y, et al. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models[C]//Proc of the 10th ACM Workshop on Artificial Intelligence and Security. New Work: ACM , 2017: 15-26
[35]Ru B, Cobb A, Blaas A, et al. Bayesopt adversarial attack[C/OL]//Int Conf on Learning Representations, 2019[2021-1-29].
https://scholar.google.com.sg/scholar?hl=zh-CN&as_sdt=0%2C5&as_vis=1&q=Bayesopt+adversarial+attack&btnG=
[36]Meunier L, Atif J, Teytaud O. Yet another but more efficient black-box adversarial attack: tiling and evolution strategies[J]. arXiv preprint arXiv:1910.02244, 2019
[37]Du J, Zhang H, Zhou J T, et al. Query-efficient meta attack to deep neural networks[J]. arXiv preprint arXiv:1906.02398, 2019
[38]Narodytska N, Kasiviswanathan S P. Simple black-box adversarial perturbations for deep networks[J]. arXiv preprint arXiv:1612.06299, 2016
[39]Athalye A, Engstrom L, Ilyas A, et al. Synthesizing robust adversarial examples[C/OL]//International conference on machine learning. PMLR, 2018: 284-293[2021-1-29].
https://arxiv.org/pdf/1707.07397
[40]McLaren K. XIII—The development of the cie 1976 (L* a* b*) uniform colour space and colour‐difference formula[J]. Journal of the Society of Dyers and Colourists, 1976, 92(9): 338-341
[41]Brendel W, Rauber J, Bethge M. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models[J]. arXiv preprint arXiv:1712.04248, 2017
[42]Xiao C , Li B , Zhu J Y , et al. Generating adversarial examples with Adversarial Networks[C/OL]//Twenty-Seventh International Joint Conference on Artificial Intelligence IJCAI-18.2018[2021-1-29].
https://www.researchgate.net/publication/322328780_Generating_adversarial_examples_with_adversarial_networks
[43]Xiao C, Zhu J Y, Li B, et al. Spatially transformed adversarial examples[J]. arXiv preprint arXiv:1801.02612, 2018
[44]Papernot N, McDaniel P, Goodfellow I, et al. Practical black-box attacks against machine learning[C]//Proc of the Asia Conf on Computer and Communications Security. New Work: ACM, 2017: 506-519
[45]Shi Y C, Wang SY, Han Y H. Curls & Whey: Boosting Black-Box Adversarial Attacks[C]//Proc of the Conf on Computer Vision and Pattern Recognition (CVPR).Piscataway, NJ: IEEE, 2019: 6519-6527
[46]Xie C, Zhang Z, Zhou Y, et al. Improving transferability of adversarial examples with input diversity[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 2730-2739
[47]Yanpei Liu, Xinyun Chen, Chang Liu, et al. Delving into transferable adversarial examples and black-box attacks[C/OL].5th International Conference on Learning Representations, ICLR 2017 -Conference Track Proceedings, 2019: 1-7.[2021-1-29]https://arxiv.org/pdf/1611.02770
[48]张文翔.基于批量梯度的对抗样本生成方法的研究[D].武汉：华中科技大学，2019
[49]Sharif M, Bhagavatula S, Bauer L, et al. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition[C]//Proc of the Sigsac Conf on Computer and Communications Security. New Work: ACM , 2016: 1528-1540
[50]Eykholt K, Evtimov I, Fernandes E, et al. Robust physical- world attacks on deep learning visual classification[C]//Proc of the Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 1625-1634
[51]Brown T B, Mané D, Roy A, et al. Adversarial patch[J]. arXiv preprint arXiv:1712.09665, 2017
[52]Dziugaite G K, Ghahramani Z, Roy D M. A study of the effect of jpg compression on adversarial images[J].arXiv preprint arXiv:1608.00853, 2016
[53]Das N, Shanbhogue M, Chen S T, et al. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression[J]. arXiv preprint arXiv:1705.02900, 2017
[54]Liu Z, Liu Q, Liu T, et al. Feature distillation: Dnn-oriented jpeg compression against adversarial examples[C]//Proc of Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2019: 860-868
[55]Jia X, Wei X, Cao X, et al. Comdefend: An efficient image compression model to defend adversarial examples[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 6084-6092
[56]Raff E, Sylvester J, Forsyth S, et al. Barrage of random transforms for adversarially robust defense[C]//Proc of the Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 6528-6537
[57]Guo C, Rana M, Cisse M, et al. Countering adversarial images using input transformations[J].arXiv preprint arXiv:1711.00117, 2017
[58]Prakash A, Moran N, Garber S, et al. Deflecting adversarial attacks with pixel deflection[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 8571-8580
[59]Chang S G, Yu B, Vetterli M. Adaptive wavelet thresholding for image denoising and compression[J]. IEEE Trans on Image Processing, 2000, 9(9): 1532-1546
[60]Sun B, Tsai N, Liu F, et al. Adversarial defense by stratified convolutional sparse coding[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2019: 11447-11456
[61]Osadchy M , Hernandez-Castro J , Gibson S , et al. No bot expects the deep captcha! Introducing immutable adversarial examples, with applications to captcha generation[J].IEEE Trans on Information Forensics and Security, 2019, 12(11):2640-2653
[62]Liao F, Liang M, Dong Y, et al. Defense against adversarial attacks using high-level representation guided denoiser[C]//Proc of the Conf on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ: IEEE, 2018: 1778-1787
[63]Ronneberger O , Fischer P , Brox T . U-Net: Convolutional networks for biomedical image segmentation[C]// Int Conf on Medical Image Computing and Computer-Assisted Intervention. New work :Springer, 2015
[64]Mustafa A, Khan S H, Hayat M, et al.Image super-resolution as a defense against adversarial attacks[J]. IEEE Trans on Image Processing, 2019, 29(10): 1711-1724
[65]Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proc of the Conf on Computer Vision and Pattern Recognition Workshops(CVPR). Piscataway, NJ: IEEE , 2017: 136-144
[66]Yang Y Y, Rashtchian C, Zhang H, et al. A closer look at accuracy vs. robustness[J/OL].Advances in Neural Information Processing Systems, 2020[2021-1-29].
https://arxiv.org/pdf/2003.02460.pdf
[67]Kurakin A, Goodfellow I, Bengio S. Adversarial examples in the physical world[C/OL]//Proc of the 5th Int Conf on Learning Representations.2017[2021-1-29].
https://arxiv.org/pdf/1607.02533
[68]Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift[C/OL]//International conference on machine learning. PMLR, 2015: 448-456[2021-1-29].
https://arxiv.org/pdf/1502.03167
[69]Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks[J]. arXiv preprint arXiv:1706.06083, 2017.
[70]Sharma Y, Ding G W, Brubaker M. On the effectiveness of low frequency perturbations[J]. arXiv preprint arXiv:1903.00073, 2019
[71]刘野，黄贤英，刘文星，等．基于自适应噪声添加的防御对抗样本的算法[J/OL].计算机应用研究.[2021-1-29]
https://doi.org/10.19734/j.issn.1001-3695.2020.03.0055
[72]Gu S, Rigazio L. Towards deep neural network architectures robust to adversarial examples[C/OL].ICLR (Workshop). 2015.[2021-1-29].https://arxiv.org/pdf/1412.5068.pdf
[73]Ross A, Doshi-Velez F. Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients[C]//Proc of the AAAI Conf on Artificial Intelligence. Menlo Park,CA:AAAI, 2018, 32(1)
[74]刘嘉阳.针对图像分类的对抗样本防御方法研究[D].合肥:中国科学技术大学，2020
[75]Papernot N，McDaniel P，Wu X，et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]//Proc of Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2016: 582－597
[76]Hinton G , Vinyals O , Dean J . Distilling the knowledge in a neural network[J]. Computer ence, 2015, 14(7):38-39
[77]Lee H, Han S, Lee J. Generative adversarial trainer: Defense to adversarial perturbations with gan[J].arXiv preprint arXiv:1705.03387, 2017
[78]Samangouei P, Kabkab M, Chellappa R. Defense-gan: Protecting classifiers against adversarial attacks using generative models[J]. arXiv preprint arXiv:1805.06605, 2018
[79]Song Y, Kim T, Nowozin S, et al. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples[J]. arXiv preprint arXiv:1710.10766, 2017
[80]Xu W, Evans D, Qi Y. Feature squeezing:Detecting adversarial examples in deep neural networks[J]. arXiv preprint arXiv:1704.01155, 2017
[81]王晓鹏，罗威，秦克，等.一种针对快速梯度下降对抗攻
击的防御方法[J/OL].计算机工程.[2021-1-29].
https://doi.org/10.19678/j.issn.1000-3428.0059367
[82]陈岳峰，毛潇峰，李裕宏，等.AI安全-对抗样本技术综述与应用[J].信息安全研究，2019，5(11):1000-1007