Journal of Information Security Reserach ›› 2024, Vol. 10 ›› Issue (7): 586-.

    Next Articles

A Review of GPU Acceleration Technology for Deep Learning in Plaintext  and Private Computing Environments

Qin Zhixiang1, Yang Hongwei1, Hao Meng1, He Hui1, and Zhang Weizhe1,2,3#br#

#br#
  

  1. 1(School of Cyberspace Science, Harbin Institute of Technology, Harbin 150001)
    2(School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, Guangdong 518055)
    3(Department of New Networks, Peng Cheng Laboratory, Shenzhen, Guangdong 518055)

  • Online:2024-07-14 Published:2024-07-14

隐私计算环境下深度学习的GPU加速技术综述

秦智翔1杨洪伟1郝萌1何慧1张伟哲1,2,3


  

  1. 1(哈尔滨工业大学网络空间安全学院哈尔滨150001)
    2(哈尔滨工业大学(深圳)计算机科学与技术学院广东深圳518055)
    3(鹏城实验室新型网络研究部广东深圳518055)

  • 通讯作者: 杨洪伟 博士,助理研究员.主要研究方向为数据挖掘、隐私计算、网络空间安全. yanghongwei@hit.edu.cn
  • 作者简介:秦智翔 硕士研究生.主要研究方向为安全多方计算、大数据安全. qzxqzc@gmail.com 杨洪伟 博士,助理研究员.主要研究方向为数据挖掘、隐私计算、网络空间安全. yanghongwei@hit.edu.cn 郝萌 博士,讲师.主要研究方向为高性能计算、并行应用性能优化. haomeng@hit.edu.cn 何慧 博士,教授.主要研究方向为云计算、数据安全与隐私保护、网络空间安全. hehui@hit.edu.cn 张伟哲 博士,教授.主要研究方向为网络空间安全、数据安全、高性能计算. wzzhang@hit.edu.cn

Abstract: With the continuous development of deep learning technology, the training time of neural network models is getting longer and longer, and using GPU computing to accelerate neural network training has increasingly become a key technology. In addition, the importance of data privacy has also promoted the development of private computing technology. This article first introduces the concepts of deep learning, GPU computing, and two privacy computing technologies, secure multiparty computing and homomorphic encryption, and then discusses the GPU acceleration technology of deep learning in plaintext environment and private computing environment. In the plaintext environment, the two basic deep learning parallel training modes of data parallelism and model parallelism are introduced, two different memory optimization technologies of recalculation and video memory swapping are analyzed, and gradient compression in the training process of distributed neural network is introduced. technology. This paper introduces two deep learning GPU acceleration techniques: Secure multiparty computation and homomorphic encryption in a privacy computing environment. Finally, the similarities and differences of GPUaccelerated deep learning methods in the two environments are briefly analyzed.

Key words: deep learning, GPU (graphics processing unit) computing, private computing, secure multiparty computation, homomorphic encryption

摘要: 随着深度学习技术的不断发展,神经网络模型的训练时间越来越长,使用GPU计算对神经网络训练进行加速便成为一项关键技术.此外,数据隐私的重要性也推动了隐私计算技术的发展.首先介绍了深度学习、GPU计算的概念以及安全多方计算、同态加密2种隐私计算技术,而后探讨了明文环境与隐私计算环境下深度学习的GPU加速技术.在明文环境下,介绍了数据并行和模型并行2种基本的深度学习并行训练模式,分析了重计算和显存交换2种不同的内存优化技术,并介绍了分布式神经网络训练过程中的梯度压缩技术.介绍了在隐私计算环境下安全多方计算和同态加密2种不同隐私计算场景下的深度学习GPU加速技术.简要分析了2种环境下GPU加速深度学习方法的异同.

关键词: 深度学习, GPU计算, 隐私计算, 安全多方计算, 同态加密

CLC Number: