信息安全研究 ›› 2024, Vol. 10 ›› Issue (12): 1153-.

• 综合安全防御体系专题 • 上一篇    下一篇

大模型应用可信框架研究

韦韬1刘焱1翁海琴1仲震宇1朱泽韬1王宇1王美琴2   

  1. 1(蚂蚁集团安全可信事业部杭州310023)
    2(山东大学网络空间安全学院山东青岛266237)
  • 出版日期:2024-12-25 发布日期:2024-12-30
  • 通讯作者: 韦韬 博士.主要研究方向为复杂系统和主流操作系统的安全性、可靠性,安全软件开源发展. lenx.wei@antgroup.com
  • 作者简介:韦韬 博士.主要研究方向为复杂系统和主流操作系统的安全性、可靠性,安全软件开源发展. lenx.wei@antgroup.com 刘焱 主要研究方向为数据安全和AI安全. bencao.ly@antgroup.com 翁海琴 博士.主要研究方向为异常检测、AI安全和AI对抗智能算法. haiqin.wenghaiqin@antgroup.com 仲震宇 博士.主要研究方向为AI对抗智能、大模型在垂直安全领域的可信安全. edward.zhong@antgroup.com 朱泽韬 主要研究方向为安全攻防与安全大模型. zetao.zzt@antgroup.com 王宇 硕士.主要研究方向为基础设施与办公网安全、应用安全、流量安全、移动与IOT安全、威胁情报、对抗智能及高等级安全攻防. dy.xiaoyu@antgroup.com 王美琴 博士,教授.主要研究方向为对称密码分析与设计. mqwang@sdu.edu.cn

A Trust Framework for Large Language Model Application

Wei Tao1, Liu Yan1, Weng Haiqin1, Zhong Zhenyu1, Zhu Zetao1, Wang Yu1, and Wang Meiqin2   

  1. 1(Department of Security and Trust Division, Ant Group, Hangzhou 310023)
    2(School of Cyber Science and Technology, Shandong University, Qingdao, Shandong 266237)
  • Online:2024-12-25 Published:2024-12-30

摘要: 大模型技术的出现极大推动了人工智能在各行各业的快速应用,但大模型在实际应用过程中面临着一系列由“模型幻觉”导致的安全可信挑战.这些挑战导致大模型应用落地时,尤其是安全攸关的专业性推理和研判时,其给出的结果难以被轻易信任采纳.在诸多专业领域实践中发现,大模型应用过程中缺乏一个统一的技术框架保证其结果的可信,严重阻碍了大模型技术在专业领域的应用. 针对这个问题,结合在金融、医疗、安全等专业领域的应用实践,提出一种集充足数据(data,D)、专业知识(knowledge,K)、能力协同(collaboration,C)、高效反馈(feedback,F)为一体的大模型可信应用框架DKCF. 希望DKCF可以推动行业大模型安全可信,助力大模型应用变革,推动越来越多的行业迎来革命.

关键词: DKCF, 大模型, 可信, 可信框架, 网络安全

Abstract: The emergence of large language model has greatly propelled the rapid application of artificial intelligence across various domains. In practice, however, there are a series of security and trust challenges in the applications of large language models caused by “model hallucinations”. These challenges make it difficult for practical applications to trust and adopt the results returned by the large language models, especially in securityrelated application domains. In many professional fields, we find that there lacks a unified technical framework to ensure the trustworthiness of results returned by large language models, which seriously hinders the application of largescale model technology in professional fields. To address this issue, a largescale model trusted application framework DKCF, integrating sufficient data (D), expertise knowledge (K), intellectual collaboration (C), and efficient feedback (F), is proposed. This framework is developed based on our practical applications in professional fields such as finance, healthcare, and security. We believe that DKCF can shed light on secure and reliable applications of large language models, and facilitate the intellectual revolution across various professional domains.

Key words: DKCF, large language model, trustworthiness, trust framework, cyber security

中图分类号: