Abstract: Large language models (LLMs) often exhibit hallucinations in various occasions, leading to unreliable inferences. Such vulnerabilities render it  critical for LLMs to be adopted cautiously in vertical domains such as financial, medical, and cybersecurity domains. In preLLM era, humans have accumulated the best practices to ensure reliabilities of complicated tasks through careful engineering. Standard operating procedures (SOP) and Check List are the exemplars of these best practices. Likewise, in LLM era, we propose highorder program (HOP)to achieve the reliability breakthroughs. By fusing both accurate execution of traditional programing languages, and superior knowledge intrinsics of LLMs, HOP sets the backbone of the control system required by vertical LLM applications. HOP achieves automations by leveraging key vertical knowledge and practices. More importantly, it delivers expected reliability through verifications. HOP itself can be autogenerated by LLMs, which further incentivizes its wide adoptions. Lately, we have applied HOP in different scenarios including fulllifecycle financial risk management in cryptographic computing settings, duplicate charges in medical diagnosis, and intrusion detection. HOP has achieved 5 to 10 folds of efficiency improvement, and an accuracy as good as 99% across aforementioned scenarios.

Key words: LLM, highorder program, verification, reliability, hallucination

摘要： 大模型因幻觉导致可靠性不足，难以满足专业领域(如金融、医疗、网络安全等)对精确性和可靠性的严苛要求.在传统专业领域，人类已经积累了大量的经验通过工程化的最佳实践实现高可靠性.这些最佳实践包括标准作业程序(standard operating procedure, SOP)和检查清单等机制.同样地，在大模型时代，首次提出大模型需借助高阶程序(highorder program, HOP)突破可靠性瓶颈.具体来说，HOP设计了一套新颖的任务融合描述与执行的语言，结合程序语言的精确执行能力与自然语言的知识表达优势，承载了专业领域的关键知识和实践并将其自动化.更重要的是其核验机制，补齐了当前大模型执行不验证的短板，保障了可靠性.在实践层面，HOP已经证明了其作为大模型行业应用急需且必须的控制体系的重大意义：HOP已经在密算金融风控全链路、网络入侵检测、医疗重复计费等多行业场景中初步应用，其时效性有显著提升(5~10倍)，可靠性准确率均可达到99%.

关键词: 大模型, 高阶程序, 核验, 可靠性, 幻觉