Abstract: To address the challenges of high memory consumption, low computational efficiency, and homomorphic noise accumulation in fully homomorphic encryption (FHE) for privacypreserving inference in convolutional neural network (CNN), this paper proposes a collaborative optimization framework. The framework introduces a hierarchical memory scheduling strategy, which employs a dynamic key loading mechanism and an adaptive compression technique for polynomial ring slot numbers (reducing available slots exponentially based on network depth), thereby significantly decreasing memory usage. Additionally, a noise suppression residual module is developed, incorporating a noise propagation dynamics model to design a realtime noise monitoringbased ondemand bootstrapping trigger mechanism, which reduces bootstrapping frequency and enhances inference efficiency. Experimental results on the CIFAR10 dataset demonstrate that this framework enables homomorphic encrypted inference of ResNet20 in approximately 500s with only 20GB of memory, achieving a 3.5× improvement in inference efficiency and a 94% reduction in memory consumption compared to existing CKKSbased solutions (2271s384GB). This framework provides a novel technical paradigm for privacypreserving machine learning in resourceconstrained scenarios.

Key words: privacypreserving machine learning, fully homomorphic encryption, noise suppression, residual network, hierarchical memory scheduling

摘要： 针对全同态加密(fully homomorphic encryption, FHE)在卷积神经网络隐私推理中面临的高内存占用、低计算效率及同态噪声累积等挑战，提出了一种协同优化框架：层次化内存调度策略.通过动态密钥按需加载机制与多项式环插槽数自适应压缩技术(根据网络深度指数衰减可用插槽数量)，实现内存占用量级缩减；提出噪声抑制残差模块，通过构建噪声传播动力学模型设计基于实时噪声监测的按需自举触发机制，实现自举频率降低从而提高推理效率.在CIFAR10数据集上的实验表明，完成ResNet20同态加密推理仅需约500s和20GB内存，该方案较现有CKKS(CheonKimKimSong)方案(2271s384GB)在推理效率上提升约3.5倍，内存消耗降低约94%.为资源受限场景的隐私保护机器学习提供了一条高效可行的技术路径.

关键词: 隐私保护机器学习, 全同态加密, 噪声抑制, 残差网络, 层次化内存调度