关键词: 公共安全事件, 中文实体识别, 领域预训练, 双向长短时记忆网络, 条件随机场, RoBERTa预训练语言模型

Abstract: To improve the efficiency of chinese named entity recognition in public safety events, we study the "Chinese Emergency Corpus", and propose a novel domain adaptive pre-training based named entity recognition model by optimizing the pre-training subtasks and transfer learning of domain datasets. First, the dictionary of pre-training model RoBERTa is updated by adding the terms of public safety events, and the mask subtask of pre-training with single character in chinese RoBERTa model is replaced with the chinese whole word masking, which can learn the more grammatical and semantic information of public safety events. And then, 100k online news unlabeled corpus is pre-trained to enhance the ability of identify downstream named entities, and the chinese pre-training model RoBERTa+ is generated for public security. The bidirectional long short-term memory network BiLSTM is employed to acquire the contextual abstraction feature, and the entities is final recognized by the sequence decoding with the conditional random field. Experimental result shows that the proposed model reaches an accuracy rate of 87%, the recall rate and F1-value of 81%, which indicates that the domain adaptive pre-training has considerable potential for natural language processing tasks.

Key words: public security, chinese named entity recognition, domain pre-training, BiLSTM, CRF, pre-trained language model RoBERTa