基于改进Single-Pass的新闻话题检测与追踪技术研究

信息安全研究 ›› 2020, Vol. 6 ›› Issue (5): 396-403.

基于改进Single-Pass的新闻话题检测与追踪技术研究

张帆¹,潘亚雄²,胡勇³

1. 四川大学网络空间安全学院
2. 中物院成都科学技术发展中心
3. 四川大学

收稿日期:2020-04-29 出版日期:2020-05-15 发布日期:2020-04-29
通讯作者: 张帆
作者简介:张帆硕士研究生，主要研究方向为网络数据分析与数据安全 1105988653@qq.com 潘亚雄硕士研究生，高级工程师，主要研究方向为网络信息安全 panyaxiong@163.com 胡勇博士研究生，研究员，主要研究方向为网络信息安全 huyong@scu.edu.cn

Research on News Topic Detection and Tracking Technology Based on Improved Single-Pass

Received:2020-04-29 Online:2020-05-15 Published:2020-04-29

摘要/Abstract

摘要： 为解决如何从海量新闻报道中检测并追踪到目标话题，选择了自增式聚类Single-Pass算法进行研究。在原有的基础上对其进行改进得到改进后的Single-Pass聚类算法，期望能得到更好的解决方法。对于原有算法进行的改进主要有在新闻文本的特征词选取中加入权重系数表达特征词位置信息，同时辅以时间特征进行新闻文本相似度计算，并且在Single-Pass聚类算法步骤中添加子话题阈值判断过程。实验验证改进后的Single-Pass聚类算法不止可得到不同粒度的话题聚类效果，同时也提升了聚类效率。实验结果证明，在相同条件下，改进Single-Pass聚类算法的漏检率和误检率上有明显的改善。

关键词: 新闻话题, Single-Pass聚类算法, 时间特征, 相似度, 子话题

Abstract: In order to solve the problem of how to detect and track the target topic from massive news reports, an auto-increasing clustering Single-Pass algorithm was selected to research. Based on the improvement of the original Single-Pass clustering algorithm, it is expected to get a better solution. The improvement of the original algorithm mainly includes adding weight coefficients to select feature words in news text to express feature word position information, supplemented by temporal features to calculate similarity of news text, and adding sub-segments in the Single-Pass clustering algorithm Topic threshold judgment process. The experiments verify that the improved Single-Pass clustering algorithm can not only obtain the clustering effect of topics with different granularities, but also improve the clustering efficiency. The experimental results show that under the same conditions, the missed detection rate and false detection rate of the improved Single-Pass clustering algorithm are significantly improved.

Key words: News topics, Single-Pass clustering algorithm, temporal characteristics, similarity, sub-topic

张帆潘亚雄胡勇. 基于改进Single-Pass的新闻话题检测与追踪技术研究[J]. 信息安全研究, 2020, 6(5): 396-403.

参考文献

[1] 孙红光, 高星, 孙铁利, et al. 基于改进Single-Pass算法的网络新闻话题发现[J]. 吉林大学学报(理学版), 2018, v.56;No.229(01):120-124. [2] Wayne C L. Multilingual Topic Detection and Tracking: Successful Research Enable by Corpora and Evaluation[C]//Proceedings of the 2nd International Conference on Language Resources and Conference(LREC). Paris: European Language Resources Association, 2000: 1487-1493. [3] Yang Y , Carbonell J , Brown R , et al. Multi-strategy learning for topic detection and tracking: a joint report of CMU approaches to multilingual TDT[M]. Kluwer Academic Publishers, 2002. [4] 刁洪祥. 话题检测与跟踪关键技术研究[J]. 信息与电脑, 2016(7): 31-32. [5] Kanagasabi Rajaraman, Ah-Hwee Tan, et al. Topic Detection, Tracking and Trend Analysis Using Self-organizing Neural Networks[C]// Proceedings of the 5th Pacific-Asia Conference on Knowledge Discovery and Data Mining. Hong Kong: David CheungGraham J. WilliamsQing Li, 2001:102-107. [6] Wang M , Jayaraman P P , Solaiman E , et al. A multi-layered performance analysis for cloud-based topic detection and tracking in Big Data applications[J]. Future Generation Computer Systems, 2018, 87. [7] Wang Y , Zhang Y , Zhou B , et al. Semi-Supervised Collective Matrix Factorization for Topic Detection and Document Clustering[C]// 2017 IEEE Second International Conference on Data Science in Cyberspace (DSC). IEEE, 2017. [8] 张仰森, 段宇翔, 黄改娟, et al. 社交媒体话题检测与追踪技术研究综述[J]. 中文信息学报, 2019, 33(7): 1-10. [9] Wartena C , Brussee R . Topic Detection by Clustering Keywords[C]// Database and Expert Systems Application, 2008. DEXA '08. 19th International Conference on. IEEE, 2008. [10] Liu G , Cheng M . A Topic Detection Method for Chinese Microblog[J]. 2012. [11] 徐建民, 孙晓磊, 吴树芳. 结合时间信息的事件追踪的动态模型[J]. 计算机应用, 2013, 33(10):2807-2810. [12] 韩忠明, 张梦玫, 李梦琪, et al. 面向复杂主题建模的流公式层次狄里克雷过程[J]. 计算机学报, 2019(7): 1039-1552. [13] 陈梦伟, 吕钊, 崔修涛. 基于联合非负矩阵分解的话题变迁检测方法[J]. 计算机工程, 2018, 44(1). [14] 许强. 基于Spark的话题检测与跟踪技术研究[D].四川:电子科技大学,2018. [15] 洪宇, 张宇, 刘挺, et al. 话题检测与跟踪的评测及研究综述[J]. 中文信息学报, 2007, 21(6):71-87. [16] LARRANAGA P, LOZANO JA. Estimation of distribution algorithms[M].Dordrecht: Kluwer Academic Publishers, 2001. [17] YANG CC, SHI X D, WEI C P. Tracing the event evolution of terror attacks from on-line news[C]// Intelligence and Security Infomatics, LNCS 3975. Berlin: Springer-Verlag, 2006: 343-354.