关键词: 文本情感分析, BERT, 双向长短时记忆网络, 注意力机制, 词向量

Abstract: Most of the existing models adopt Word2Vec, GloVe(global vectors) and other methods to obtain the word vector representation of the text, ignoring the context relationship of the word. Aiming at this problem, a neural network model based on Bidirectional Encoder Representations from Transformers (BERT) pretraining language model with bidirectional long shortterm memory network (BLSTM) and attention mechanism was proposed for text sentiment analysis. Firstly, the word vector containing contextual semantic information is obtained through BERT pretraining language model. Secondly, BLSTM is used to extract contextual features for deep learning. Finally, attention mechanism is introduced to assign corresponding weights to the extracted deep information of the text to highlight the key information for text sentiment classifications. The accuracy rate can be 88.91% on the SST(stanford sentiment treebank) dataset. The experimental results show that the model performs better than other methods.

Key words: text sentiment analysis, BERT, BLSTM, attention mechanism, word vector