Abstract

Info

Author(s):: REN Jianhua; ZHAO Ruohan; School of Electronic and Information Engineering, Liaoning Technical University, Huludao 125105, China

Keywords:: Keywords: diabetes named entity recognition feature fusion attention mechanism

Abstract:: Abstract: To overcome the challenges of entity diversity and data scarcity in diabetes named entity recognition, feature fusion based named entity recognition is proposed. With BERT+BILSTM+CRF as the benchmark model, improvements are made in 3 aspects. (1) The pre-trained model RoBERTa-wwm-ext is introduced as the model embedding layer to provide character-level embedding, and the whole word mask is used in the training stage to obtain semantic representation containing prior knowledge. (2) bidirectional long short-term memory network and iterated dilated convolutional neural network are used to extract features in parallel to obtain features of different granularities. At the same time, the dynamic feature fusion is combined with the attention mechanism to better understand the key information of the data， thus obtaining richer contextual features. (3) The conditional random field is decoded to obtain the final prediction results. The proposed model achieves an F1 value of 79.58% which is 5.38% higher than High-Order MKGraph on DiaKG, a Chinese diabetes data set containing 18 entity categories, fully demonstrating that the feature fusion based method can effectively identify diabetic entities.