Chinese Short Text Keyphrase Extraction Model Based on Attention

Abstract

Abstract: Keyphrase extraction technology is a research hotspot in the field of natural language processing.In the current keyphrase extraction algorithm,the deep learning method seldom takes into account the characteristics of Chinese,the information of Chinese character granularity is not fully utilized,and the extraction effect of Chinese short text keyworks still has a large improvement space.In order to improve the effect of the keyphrase extraction for short text,a model for automatic keyphrase extraction abstracts was proposed,namely BAST model,which combines the bidirectional long short-term memory and attention mechanism based on sequence tagging model.Firstly, word vectors in the word granularity and character vectors in the character granularity are used to represent input text information.Secondly,the BAST model is trained,text features are extracted by using BiLSTM and attention mechanism,and the label of each word is classified.Finally,the character vector model is used to correct the extraction results of the word vector model.The experimental results show that the F1-measure of the BAST model reaches 66.93% on 8159 abstract data,which is 2.08% higher than that of the BiLSTM-CRF(Bidirectional Long Shoft-Term Memory and Conditional Random Field) algorithm,and is further improved than other traditional keyphrase extraction algorithms.The innovation of the model lies in the combination of the extraction results of the word vector and the character vector model.The model makes full use of the characteristics of the Chinese text information and can effectively extract keyphrases from the short text,and extraction effect is further improved.

Key words: Attention mechanism, Character embedding, Keyphrase extraction, LSTM, Word embedding

CLC Number:

TP391

YANG Dan-hao,WU Yue-xin,FAN Chun-xiao. Chinese Short Text Keyphrase Extraction Model Based on Attention[J].Computer Science, 2020, 47(1): 193-198.

References

[1]GOLLAPALLI S,CARAGRA C.Extracting Keyphrases from Research Papers Using Citation Networks [C]∥ Proceedings of the National Conference on Artificial Intelligence.Quebec:AAAI Press,2014:1629-1635.
[2]FLORESCU C,CARAGEA C.Positionrank:An unsupervised approach to keyphrase extraction from scholarly documents[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:1105-1115.
[3]HASAN K,NG V.Automatic keyphrase extraction:A survey of the state of the art[C]∥Proceedings of the 27th International Conference on Computational Linguistics.Baltimore,Maryland,2014:1262-1273.
[4]LI G,WANG H.Improved automatic keyword extraction based on textrank using domain knowledge[C]∥ Proceedings of the 2014 Natural Language Processing and Chinese Computing.Berlin:Springer-Verlag,2014:403-413.
[5]BOUGOUIN A,BOUDIN F,DAILLE B.TopicRank:Graph- Based Topic Ranking for Keyphrase Extraction[C]∥Procee-dings of theInternational Joint Conference on Natural Language Processing.Nagoya,Japan,2013:543-551.
[6]TENEVA N,CHENG W.Salience rank:efficient keyphrase extraction with topic modeling[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:530-535.
[7]FLORESCU C,CARAGEA C.A Position-Biased PageRank Algorithm for Keyphrase Extraction[C]∥Proceedings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:4923-4924.
[8]ZHANG C,WANG H,LIU Y,et al.Automatic keyword extraction from documents using conditional random fields[J].Journal of Computational Information Systems,2008,4(3):1169-1180.
[9]HADDOUD M,MOKHRARI A,LECROQ T,et al.Accurate Keyphrase Extraction from Scientific Papers by Mining Linguistic Information[C]∥Proceedings of The Workshop on Mining Scientific Papers:Computational Linguistics and Bibliometrics.Istanbul,Turkey:CEUR-WS,2015:12-17.
[10]ONAN A,KORUKOGLU S,BULUT H.Ensemble of keyword extraction methods and classifiers in text classification[J].Expert Systems with Applications,2016,57(3):232-247.
[11]GOLLAPALLI S,LI X,YANG P.Incorporating expert know- ledge into keyphrase extraction[C]∥ Processings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:3180-3187.
[12]ZHANG Q,WANG Y,GONG Y,et al.Keyphrase Extraction Using Deep Recurrent Neural Networks on Twitter[C]∥ Proceedings of Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2016:836-845.
[13]REKIA K,ZHANG Y,ZHANG W,et al.CCG Supertagging via Bidirectional LSTM-CRF Neural Architecture[J].Neurocomputing,2017,283(12):31-37.
[14]MOURAD G.Character-level neural network for biomedical named entity recognition[J].Journal of Biomedical Informatics,2017,70(5):85-91.
[15]ANDREJ Z,YORAM B,PASHA M,et al.Neural Named Entity Recognition Using a Self-Attention Mechanism[C]∥Procee-dings of International Conference on TOOLS with Artificial Intelligence.Boston:IEEE Computer Society,2017:652-656.
[16]SI Y,XIAO Y,XU J,et al.Recurrent neural network language model with vector-space word representations[C]∥Proceedings of the International Conference on Learning Representations.Beijing:International Institute of Acoustics and Vibrations,2014:3024-3031.
[17]SUNDERMEYER M,SCHLUTER R,NEY H.LSTM Neural Networks for Language Modeling[C]∥Proceedings of the 13th Annual Conference of the International Speech Communication Association Interspeech.Portland,OR,2012:194-197.
[18]GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J].Nrural Networks,2005,18(5):602-610.
[19]FENG S,LIU S,YANG N,et al.Improving attention modeling with implicit distortion and fertility for machine translation[C]∥Proceedings of 26th International Conference on Computational Linguistics.Osaka,Japan,2016:3082-3092.
[20]TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[C]∥Proceedings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:4923-4924.

Related Articles 15

[1]	ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2]	DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3]	ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4]	XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5]	RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6]	JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7]	ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[8]	SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9]	YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[10]	WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[11]	HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[12]	JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
[13]	XIONG Luo-geng, ZHENG Shang, ZOU Hai-tao, YU Hua-long, GAO Shang. Software Self-admitted Technical Debt Identification with Bidirectional Gate Recurrent Unit and Attention Mechanism [J]. Computer Science, 2022, 49(7): 212-219.
[14]	PENG Shuang, WU Jiang-jiang, CHEN Hao, DU Chun, LI Jun. Satellite Onboard Observation Task Planning Based on Attention Neural Network [J]. Computer Science, 2022, 49(7): 242-247.
[15]	ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!