Sage Journals: Discover world-class research

Abstract

As one of the fundamental tasks in natural language processing, Multi-Label Text Classification (MLTC) is used to mark one or more relevant labels for a given text from a large set of labels. Existing MLTC methods have increasingly focused on improving classification effectiveness by fusing the correlations of labels. Still, the research suffers from difficulties in comprehensively extracting text features and distinguishing similar labels. This paper proposed a multi-label text classification model based on keyword extraction and attention mechanism. The model proposed using keywords to represent labels, adopting both self-attention and interactive attention mechanisms (between labels and text) to extract text features and create text vectors. Finally, fusing text vectors as the classifier’s input. Experiments were conducted on two public datasets and a self-built dataset of illegal advertisements. The experimental results showed that the keyword-based label representation approach proposed in this paper can better obtain label semantics, avoid noise and improve the performance of the multi-label text classification.

Keywords

Multi-label text classification keyword extraction attention mechanism label indicates natural language processing

Get full access to this article

View all access options for this article.

References

Xiao

and Huang

, Label-specific document representation for multi-label text classification in: EMNLP-IJCNLP, 2019, pp. 466–475.

Hao

and Qiu

, Incorporating bert and graph attention network for multi-label text classification, Computer Systems and Applications 6 (2022), 167–174.

Yang

and Ma

, Multi-label text classification model combining cnn-sam and gat,Computer Engineering and Applications 2022, pp. 1–10.

Song

and LI

, Multi-label classification of legal text with fusion of label relations, Pattern Recognition and Artificial Intelligence 2 (2022), 185–192.

Liu

and Zhong

, Multi-label text classification with improved graphrnn, Small Microcomputer Systems 2022, pp. 1–8.

Kurata

and Xiang

, Improved neural network-based multi-label classification with better initialization leveraging label co-occurrence, Human Language Technologies 2016, pp. 521–526.

Yang

and Sun

, SGM: sequence generation model for multi-label classification, 2018.

Xun

and Jha

, Correlation networks for extreme multilabel text classification, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2020, pp. 1074–1082.

and Song

, Knn-bert: fine-tuning pre-trained models with knn classifier, 2021.

10.

Babbar

and Scholkopf

, Dismec: Distributed sparse machines for extreme multi-label classification, in: Proceedings of the tenth ACM international conference on web search and data mining, 2017, pp. 721–729.

11.

Yen

I.E.

and Huang

, Ppdsparse: A parallel primal-dual sparse method for extreme classification, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 545–553.

12.

Prabhu

and Kag

, Parabel: Partitioned label trees for extreme classification with application to dynamic search advertising, in: Proceedings of the 2018 World Wide Web Conference, 2018, pp. 993–1002.

13.

Khandagale

and Xiao

, Bonsai: diverse and shallow trees for extreme multi-label classification, Machine Learning 11 (2020), 2099–2119.

14.

Bhatia

and Jain

, Sparse local embeddings for extreme multi-label classification, Advances in Neural Information Processing Systems, 2015.

15.

Tagami

, Annexml: Approximate nearest neighbor search for extreme multi-label classification, in: Proceedings of the 23rd ACMSIGKDD international conference on knowledge discovery and data mining, 2017, pp. 455–464.

16.

Liu

and Chang

, Deep learning for extreme multi-label text classification, in: Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, 2017, pp. 115–124.

17.

You

and Zhang

, Attentionxml: Label tree-based attention-aware deep model for high-performance extreme multi-label text classification, Advances in Neural Information Processing Systems, 2019.

18.

Mikolov

and Chen

, Efficient estimation of word representations in vector space, 2013.

19.

Lewis

D.D.

and Yang

, Rcv1: A new benchmark collection for text categorization research,, Journal of Machine Learning Research 4 (2004), 361–397.

20.

Loza Mencía

and Fürnkranz

, Efficient pairwise multilabel classification for large-scale problems in the legal domain, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2008, pp. 50–65.

21.

Jain

and Prabhu

, Extreme multi-label loss functions for recommendation, tagging, ranking and other missing label applications, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 935–944.

Incorporating keyword extraction and attention for multi-label text classification

Abstract

Keywords

Get full access to this article

References