Understanding semantic themes of short texts is challenging due to limited word co-occurrence information. Utilising pre-trained word embeddings or incorporating contextual information from external sources is likely to increase noise and mislead the thematic representation of the short texts, which declines the classification performance. For higher accuracy in classifying short texts, we propose a knowledge graph-enhanced topic model called Graph Convolutional Embedded Topic Model (GCETM), which simultaneously learns the graph network and topic modelling. GCETM employs the Graph Convolutional Network (GCN) to infuse prior human knowledge of the current short texts into the topic embedding space. For model fitting, we propose a data-driven regularisation for amortised variational inference. Besides GCETM topic inference, we utilise corpus statistics for semantically enriched vectorial representation of short text for their classification. Experimental results using a linear Support Vector Machine (SVM) classifier outperform several state-of-the-art baselines by achieving 97.15% accuracy on the AgNews data set, 97.75% accuracy on the SearchSnippets data set, 98.73% accuracy on the Movie Review (MR) data set, 98.7% accuracy on the TMNews data set, 96.5% accuracy on the Twitter data set and 98.44% accuracy on the R8 data set.
DeerwesterSDumaisSTFurnasGW, et al. Indexing by latent semantic analysis. J Am Soc Inform Sci1990; 41(6): 391–407.
6.
HofmannT. Probabilistic latent semantic indexing. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, 1999, pp. 50–57.
7.
DingRNallapatiRXiangB.Coherence-aware neural topic modeling. In: Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 830–836.
8.
DiengABRuizFJBleiDM.Topic modeling in embedding spaces. Trans Assoc Comput Linguist2020; 8: 439–453.
9.
HongLDavisonBD. Empirical study of topic modeling in twitter. In: Proceedings of the first workshop on social media analytics, 2010, pp. 80–88.
10.
WangYLiuJQuJ, et al. Hashtag graph based topic model for tweet mining. In: IEEE international conference on data mining, 2014, pp. 1025–1030.
11.
LinHZuoYLiuG, et al. A Pseudo-document-based Topical N-grams model for short texts. World Wide Web2020; 23(6): 3001–3023.
12.
ZuoYWuJZhangH, et al. Topic modeling of short texts: a pseudo-document view. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 2105–2114.
13.
WengJLimEPJiangJ, et al. Twitterrank: finding topic-sensitive influential twitterers. In: Proceedings of the third ACM international conference on web search and data mining, 2010, pp. 261–270.
14.
MehrotraRSannerSBuntineW, et al. Improving LDA topic models for microblogs via tweet pooling and automatic labeling. In: Proceedings of the 36th international ACM SIGIR conference on research and development in information retrieval, 2013, pp. 889–892.
15.
SenguptaAPakaWSRoyS, et al. An embedding-based joint sentiment-topic model for short texts. In: International AAAI conference on web and social media (ICWSM), 2021, pp. 633–643.
16.
MurakamiRChakrabortyB. Neural topic models for short text using pretrained word embeddings and its application to real data. In: IEEE 4th international conference on knowledge innovation and invention (ICKII), 2021, pp. 146–150.
17.
PaiSCostabelloL.Learning embeddings from knowledge graphs with numeric edge attributes. arXiv Preprint 2105.08683, 2021, https://arxiv.org/abs/2105.08683
18.
KristiadiAKhanMALukovnikovD, et al. Incorporating literals into knowledge graph embeddings. In: International semantic web conference, 2019, pp. 347–363.
19.
GhoorchianKSahlgrenM.GDTM: graph-based dynamic topic models. In: Progress in artificial intelligence, 2020, pp. 195–207.
20.
AlMousaMBenlamriRKhouryR.A novel word sense disambiguation approach using WordNet knowledge graph. Comput Speech Lang2022; 74: 101337.
21.
OrtizVJSegarraJVillazónTB, et al. REDI: towards knowledge graph-powered scholarly information management and research networking. J Inform Sci2022; 48(2): 167–181.
22.
KipfTNWellingM.Semi-supervised classification with graph convolutional networks. In: 5th international conference on learning representations (ICLR-17), 2017.
23.
VashishthSBhandariMYadavP, et al. Incorporating syntactic and semantic information in word embeddings using graph convolutional networks. In: Proceedings of the 57th annual meeting of the association for computational linguistics, 2019, pp. 3308–3318.
24.
BastingsJTitovIAzizW, et al. Graph convolutional encoders for syntax-aware neural machine translation. In: Proceedings of the 2017 conference on empirical methods in natural language processing, 2017, pp. 1957–1967.
25.
AnnervazKChowdhurySBRDukkipatiA. Learning beyond datasets: knowledge graph augmented neural networks for natural language processing. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics, 2018, pp. 313–322.
26.
YaoLMaoCLuoY. Graph convolutional networks for text classification. In: Proceedings of the AAAI conference on artificial intelligence, 2019, pp. 7370–7377.
27.
MillerGA.WordNet: a lexical database for English. Commun ACM1995; 38(11): 39ACM–41.
28.
ZhouMXuWZhangW, et al. Leverage knowledge graph and GCN for fine-grained-level clickbait detection. World Wide Web2022; 25(3): 1243–1258.
29.
KimY. Convolutional neural network for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1746–1751.
30.
LiuPQiuXHuangX.Recurrent neural network for text classification with multi-task learning. arXiv Preprint 1605.05101, 2016, https://arxiv.org/abs/1605.05101
31.
ZhangYYuXCuiZ, et al. Every document owns its structure: inductive text classification via graph neural networks. arXiv Preprint 2004.13826, 2020, https://arxiv.org/abs/2004.13826
32.
LiKYeW.Semi-supervised node classification via graph learning convolutional neural network. Appl Intel2022, pp. 1–13.
33.
SongRGiunchigliaFZhaoK, et al. Graph topology enhancement for text classification. Appl Intel2022, pp 1–14.
34.
ZhouPWuZWenG, et al. Multi-scale graph classification with shared graph neural network. World Wide Web2022, pp. 1–18.
35.
YaoLZhangYWeiB, et al. Incorporating knowledge graph embeddings into topic modeling. In: Thirty-first AAAI conference on artificial intelligence, 2017.
36.
SongDGaoJPangJ, et al. Knowledge base enhanced topic modeling. In: IEEE international conference on knowledge graph (ICKG), 2020, pp. 380–387. New York: IEEE.
37.
VasanthaKVSendhikumarS.Developing a conceptual framework for short text categorization using hybrid CNN-LSTM based Caledonian crow optimization. Expert Syst Appl2023; 212: 1185.
38.
LiangWFengRLiuX, et al. GLTM: a global and local word embedding-based topic model for short texts. IEEE Access2018; 6: 43612–43621.
39.
JinJZhaoHJiP.Topic attention encoder: a self-supervised approach for short text clustering. J Inform Sci2020.
40.
XuTPengQ.Extended information inference model for unsupervised categorization of web short texts. J Inform Sci2012; 38(6): 512–531.
41.
CuiXKojakuSMasudaN, et al. Solving feature sparseness in text classification using core-periphery decomposition. In: Proceedings of the seventh joint conference on lexical and computational semantics, 2018, pp. 255–264.
WuXLuuATDongX. Mitigating data sparsity for short text topic modeling by topic-semantic contrastive learning. arxiv Preprint 2211.12878, 2022, https://arxiv.org/abs/2211.12878
44.
CaoMYuanJYuH, et al. Self-supervised short text classification with heterogeneous graph neural networks. Xpert Systems2023: e13249.
45.
LiuYLiPHuX.Combining context-relevant features with multi-stage attention network for short text classification. Comput Speech Lang2022; 71: 101268.
46.
WangXZhangYWangX, et al. A knowledge graph enhanced topic modeling approach for herb recommendation. In: International conference on database systems for advanced applications, 2019, pp. 709–724.
47.
PetrželkováNŠkrljBLavračN.Knowledge graph aware text classification. Inform Soc2020.
48.
LaiYZhangLHanD, et al. Fine-grained emotion classification of Chinese microblogs based on graph convolution networks. World Wide Web2020; 23(5): 2771–2787.
49.
VanLNBachTXThanK.A graph convolutional topic model for short and noisy text streams. Neurocomputing2022; 468: 345–335.
50.
ZhouMLiZXieP.Self-supervised regularization for text classification. Trans Assoc Comput Ling2021; 9: 641–656.
51.
SantosIMHamacherSOliveiraF.A data-driven optimization model for the workover rig scheduling problem: case study in an oil company. Trans Assoc Comput Ling2021; 170: 108088.
52.
ConmyAMukherjeeSSchönliebCB. StyleGAN-induced data-driven regularization for inverse problems. In: ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP), 2022, pp. 3788–3792.
53.
LiWDasarathyGBerishaV.Regularization via structural label smoothing. In: Proceedings of the 23rd international conference on artificial intelligence and statistics (AISTATS), 2020, pp. 1453–1463.
54.
WeiJZouK.Eda: easy data augmentation techniques for boosting performance on text classification tasks. arXiv Preprint, 1901.11196, 2019, https://arxiv.org/abs/1901.11196
55.
HeJHuZBerg-KirkpatrickT, et al. Efficient correlated topic modeling with topic embedding. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, 2017, pp. 225–233.
56.
MarinoJYueYMandtS.Iterative amortized inference. In: International conference on machine learning, 2018, pp. 3403–3412.
57.
WuMChoiKGoodmanN, et al. Meta-amortized variational inference and learning. Proc AAAI Conf Artif Intel2020; 34(4): 6404–6412.
58.
OrhanUTuluCN.A novel embedding approach to learn word vectors by weighting semantic relations: SemSpace. Expert Syst Appl2021; 180: 115146.
59.
UddinFChenYZhangZ, et al. Corpus statistics empowered document classification. Electronics2022; 11(14): 2168.
60.
LopezRBoyeauPYosefN, et al. Decision-making with auto-encoding variational Bayes. Adv Neural Inform Proces Syst2020; 33: 5081–5092.
61.
HsiehSKTsengYHChiangCY. Modeling the idiomaticity of Chinese Quadra-syllabic idiomatic expressions. In: Proceedings of the 33rd Pacific Asia conference on language, information and computation, 2019, pp. 68–75.
62.
AbdeljaberHA.Automatic Arabic short answers scoring using longest common subsequence and Arabic WordNet. IEEE Access2021; 9: 76433–76445.
63.
JoshiMLMittalNJoshiN.Improving the performance of semantic graph-based keyword extraction and text summarization using fuzzy relations in Hindi Wordnet. J Intel Fuzzy Syst2022; 1–18.
64.
QuispeLVCTohalinoJAVAmancioDR. Using virtual edges to improve the discriminability of co-occurrence text networks. Phys A: Stat Mech Appl2021; 562: 125344.
65.
KampffmeyerMChenYLiangX, et al. Rethinking knowledge graph propagation for zero-shot learning. Phys A: Stat Mech Appl2019; 11487–11496.
66.
JoulinAGraveEBojanowskiP, et al. Bag of tricks for efficient text classification. arXiv Preprint 1607.01759, 2016, https://arxiv.org/abs/1607.01759
67.
ShenDWangGWangW, et al. Baseline needs more love: on simple word-embedding-based models and associated pooling mechanisms. arXiv Preprint 1805.09843, 2018, https://arxiv.org/abs/1805.09843
68.
ZhouPShiWTianJ, et al. Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics, 2016, pp. 207–212.
YanXGuoJLanY, et al. A biterm topic model for short texts. In: Proceedings of the 22nd international conference on World Wide Web, 2013, pp. 1445–1456.
71.
NguyenDQBillingsleyRDuL, et al. Improving topic models with latent feature word representations. Trans Assoc Comput Linguist2015; 3: 299–313.
72.
LiCWangHZhangZ, et al. Topic modeling for short texts with auxiliary word embeddings. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval, 2016, pp. 165–174.
73.
ZuoYZhaoJXuK.Word network topic model: a simple but general solution for short and imbalanced texts. Knowl Inform Syst2016; 48(2): 379–398.
