Sage Journals: Discover world-class research

Abstract

Knowledge graph embedding is aimed at capturing the semantic information of entities by modeling the structural information between entities. For long-tail entities which lack sufficient structural information, general knowledge graph embedding models often show relatively low performance in link prediction. In order to solve such problems, this paper proposes a general knowledge graph embedding framework to learn the structural information as well as the attribute information of the entities simultaneously. Under this framework, a H-AKRL (Hypergraph Neural Networks based Attribute-embodied Knowledge Representation Learning) model is put forward, where the hypergraph neural network is used to model the correlation between entities and attributes at a higher level. The complementary relationship between attribute information and structural information is taken full advantage of, enabling H-AKRL to finally achieve the goal of improving link prediction performance. Experiments on multiple real-world data sets show that the H-AKRL model has significantly improved the link prediction performance, especially in the embeddings of long tail entities.

Keywords

Attribute network hypergraph neural network knowledge graph embedding long-tail entity

Get full access to this article

View all access options for this article.

References

Auer

et al., Dbpedia: A nucleus for a web of open data, The semantic web, 2007, 722–735.

Bai

Zhang

and Torr

P.H.

, Hypergraph convolution and hypergraph attention, Pattern Recognition 110 (2021), 107637.

Balaževic

Allen

and Hospedales

T.M.

, TuckER: Tensor Factorization for Knowledge Graph Completion, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019.

Bollacker

et al., Freebase: a collaboratively created graph database for structuring human knowledge, in: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, 2008.

Bordes

et al., Translating embeddings for modeling multi-relational data, Neural Information Processing Systems, 2013.

Meilicke

et al., Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion, in: International Semantic Web Conference, 2018, pp. 3–20.

Carlson

et al., Toward an architecture for never-ending language learning, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2010.

Chen

et al., Co-training embeddings of knowledge graphs and entity descriptions for cross-lingual entity alignment, in: Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018.

Cohen

W.W.

et al., Scalable Neural Methods for Reasoning with a Symbolic Knowledge Base, in: International Conference on Learning Representations, 2019.

10.

Das

et al., Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning, in: International Conference on Learning Representations, 2018.

11.

Dettmers

et al., Convolutional 2d knowledge graph embeddings, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2018.

12.

Dong

et al., Knowledge vault: A web-scale approach to probabilistic knowledge fusion, in: Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014.

13.

Fan

et al., Transition-based knowledge graph embedding with relational mapping properties, in: Proceedings of the 28th Pacific Asia Conference On Language, Information and Computing, 2014.

14.

Feng

et al., Knowledge graph embedding by flexible translation, in: Proceedings of the Fifteenth International Conference on Principles of Knowledge Representation and Reasoning, 2016.

15.

Feng

et al., Hypergraph neural networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019.

16.

Garcia-Duran

and Niepert

, Kblrn: End-to-end learning of knowledge base representations with latent, relational, and numerical features, arXiv preprint arXiv:1709.04676, 2017.

17.

Guan

Song

and Liao

, Knowledge graph embedding with concepts, Knowledge-Based Systems 164 (2019), 38–44.

18.

Guo

Sun

and Hu

, Learning to exploit long-term relational dependencies in knowledge graphs, in: International Conference on Machine Learning, 2019.

19.

Hao

et al., Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts, in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.

20.

Hoffart

et al., YAGO2: A spatially and temporally enhanced knowledge base from Wikipedia, Artificial Intelligence 194 (2013), 28–61.

21.

Hong

et al., Deep attributed network embedding by preserving structure and attribute information, in: IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2019.

22.

Hou

and Tang

, RoSANE: Robust and scalable attributed network embedding for sparse networks, Neurocomputing 409 (2020), 231–243.

23.

et al., Knowledge graph completion with adaptive sparse transfer matrix, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2016.

24.

et al., Knowledge graph embedding via dynamic mapping matrix, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015.

25.

Jiang

et al., Dynamic Hypergraph Neural Networks, IJCAI, 2019, 2635–2641.

26.

Kingma

D.P.

and Ba

, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2014.

27.

Yin

and Chen

, SEAL: Semisupervised Adversarial Active Learning on Attributed Graphs, in: IEEE Transactions on Neural Networks and Learning Systems, 2020.

28.

Lin

et al., Learning entity and relation embeddings for knowledge graph completion, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2015.

29.

Lin

Liu

and Sun

, Knowledge representation learning with entities, attributes and relations, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 2016.

30.

Liu

and Yang

, Analogical inference for multi-relational embeddings, in: Proceedings of the 34th International Conference on Machine Learning, 2017.

31.

Liu

et al., Probabilistic reasoning via deep learning: Neural association models, arXiv preprint arXiv:1603.07704, 2016.

32.

et al., Differentiating Concepts and Instances for Knowledge Graph Embedding, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018.

33.

Mahdisoltani

Biega

and Suchanek

, Yago3: A knowledge base from multilingual wikipedias, in: 7th Biennial Conference on Innovative Data Systems Research, 2014.

34.

Miller

G.A.

, WordNet: a lexical database for English, Communications of the ACM, 1995, 39–41.

35.

Mitchell

et al., Never-ending learning, Communications of the ACM, 2018, 103–115.

36.

Nathani

et al., Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.

37.

Neelakantan

Roth

and McCallum

, Compositional Vector Space Models for Knowledge Base Completion, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 2015.

38.

Nguyen

D.Q.

et al., A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network, in: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2018.

39.

Nguyen

D.Q.

Nguyen

T.D.

and Phung

, A Relational Memory-based Embedding Model for Triple Classification and Search Personalization, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.

40.

Nickel

Rosasco

and Poggio

, Holographic embeddings of knowledge graphs, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2016.

41.

Nickel

Tresp

and Kriegel

H.P.

, A three-way model for collective learning on multi-relational data, in: Proceedings of the 28th International Conference on International Conference on Machine Learning, 2011.

42.

Pezeshkpour

Chen

and Singh

, Embedding Multimodal Relational Data for Knowledge Base Completion, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018.

43.

Rastogi

et al., Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020.

44.

Rossi

et al., Knowledge graph embedding for link prediction: A comparative analysis, ACM Transactions on Knowledge Discovery from Data, 2021, 1–49.

45.

Schlichtkrull

et al., Modeling relational data with graph convolutional networks, in: European Semantic Web Conference, 2018.

46.

Shang

et al., End-to-end structure-aware convolutional networks for knowledge base completion, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019.

47.

Shi

and Weninger

, ProjE: Embedding projection for knowledge graph completion, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2017.

48.

Socher

et al., Reasoning with neural tensor networks for knowledge base completion, in: Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013.

49.

Suchanek

F.M.

Kasneci

and Weikum

, Yago: a core of semantic knowledge, in: Proceedings of the 16th International Conference on World Wide Web, 2007.

50.

Sun

et al., RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space, in: International Conference on Learning Representations, 2018.

51.

Tay

et al., Multi-task neural network for non-discrete attribute prediction in knowledge graphs, in: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017.

52.

Toutanova

and Chen

, Observed versus latent features for knowledge base and text inference, in: Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality, 2015.

53.

Trisedya

B.D.

and Zhang

, Entity alignment between knowledge graphs using attribute embeddings, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2019.

54.

Trouillon

et al., Complex embeddings for simple link prediction, in: Proceedings of the 33rd International Conference on International Conference on Machine Learning, 2016.

55.

Vashishth

et al., Composition-based Multi-Relational Graph Convolutional Networks, in: International Conference on Learning Representations, 2019.

56.

Vashishth

et al., RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information, in: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018.

57.

et al., A capsule network-based embedding model for knowledge graph completion and search personalization, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019.

58.

Wang

et al., Knowledge graph embedding via graph attenuated attention networks, IEEE Access 8 (2019), 5212–5224.

59.

Wang

et al., Knowledge graph embedding by translating on hyperplanes, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2014.

60.

et al., Dual-view hypergraph neural networks for attributed graph learning, Knowledge-Based Systems, 2021, 107185.

61.

and Wang

, Knowledge graph embedding with numeric attributes of entities, in: Proceedings of The Third Workshop on Representation Learning for NLP, 2018.

62.

Xiao

Huang

and Zhu

, From one point to a manifold: knowledge graph embedding for precise link prediction, in: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 2016.

63.

Xie

et al., Image-embodied knowledge representation learning, in: Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017.

64.

Xie

et al., Representation learning of knowledge graphs with entity descriptions, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2016.

65.

Yang

et al., Embedding entities and relations for learning and inference in knowledge bases, in: International Conference on Learning Representations, 2015.

66.

Zhang

et al., Multi-view knowledge graph embedding for entity alignment, in: Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 5429–5435.

67.

Zhang

et al., Distilling knowledge from well-informed soft labels for neural relation extraction, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2020.

68.

Zhang

et al., Representation learning of knowledge graphs with entity attributes, IEEE Access 8 (2020), 7435–7441.

69.

Zhou

et al., Improving conversational recommender systems via knowledge graph based semantic fusion, in: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020.