Knowledge bases (KBs) provide a large amount of structured information for entities and relations, which are successfully leveraged in many natural language processing tasks. However, distantly supervised relation extraction only utilizes KBs to automatically generate datasets, while ignoring the background information in KBs during the relation extraction process. We herein propose a knowledge-embodied attention that leverages knowledge information in KBs to reduce the impact of noisy data for distantly supervised relation extraction. Specifically, we pre-train distributed representations of KBs with the knowledge representation learning (KRL) model, and subsequently incorporate them into relation extraction to learn sentence-level attention weights. The experimental results demonstrate that our approach outperforms all baselines, thus indicating that we can focus our attention on valid data by leveraging background information in KBs.
ZhangM.ZhangJ.SuJ. and ZhouG., A composite kernel to extract relations between entities with both flat and structured features, in: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, Association for Computational Linguistics, 2006, pp. 825–832.
2.
BachN. and BadaskarS., A review of relation extraction, Literature review for Language and Statistics II 2.
3.
MintzM.BillsS.SnowR. and JurafskyD., Distant supervision for relation extraction without labeled data, in: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2, Association for Computational Linguistics, 2009, pp. 1003–1011.
4.
ZengD.LiuK.LaiS.ZhouG. and ZhaoJ., Relation classification via convolutional deep neural network, in: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, 2014, pp. 2335–2344.
5.
ZengD.LiuK.ChenY. and ZhaoJ., Distant supervision for relation extraction via piecewise convolutional neural networks, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 1753–1762.
6.
LinY.ShenS.LiuZ.LuanH. and SunM., Neural relation extraction with selective attention over instances, in: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1, 2016, pp. 2124–2133.
7.
VuN.T.AdelH.GuptaP. and SchützeH., Combining Recurrent and Convolutional Neural Networks for Relation Classification, HLT-NAACL 2016, pp. 534–539.
8.
ZhouP.ShiW.TianJ.QiZ.LiB.HaoH. and XuB., Attention-based bidirectional long short-term memory networks for relation classification, in: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Vol. 2, 2016, pp. 207–212.
9.
YangB. and MitchellT., Leveraging knowledge bases in lstms for improving machine reading, in: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1, 2017, pp. 1436–1446.
10.
WangH.ZhangF.XieX. and GuoM., Dkn: Deep knowledge-aware network for news recommendation, arXiv preprint arXiv:180108284.
11.
RiedelS.YaoL. and McCallumA., Modeling relations and their mentions without labeled text, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2010, pp. 148–163.
12.
HoffmannR.ZhangC.LingX.ZettlemoyerL. and WeldD.S., Knowledgebased weak supervision for information extraction of overlapping relations, in: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, Association for Computational Linguistics, 2011, pp. 541–550.
13.
SurdeanuM.TibshiraniJ.NallapatiR. and ManningC.D., Multi-instance multi-label learning for relation extraction, in: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, Association for Computational Linguistics, 2012, pp. 455–465.
14.
JiG.LiuK.HeS.ZhaoJ. et al., Distant supervision for relation extraction with sentence-level attention and entity descriptions, in: AAAI, 2017, pp. 3060–3066.
15.
FengX.GuoJ.QinB.LiuT. and LiuY., Effective deep memory networks for distant supervised relation extraction, in: Proceedings of the TwentySixth International Joint Conference on Artificial Intelligence, IJCAI, 2017, pp. 19–25.
16.
BordesA.UsunierN.Garcia-DuranA.WestonJ. and YakhnenkoO., Translating embeddings for modeling multi-relational data, in: Advances in neural information processing systems, 2013, pp. 2787–2795.
17.
WangZ.ZhangJ.FengJ. and ChenZ., Knowledge graph and text jointly embedding, in: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1591–1601.
18.
XieR.LiuZ.LuanH. and SunM., Image-embodied Knowledge Representation Learning, IJCAI, 2017, pp. 3140–3146.
19.
LinY.LiuZ.SunM.LiuY. and ZhuX., Learning entity and relation embeddings for knowledge graph completion, in: AAAI, Vol. 15, 2015, pp. 2181–2187.
20.
MikolovT.SutskeverI.ChenK.CorradoG.S. and DeanJ., Distributed representations of words and phrases and their compositionality, in: Advances in neural information processing systems, 2013, pp. 3111–3119.
21.
LuongM.-T.PhamH. and ManningC.D., Effective approaches to attention-based neural machine translation, arXiv preprint arXiv:150804025.
