Abstract
Knowledge Graphs (KGs) are often incomplete and sparse. Knowledge graph reasoning aims at completing the KG by predicting missing paths between entities. The reinforcement learning (RL) based method is one of the state-of-the-art approaches to this work. However, existing RL-based methods have some problems, such as unstable training and poor reward function. Although the DIVINE framework, which a novel plug-and-play framework based on generative adversarial imitation learning, improved existing RL-based algorithms without extra reward engineering, the rate of policy update is slow. This paper proposes the EN-DIVINE framework, using Proximal Policy Optimization algorithms to perform gradient descent when discriminator parameters take policy steps to improve the framework’s training speed. Experimental results show that our work can provide an accessible improvement for the DIVINE framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data (SIGMOD 2008), pp. 1247–1250. Association for Computing Machinery, New York (2008). https://doi.org/10.1145/1376616.1376746
Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., et al.: Toward an architecture for never-ending language learning. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2010), pp. 1306–1313. AAAI Press (2010)
Miller, G.: WordNet: a lexical database for English. In: Proceedings of the workshop on Human Language Technology (HLT 1993), p. 409. Association for Computational Linguistics, USA (1993). https://doi.org/10.3115/1075671.1075788
Lao, N., Mitchell, T.M., Cohen, W.W.: Random walk inference and learning in a large scale knowledge base. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2011), pp. 529–539. Association for Computational Linguistics, USA (2011)
Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: NIPS (2013)
Richardson, M., Pedro, M.: Domingos: Markov logic networks. Mach. Learn. 62, 107–136 (2006)
Xiong, W., Hoang, T., Wang, W.Y.: DeepPath: a reinforcement learning method for knowledge graph reasoning. In: EMNLP, pp. 564–573 (2017). https://doi.org/10.18653/v1/D17-1060
Das, R., et al.: Go for a walk and arrive at the answer: reasoning over paths in knowledge bases using reinforcement learning. arXiv abs/1711.05851 (2018)
Li, R., Cheng, X.: DIVINE: a generative adversarial imitation learning framework for knowledge graph reasoning. In: EMNLP/IJCNLP, pp. 2642–2651 (2019). https://doi.org/10.18653/v1/D19-1266
Dong, L., Lapata, M.: Coarse-to-fine decoding for neural semantic parsing. In: ACL, pp. 731–742 (2018). https://doi.org/10.18653/v1/P18-1068
Hu, S., Zou, L., Zhang, X.: A state-transition framework to answer complex questions over knowledge base. In: EMNLP, pp. 2098–2108 (2018). https://doi.org/10.18653/v1/D18-1234
Luo, K., Lin, F., Luo, X., Zhu, K.Q.: Knowledge base question answering via encoding of complex query graphs. In: EMNLP, pp. 2185–2194 (2018). https://doi.org/10.18653/v1/D18-1242
Wang, X., Wang, D., Xu, C., He, X., Cao, Y., Chua, T.-S.: Explainable reasoning over knowledge graphs for recommendation. arXiv:1811.04540 (2019)
Schulman, J., Levine, S., Abbeel, P., Jordan, M.I., Moritz, P.: Trust region policy optimization. arXiv:1502.05477 (2015)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv:1707.06347 (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2015)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., et al.: Asynchronous methods for deep reinforcement learning. arXiv:1602.01783 (2016)
Ho, J., Ermon, S.: Generative adversarial imitation learning. arXiv:1606.03476 (2016)
Ruder, S.: An overview of gradient descent optimization algorithms. arXiv:1609.04747 (2016)
Zeiler, M.D.: ADADELTA: an adaptive learning rate method. arXiv:1212.5701 (2012)
Acknowlegements
This work is supported by the National Natural Science Foundation of China under Grant No.61662054, Inner Mongolia Colleges and Universities of Young Technology Talent Support Program under Grant No. NJYT-19-A02, the Major Project of Inner Mongolia Natural Science Foundation: Research on Key Technologies of Cloud Support for Big Data Intelligent Analysis under Grant No.2019ZD15, Research and application on Key Technologies for Discipline Inspection and Supervision’s Big Data under Grant No.2019GG372, Inner Mongolia Science and Technology Innovation Team of Cloud Computing and Software Engineering, and Inner Mongolia Application Technology Research and Development Funding Project “Mutual Creation Service Platform Research and Development Based on Service Optimizing and Operation Integrating”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wu, Y., Zhou, J. (2021). EN-DIVINE: An Enhanced Generative Adversarial Imitation Learning Framework for Knowledge Graph Reasoning. In: Qiu, H., Zhang, C., Fei, Z., Qiu, M., Kung, SY. (eds) Knowledge Science, Engineering and Management. KSEM 2021. Lecture Notes in Computer Science(), vol 12815. Springer, Cham. https://doi.org/10.1007/978-3-030-82136-4_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-82136-4_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82135-7
Online ISBN: 978-3-030-82136-4
eBook Packages: Computer ScienceComputer Science (R0)