Knowledge Graph Representation Learning Model Based on Capsule Network and Information Fusion

Chu Zhao; Gilja  So; Rui  Chen

doi:10.6180/jase.202601_29(1).0009

Knowledge Graph Representation Learning Model Based on Capsule Network and Information Fusion

Computer Science and Information Engineering

Chu Zhao^1,3, Gilja So²This email address is being protected from spambots. You need JavaScript enabled to view it., and Rui Chen³

¹Department of Computer and Information Engineering, Graduate School Youngsan University, South Korea

²Department of Cyber Security Youngsan University, South Korea

³Software College, Shenyang Normal University, Shenyang, 110034, China

Received: August 30, 2024
Accepted: March 31, 2025
Publication Date: May 1, 2025

Copyright The Author(s). This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are cited.

Download Citation: ||https://doi.org/10.6180/jase.202601_29(1).0009

In recent years, knowledge representation learning has played a key role in intelligent recommendation, intelligent question-answering, and intelligent retrieval, and has been widely concerned. Knowledge representation learning aims to vectorize semantic information and deduce knowledge through mathematical formulas with the help of low dimensional embedding of entity and relation. Although knowledge representation learning based on knowledge graph can obtain entity structure and relational embedding, it lacks semantic information utilization of entity description text. In addition, with the increase of the scale of the knowledge graph, the categories and quantities of entities and relationships, as well as the content and sources of entity descriptions, the correspondence between the textual descriptions of entities and the triplet structure information becomes more difficult to obtain. Therefore, we propose a novel knowledge graph representation learning model based on capsule network and information fusion in this paper. Based on anchor node and neighbor node and the relational sampling strategy, each node on the knowledge graph is represented by the predicted operator graph. The capsule network is used to gather the image features for each node to obtain the node representation vector, which is finally input to the decoder to calculate the score. In particular, we construct a loss function for the multi-layer attention mechanism of entity structure and semantic fusion. Experimental results show that the proposed method can effectively deduce the hidden link relationship between entities containing complex entity descriptions, and has more accurate classification accuracy than other methods in triplet classification tasks.

Keywords: Knowledge representation learning; Capsule network; Information fusion; Multi-layer attention mechanism

[1] Z. Chen, Y. Wang, B. Zhao, J. Cheng, X. Zhao, and Z. Duan, (2020) “Knowledge graph completion: A review" Ieee Access 8: 192435–192456. DOI: 10.1109/ACCESS.2020.3030076.
[2] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives. “Dbpedia: A nucleus for a web of open data”. In: international semantic web conference. Springer. 2007, 722–735. DOI: 10.1007/978-3-540 76298-0_52.
[3] A. Madkour, W. G. Aref, and S. Basalamah. “Knowledge cubes—A proposal for scalable and semantically-guided management of Big Data”. In: 2013 IEEE International Conference on Big Data. IEEE. 2013, 1–7. DOI: 10.1109/BigData.2013.6691800.
[4] F. M. Suchanek, G. Kasneci, and G. Weikum, (2008) “Yago: Alarge ontology from wikipedia and wordnet"Journal of Web Semantics 6(3): 203–217. DOI: 10.1016/j. websem.2008.06.001.
[5] R. Xie, Z. Liu, J. Jia, H. Luan, and M. Sun. “Representation learning of knowledge graphs with entity descriptions”. In: Proceedings of the AAAI conference on artificial intelligence. 30. 1. 2016. DOI: 10.1609/aaai.v30i1.10329.
[6] H. Zhu, D. Xu, Y. Huang, Z. Jin, W. Ding, J. Tong, and G. Chong, (2024) “Graph structure enhanced pre training language model for knowledge graph completion" IEEE Transactions on Emerging Topics in Compu tational Intelligence: DOI: 10.1109/TETCI.2024.3372442.
[7] S. Feng, C. Zhou, Q. Liu, X. Ji, and M. Huang, (2024) “Temporal Knowledge Graph Reasoning Based on Entity Relationship Similarity Perception" Electronics 13(12): 2417. DOI: 10.3390/electronics13122417.
[8] H. Yang and J. Liu. “Knowledge graph representation learning as groupoid: unifying TransE, RotatE, QuatE, ComplEx”. In: Proceedings of the 30th ACM international conference on information & knowledge management. 2021, 2311–2320. DOI: 10.1145/3459637.3482442.
[9] C. Jin, R. Cui, and Y. Zhao. “Research on Chinese Korean Entity Alignment Method Combining TransH and GAT”. In: China Conference on Knowledge Graph and Semantic Computing. Springer. 2021, 134–144. DOI: 10.1007/978-981-16-6471-7_10.
[10] F. Liu, Y. Shen, T. Zhang, and H. Gao, (2020) “Entity related paths modeling for knowledge base completion" Frontiers of Computer Science 14: 1–10. DOI: 10.1007/s11704-019-8264-4.
[11] M.Zhang and Y.Chen, (2018) “Link prediction based on graph neural networks" Advances in neural infor mation processing systems 31: DOI: 10.1007/978981-16-6054-2_10.
[12] H. Xiao, M. Huang, Y. Hao, and X. Zhu, (2015) “TransG: A generative mixture model for knowledge graph embedding" arXiv preprint arXiv:1509.05488: DOI: 10. 48550/arXiv.1509.05488.
[13] X.Tang, L.Chen, J.Cui, and B.Wei,(2019)“Knowledge representation learning with entity descriptions, hierarchical types, and textual relations" Information Processing & Management 56(3): 809–822. DOI: 10.1016/j.ipm.2019.01.005.
[14] B. An, B. Chen, X. Han, and L. Sun. “Accurate text enhanced knowledge graph representation learning”. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018, 745–755. DOI: 10.18653/v1/N18-1068.
[15] Y. Zhao, H. Feng, H. Zhou, Y. Yang, X. Chen, R. Xie, F. Zhuang, and Q. Li, (2022) “EIGAT: Incorporating global information in local attention for knowledge representation learning" Knowledge-Based Systems 237: 107909. DOI: 10.1016/j.knosys.2021.107909.
[16] Z. Li, X. Liu, X. Wang, P. Liu, and Y. Shen, (2023) “Transo: a knowledge-driven representation learning method with ontology information constraints" World Wide Web26(1): 297–319. DOI: 10.1007/s11280-022-01016-3.
[17] Z. Li, X. Jin, W. Li, S. Guan, J. Guo, H. Shen, Y. Wang, and X. Cheng. “Temporal knowledge graph reasoning based one volutional representation learning”. In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval. 2021, 408–417. DOI: 10.1145/3404835.3462963.
[18] J. Hao, M. Chen, W. Yu, Y. Sun, and W. Wang. “Uni versal representation learning of knowledge bases by jointly embedding instances and ontological concepts”. In: Proceedings of the 25th ACM SIGKDD Inter national Conference on Knowledge Discovery & Data Mining. 2019, 1709–1719. DOI: 10.1145/3292500.3330838.
[19] H.Mousselly-Sergieh, T. Botschen, I. Gurevych, and S. Roth. “A multimodal translation-based approach for knowledge graph representation learning”. In: Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. 2018, 225–234. DOI: 10.18653/v1/S18-2027.
[20] N. Passalis, M. Tzelepi, and A. Tefas, (2020) “Proba bilistic knowledge transfer for lightweight deep representation learning" IEEE Transactions on Neural Networks and learning systems 32(5): 2030–2039. DOI: 10.1109/TNNLS.2020.2995884.
[21] T. Chen, L. Lin, R. Chen, Y. Wu, and X. Luo, (2018) “Knowledge-embedded representation learning for fine-grained image recognition" arXiv preprint arXiv:1807.00505: DOI: 10.48550/arXiv.1807.00505.
[22] W.Gan, Y. Sun, and Y. Sun, (2022) “Knowledge structure enhanced graph representation learning model for attentive knowledge tracing" International Journal of Intelligent Systems 37(3): 2012–2045. DOI: 10.1002/int.22763.
[23] B. Wang, T. Shen, G. Long, T. Zhou, Y. Wang, and Y. Chang. “Structure-augmented text representation learning for efficient knowledge graph completion”. In: Proceedings of the Web Conference 2021. 2021, 1737–1748. DOI: 10.1145/3442381.3450043.
[24] C. Zhao, J. Jiang, Y. Guan, X. Guo, and B. He, (2018) “EMR-based medical knowledge representation and inference via Markov random fields and distributed representation learning" Artificial intelligence in medicine 87: 49–59. DOI: 10.1016/j.artmed.2018.03.005.
[25] J. Zhang, S. Liang, Y. Sheng, and J. Shao, (2022) “Tem poral knowledge graph representation learning with local and global evolutions" Knowledge-Based Systems251: 109234. DOI: 10.1016/j.knosys.2022.109234.
[26] G. Niu, Y. Zhang, B. Li, P. Cui, S. Liu, J. Li, and X. Zhang. “Rule-guided compositional representation learning on knowledge graphs”. In: Proceedings of the AAAI conference on artificial intelligence. 34. 03. 2020, 2950–2958. DOI: 10.1609/aaai.v34i03.5687.
[27] H. Liu, C. Li, Y. Li, and Y. J. Lee. “Improved base lines with visual instruction tuning”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024, 26296–26306. DOI: 10.1109/CVPR52733.2024.02484.
[28] T. Shen, Y. Mao, P. He, G. Long, A. Trischler, and W. Chen, (2020) “Exploiting structured knowledge in text via graph-guided representation learning" arXiv preprint arXiv:2004.14224: DOI: 10.48550/arXiv.2004. 14224.
[29] S. Seo, B. Oh, and K.-H. Lee, (2020) “Reliable knowledge graph path representation learning" IEEE Access 8: 32816–32825. DOI: 10.1109/ACCESS.2020.2973923.
[30] M.Fan, Q.Zhou, T.F.Zheng, and R.Grishman,(2017) “Distributed representation learning for knowledge graphs with entity descriptions" Pattern Recognition Letters 93: 31–37. DOI: 10.1016/j.patrec.2016.09.005.
[31] P.Wang, S.Li, and R.Pan.“Incorporating ganfornegative sampling in knowledge representation learning”. In: Proceedings of the AAAI conference on artificial intelligence. 32. 1. 2018. DOI: 10.1609/aaai.v32i1.11536.
[32] Z. Feng, D. Tang, X. Feng, C. Zhou, J. Liao, S. Wu, B. Qin, Y. Cao, and S. Shi, (2024) “Pretraining without wordpieces: learning over a vocabulary of millions of words" International Journal of Machine Learning and Cybernetics 15(9): 3989–3998. DOI: 10.1007/s13042-024-02132-4.
[33] Y.Jiang and S. Yin, (2023) “Heterogenous-view occluded expression data recognition based on cycle-consistent ad versarial network and K-SVD dictionary learning under intelligent cooperative robot environment" Computer Science and Information Systems 20(4): 1869–1883. DOI: 10.2298/CSIS221228034J.
[34] Z. Zhang, L. Cao, X. Chen, W. Tang, Z. Xu, and Y. Meng, (2020) “Representation learning of knowledge graphs with entity attributes" IEEE Access 8: 7435 7441. DOI: 10.1109/ACCESS.2020.2963990.
[35] S. Yin, H. Li, A. A. Laghari, T. R. Gadekallu, G. A. Sampedro, and A. Almadhor, (2024) “An Anomaly Detection Model Based on Deep Auto-Encoder and Cap sule Graph Convolution via Sparrow Search Algorithm in 6G Internet of Everything" IEEE Internet of Things Journal 11(18): 29402–29411. DOI: 10.1109/JIOT.2024. 3353337.
[36] Z.Li, Q.Zhang, F.Zhu, D.Li, C.Zheng, and Y.Zhang, (2023) “Knowledge graph representation learning with simplifying hierarchical feature propagation" Information Processing & Management 60(4): 103348. DOI: 10.1016/j.ipm.2023.103348.