Graph convolutional network combined with random walks and graph attention network for node classification

doi:10.19682/j.cnki.1005-8885.2024.1004

中国邮电高校学报(英文) ›› 2024, Vol. 31 ›› Issue (3): 1-14.doi: 10.19682/j.cnki.1005-8885.2024.1004

• Artificial Intelligence • 下一篇

Graph convolutional network combined with random walks and graph attention network for node classification

陈勇,谢小竹,翁伟

1. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China
2. Fujian Key Laboratory of Pattern Recognition and Image Understanding, Xiamen University of Technology, Xiamen 361024, China

收稿日期:2022-12-27 修回日期:2023-06-29 出版日期:2024-06-30 发布日期:2024-06-30
通讯作者: 谢小竹 E-mail:xz4xxz@gmail.com
基金资助:
the Natural Science Foundation of Xiamen (3502Z20227067).

Graph convolutional network combined with random walks and graph attention network for node classification

Chen Yong, Xie Xiaozhu, Weng Wei

1. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China
2. Fujian Key Laboratory of Pattern Recognition and Image Understanding, Xiamen University of Technology, Xiamen 361024, China

Received:2022-12-27 Revised:2023-06-29 Online:2024-06-30 Published:2024-06-30
Contact: Xiao-Zhu XIE E-mail:xz4xxz@gmail.com
Supported by:
the Natural Science Foundation of Xiamen (3502Z20227067).

摘要/Abstract

摘要： Graph conjoint attention (CAT) network is one of the best graph convolutional networks (GCNs) frameworks,
which uses a weighting mechanism to identify important neighbor nodes. However, this weighting mechanism is
learned based on static information, which means it is susceptible to noisy nodes and edges, resulting in significant
limitations. In this paper, a method is proposed to obtain context dynamically based on random walk, which allows
the context-based weighting mechanism to better avoid noise interference. Furthermore, the proposed context-based
weighting mechanism is combined with the node content-based weighting mechanism of the graph attention (GAT)
network to form a model based on a mixed weighting mechanism. The model is named as the context-based and
content-based graph convolutional network (CCGCN). CCGCN can better discover important neighbors, eliminate
noise edges, and learn node embedding by message passing. Experiments show that CCGCN achieves state-of-the-
art performance on node classification tasks in multiple datasets.

关键词: graph neural network (GNN), graph convolutional network (GCN), semi-supervised classification, graph analysis

Abstract: Graph conjoint attention (CAT) network is one of the best graph convolutional networks (GCNs) frameworks,
which uses a weighting mechanism to identify important neighbor nodes. However, this weighting mechanism is
learned based on static information, which means it is susceptible to noisy nodes and edges, resulting in significant
limitations. In this paper, a method is proposed to obtain context dynamically based on random walk, which allows
the context-based weighting mechanism to better avoid noise interference. Furthermore, the proposed context-based
weighting mechanism is combined with the node content-based weighting mechanism of the graph attention (GAT)
network to form a model based on a mixed weighting mechanism. The model is named as the context-based and
content-based graph convolutional network (CCGCN). CCGCN can better discover important neighbors, eliminate
noise edges, and learn node embedding by message passing. Experiments show that CCGCN achieves state-of-the-
art performance on node classification tasks in multiple datasets.

Key words: graph neural network (GNN), graph convolutional network (GCN), semi-supervised classification, graph analysis

Chen Yong, Xie Xiaozhu, Weng Wei. Graph convolutional network combined with random walks and graph attention network for node classification[J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(3): 1-14.

参考文献

[1] SCHADT E E. Molecular networks as sensors and drivers of common human diseases. Nature, 2009, 461(7261): 218 -223.
[2] BHAGAT S, CORMODE G, MUTHUKRISHNAN S. Node classification in social networks. AGGARWAL C C (ed). Social Network Data Analytics. Boston, MA, USA: Springer, 2011: 115 -148.
[3] XIE Z, OUYANG Z Z, LIU Q, et al. A geometric graph model for citation networks of exponentially growing scientific papers.

Physica A: Statistical Mechanics and its Applications, 2016, 456:

167 - 175.
[4] ECKLES D, ESFANDIARI H, MOSSEL E, et al. Seeding with costly network information. Proceedings of the 2019 ACM Conference on Economics and Computation (EC'19), 2019, Jun 24 - 28, Phoenix, AZ, USA. New York, NY, USA: ACM, 2019: 421 -422.
[5] XIAO S X, WANG S P, DAI Y F, et al. Graph neural networks in node classification: survey and evaluation. Machine Vision and
Applications, 2022, 33(1): 1 -19.
[6] TANG J L, AGGARWAL C, LIU H. Node classification in signed social networks. Proceedings of the 2016 SIAM International Conference on Data Mining (SDM'16), 2016, May 5 -7, Miami, FL, USA. Philadelphia, PA, USA: SIAM, 2016: 54 -62.
[7] ZHAO H, LI Z W, YOU Z H, et al. Predicting mirna-disease associations based on neighbor selection graph attention networks. IEEE/ ACM Transactions on Computational Biology and Bioinformatics, 2023, 20(2): 1298 -1307.
[8] JIA N, TIAN X L, ZHANG Y, et al. Semi-supervised node classification with discriminable squeeze excitation graph convolutional networks. IEEE Access, 2020, 8: 148226 -148236.
[9] WU Z H, PAN S R, CHEN F W, et al. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(1): 4 -24.
[10] ZHOU J, CUI G Q, HU S D, et al. Graph neural networks: A review of methods and applications. AI Open, 2020, 1: 57 -81.
[11] CRASWELL N, SZUMMER M. Random walks on the click graph. Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'07), 2007, Jul 23 - 27, Amsterdam, Netherlands. New York, NY, USA: ACM, 2007: 239 -246.
[12] RIBA P, FISCHER A, LLADÓS J, et al. Learning graph distances with message passing neural networks. Proceedings of the 24th International Conference on Pattern Recognition (ICPR'18), 2018, Aug 20 - 24, Beijing, China. Piscataway, NJ, USA: IEEE, 2018: 2239 -2244.
[13] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: Online learning of social representations. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'14), 2014, Aug 24 - 27, New York, USA. New York, NY, USA: ACM, 2014: 701 -710.
[14] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality. Advances in Neural Information Processing Systems 26: Proceedings of the 27th International Conference on Neural Information Processing Systems ( NIPS'13 ): Vol 2, 2013, Dec 5 -10, Lake Tahoe, NV, USA. Red Hook, NY, USA: Curran Associates, 2013: 3111 -3119.
[15] GROVER A, LESKOVEC J. node2vec: Scalable feature learning for networks. Proceedings of the 22nd ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining (KDD'16), 2016, Aug 13 -17, San Francisco, CA, USA. New York, NY, USA: ACM, 2016: 855 -864.
[16] HOFF P D, RAFTERY A E, HANDCOCK M S. Latent space approaches to social network analysis. Journal of the American Statistical Association, 2002, 97(460): 1090 -1098.
[17] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Message passing neural networks. SCHÜTT K T, STEFAN CHMIELA S, VON LILIENFELD O A, et al (eds). Machine Learning Meets Quantum Physics. LNP 968. Berlin, Germany: Springer, 2020: 199 -214.
[18] NIKOLENTZOS G, TIXIER A J P, VAZIRGIANNIS M. Message passing attention networks for document understanding. Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI'20), 2020, Feb 7 - 12, New York, NY, USA. Menlo Park, CA, USA: AAAI, 2020: 8544 -8551.
[19] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks. Proceedings of the 5th International Conference on Learning Representations (ICLR'17), 2017, Apr 24 -26, Toulon, France. 2017: 1 -14.
[20] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems 30: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates, 2017: 1025 -1035.
[21] VELICKOVI'C P, CUCURULL G, CASANOVA A, et al. Graph attention networks. Proceedings of the 6th International Conference
on Learning Representations (ICLR'18), 2018, Apr 30 - May 3, Vancouver, Canada. 2018: 1 -12.
[22] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Advances in Neural Information Processing Systems 30: Proceedings of the 31st International Conference on Neural Information Processing Systems ( NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates, 2017: 6000 -6010.
[23] HE T T, ONG Y S, BAI L. Learning conjoint attentions for graph neural nets. Advances in Neural Information Processing Systems
34: Proceedings of the 35th International Conference on Neural Information Processing Systems (NIPS'21), 2021, Dec 6 - 14, New Orleans, LA, USA. Red Hook, NY, USA: Curran Associates, 2021: 2641 -2653.
[24] SPERDUTI A, STARITA A. Supervised neural networks for the classification of structures. IEEE Transactions on Neural Networks, 1997, 8(3): 714 -735.
[25] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014, 15 (1): 1929 - 1958.
[26] CHEN M, WEI Z W, HUANG Z F, et al. Simple and deep graph convolutional networks. Proceedings of the 37th International Conference on Machine Learning (ICML'20): Part 3, 2020, Jul 13 - 18, Online. Red Hook, NY, USA: Curran Associates, 2020: 1703 -1713.
[27] RUIZ L, GAMA F, RIBEIRO A. Gated graph recurrent neural networks. IEEE Transactions on Signal Processing, 2020, 68: 6303 -6318.
[28] CAO S S, LU W, XU Q K. GraRep: Learning graph representations with global structural information. Proceedings of the 24th ACM International on Conference on Information and Knowledge Management ( CIKM'15 ), 2015, Oct 18 - 23, Melbourne, Australia. New York, NY, USA: ACM, 2015: 891 -900.
[29] LIU K, XUE F, HONG R. RGCF: refined graph convolution collaborative filtering with concise and expressive embedding. Intelligent Data Analysis, 2022, 26(2): 427 -445.
[30] CHIANG W L, LIU X, SI S, et al. Cluster-GCN: an efficient algorithm for training deep and large graph convolutional networks.
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD '19), 2019, Aug 4 - 8, Anchorage, AK, USA. New York, NY, USA: ACM, 2019: 257 -266.
[31] LIU M, GAO H, JI S. Towards deeper graph neural networks. Proceedings of the 26th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining (KDD'20), 2020, Jul 6 - 10, Virtual Event, CA, USA. New York, NY, USA: ACM, 2020: 338 -348.
[32] GASTEIGER J, BOJCHEVSKI A, GÜNEMANN S. Predict then propagate: graph neural networks meet personalized PageRank. Proceedings of the 6th International Conference on Learning Representations (ICLR'18), 2018, Apr 30 - May 3, Vancouver, Canada. 2018: 1 -11.
[33] XU K, LI C, TIAN Y, et al. Representation learning on graphs with jumping knowledge networks. Proceedings of the 35th International Conference on Machine Learning (ICML'18), 2018, Jul 10 -15, Stockholm, Sweden. Red Hook, NY, USA: Curran Associates, 2018: 5453 -5462.
[34] ZHAO J, DONG Y, TANG J, et al. Generalizing graph convolutional networks via heat kernel. Proceedings of the 9th International Conference on Learning Representations(ICLR'21), 2021, May 4 -8, Vienna, Austria. 2021: 553 -562.
[35] SEN P, GETOOR L. Link-based classification. London, UK: Springer, 2005: 189 -207.
[36] SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data. AI Magazine, 2008, 29(3): 93 -93.
[37] TANG J, QU M, WANG M Z, et al. LINE: Large-scale information network embedding. Proceedings of the 24th International Conference on World Wide Web (WWW'15), 2015, May 18 -22, Florence, Italy. Geneva, Switzerland: International World Wide Web Conferences Steering Committee, 2015: 1067 - 1077.
[38] VELICKOVI'C P, FEDUS W, HAMILTON W L, et al. Deep graph infomax. Proceedings of the 7th International Conference on Learning Representations ( ICLR'19), 2019, May 6 - 9, New Orleans, LA, USA. 2019: 1 -17.
[39] WU F L, ZHANG Y T, SOUZA A H, et al. Simplifying graph convolutional networks. Proceedings of the 36th International Conference on Machine Learning (ICML'19), 2019, Jun 10 -13, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates, 2019: 6861 -6871.

Graph convolutional network combined with random walks and graph attention network for node classification

Graph convolutional network combined with random walks and graph attention network for node classification

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价