Graph convolutional network combined with random walks and graph attention network for node classification

doi:10.19682/j.cnki.1005-8885.2024.1004

中国邮电高校学报(英文) ›› 2024, Vol. 31 ›› Issue (3): 1-14.doi: 10.19682/j.cnki.1005-8885.2024.1004

• Artificial Intelligence • 下一篇

Graph convolutional network combined with random walks and graph attention network for node classification

陈勇,谢小竹,翁伟

1. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China
2. Fujian Key Laboratory of Pattern Recognition and Image Understanding, Xiamen University of Technology, Xiamen 361024, China

收稿日期:2022-12-27 修回日期:2023-06-29 出版日期:2024-06-30 发布日期:2024-06-30
通讯作者: 谢小竹 E-mail:xz4xxz@gmail.com
基金资助:
the Natural Science Foundation of Xiamen (3502Z20227067).

Graph convolutional network combined with random walks and graph attention network for node classification

Chen Yong, Xie Xiaozhu, Weng Wei

1. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen 361024, China
2. Fujian Key Laboratory of Pattern Recognition and Image Understanding, Xiamen University of Technology, Xiamen 361024, China

Received:2022-12-27 Revised:2023-06-29 Online:2024-06-30 Published:2024-06-30
Contact: Xiao-Zhu XIE E-mail:xz4xxz@gmail.com
Supported by:
the Natural Science Foundation of Xiamen (3502Z20227067).

摘要/Abstract

摘要： Graph conjoint attention (CAT) network is one of the best graph convolutional networks (GCNs) frameworks,
which uses a weighting mechanism to identify important neighbor nodes. However, this weighting mechanism is
learned based on static information, which means it is susceptible to noisy nodes and edges, resulting in significant
limitations. In this paper, a method is proposed to obtain context dynamically based on random walk, which allows
the context-based weighting mechanism to better avoid noise interference. Furthermore, the proposed context-based
weighting mechanism is combined with the node content-based weighting mechanism of the graph attention (GAT)
network to form a model based on a mixed weighting mechanism. The model is named as the context-based and
content-based graph convolutional network (CCGCN). CCGCN can better discover important neighbors, eliminate
noise edges, and learn node embedding by message passing. Experiments show that CCGCN achieves state-of-the-
art performance on node classification tasks in multiple datasets.

关键词: graph neural network (GNN), graph convolutional network (GCN), semi-supervised classification, graph analysis

Abstract: Graph conjoint attention (CAT) network is one of the best graph convolutional networks (GCNs) frameworks,
which uses a weighting mechanism to identify important neighbor nodes. However, this weighting mechanism is
learned based on static information, which means it is susceptible to noisy nodes and edges, resulting in significant
limitations. In this paper, a method is proposed to obtain context dynamically based on random walk, which allows
the context-based weighting mechanism to better avoid noise interference. Furthermore, the proposed context-based
weighting mechanism is combined with the node content-based weighting mechanism of the graph attention (GAT)
network to form a model based on a mixed weighting mechanism. The model is named as the context-based and
content-based graph convolutional network (CCGCN). CCGCN can better discover important neighbors, eliminate
noise edges, and learn node embedding by message passing. Experiments show that CCGCN achieves state-of-the-
art performance on node classification tasks in multiple datasets.

Key words: graph neural network (GNN), graph convolutional network (GCN), semi-supervised classification, graph analysis

Chen Yong, Xie Xiaozhu, Weng Wei. Graph convolutional network combined with random walks and graph attention network for node classification[J]. The Journal of China Universities of Posts and Telecommunications, 2024, 31(3): 1-14.

参考文献

[1] SCHADT E E. Molecular networks as sensors and drivers of
common human diseases. Nature, 2009, 461(7261): 218 -223.
[2] BHAGAT S, CORMODE G, MUTHUKRISHNAN S. Node
classification in social networks. AGGARWAL C C (ed). Social
Network Data Analytics. Boston, MA, USA: Springer, 2011:
115 -148.
[3] XIE Z, OUYANG Z Z, LIU Q, et al. A geometric graph model for
[4] ECKLES D, ESFANDIARI H, MOSSEL E, et al. Seeding with
costly network information. Proceedings of the 2019 ACM
Conference on Economics and Computation (EC'19), 2019, Jun
24 - 28, Phoenix, AZ, USA. New York, NY, USA: ACM,
2019: 421 -422.
[5] XIAO S X, WANG S P, DAI Y F, et al. Graph neural networks
in node classification: survey and evaluation. Machine Vision and
Applications, 2022, 33(1): 1 -19.
[6] TANG J L, AGGARWAL C, LIU H. Node classification in signed
social networks. Proceedings of the 2016 SIAM International
Conference on Data Mining (SDM'16), 2016, May 5 -7, Miami,
FL, USA. Philadelphia, PA, USA: SIAM, 2016: 54 -62.
[7] ZHAO H, LI Z W, YOU Z H, et al. Predicting mirna-disease
associations based on neighbor selection graph attention networks.
IEEE/ ACM Transactions on Computational Biology and
Bioinformatics, 2023, 20(2): 1298 -1307.
[8] JIA N, TIAN X L, ZHANG Y, et al. Semi-supervised node
classification with discriminable squeeze excitation graph
convolutional networks. IEEE Access, 2020, 8: 148226 -
148236.
[9] WU Z H, PAN S R, CHEN F W, et al. A comprehensive survey
on graph neural networks. IEEE Transactions on Neural Networks
and Learning Systems, 2020, 32(1): 4 -24.
[10] ZHOU J, CUI G Q, HU S D, et al. Graph neural networks: A
review of methods and applications. AI Open, 2020, 1: 57 -81.
[11] CRASWELL N, SZUMMER M. Random walks on the click graph.
Proceedings of the 30th Annual International ACM SIGIR
Conference on Research and Development in Information Retrieval
(SIGIR'07), 2007, Jul 23 - 27, Amsterdam, Netherlands. New
York, NY, USA: ACM, 2007: 239 -246.
[12] RIBA P, FISCHER A, LLADÓS J, et al. Learning graph
distances with message passing neural networks. Proceedings of the
24th International Conference on Pattern Recognition (ICPR'18),
2018, Aug 20 - 24, Beijing, China. Piscataway, NJ, USA:
IEEE, 2018: 2239 -2244.
[13] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: Online
learning of social representations. Proceedings of the 20th ACM
SIGKDD International Conference on Knowledge Discovery and
Data Mining (KDD'14), 2014, Aug 24 - 27, New York, USA.
New York, NY, USA: ACM, 2014: 701 -710.
[14] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed
representations of words and phrases and their compositionality.
Advances in Neural Information Processing Systems 26:
Proceedings of the 27th International Conference on Neural
Information Processing Systems ( NIPS'13 ): Vol 2, 2013,
Dec 5 -10, Lake Tahoe, NV, USA. Red Hook, NY, USA:
Curran Associates, 2013: 3111 -3119.
[15] GROVER A, LESKOVEC J. node2vec: Scalable feature learning
for networks. Proceedings of the 22nd ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining (KDD'16),
2016, Aug 13 -17, San Francisco, CA, USA. New York, NY,
USA: ACM, 2016: 855 -864.
[16] HOFF P D, RAFTERY A E, HANDCOCK M S. Latent space
approaches to social network analysis. Journal of the American
Statistical Association, 2002, 97(460): 1090 -1098.
[17] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Message
passing neural networks. SCHÜTT K T, STEFAN CHMIELA S,
VON LILIENFELD O A, et al (eds). Machine Learning Meets
Quantum Physics. LNP 968. Berlin, Germany: Springer, 2020:
199 -214.
[18] NIKOLENTZOS G, TIXIER A J P, VAZIRGIANNIS M. Message
passing attention networks for document understanding.
Proceedings of the 34th AAAI Conference on Artificial Intelligence
(AAAI'20), 2020, Feb 7 - 12, New York, NY, USA. Menlo
Park, CA, USA: AAAI, 2020: 8544 -8551.
[19] KIPF T N, WELLING M. Semi-supervised classification with
graph convolutional networks. Proceedings of the 5th International
Conference on Learning Representations (ICLR'17), 2017, Apr
24 -26, Toulon, France. 2017: 1 -14.
[20] HAMILTON W L, YING R, LESKOVEC J. Inductive
representation learning on large graphs. Advances in Neural
Information Processing Systems 30: Proceedings of the 31st
International Conference on Neural Information Processing Systems
(NIPS'17), 2017, Dec 4 - 9, Long Beach, CA, USA. Red
Hook, NY, USA: Curran Associates, 2017: 1025 -1035.
[21] VELICKOVI'C P, CUCURULL G, CASANOVA A, et al. Graph
attention networks. Proceedings of the 6th International Conference
on Learning Representations (ICLR'18), 2018, Apr 30 - May 3,
Vancouver, Canada. 2018: 1 -12.
[22] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all
you need. Advances in Neural Information Processing Systems 30:
Proceedings of the 31st International Conference on Neural
Information Processing Systems ( NIPS'17), 2017, Dec 4 - 9,
Long Beach, CA, USA. Red Hook, NY, USA: Curran
Associates, 2017: 6000 -6010.
[23] HE T T, ONG Y S, BAI L. Learning conjoint attentions for graph
neural nets. Advances in Neural Information Processing Systems
34: Proceedings of the 35th International Conference on Neural
Information Processing Systems (NIPS'21), 2021, Dec 6 - 14,
New Orleans, LA, USA. Red Hook, NY, USA: Curran
Associates, 2021: 2641 -2653.
[24] SPERDUTI A, STARITA A. Supervised neural networks for the
classification of structures. IEEE Transactions on Neural
Networks, 1997, 8(3): 714 -735.
[25] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:
a simple way to prevent neural networks from overfitting. The
Journal of Machine Learning Research, 2014, 15 (1): 1929 -
1958.
[26] CHEN M, WEI Z W, HUANG Z F, et al. Simple and deep graph
convolutional networks. Proceedings of the 37th International
Conference on Machine Learning (ICML'20): Part 3, 2020, Jul
13 - 18, Online. Red Hook, NY, USA: Curran Associates,
2020: 1703 -1713.
[27] RUIZ L, GAMA F, RIBEIRO A. Gated graph recurrent neural
networks. IEEE Transactions on Signal Processing, 2020, 68:
6303 -6318.
[28] CAO S S, LU W, XU Q K. GraRep: Learning graph
representations with global structural information. Proceedings of
the 24th ACM International on Conference on Information and
Knowledge Management ( CIKM'15 ), 2015, Oct 18 - 23,
Melbourne, Australia. New York, NY, USA: ACM, 2015:
891 -900.
[29] LIU K, XUE F, HONG R. RGCF: refined graph convolution
collaborative filtering with concise and expressive embedding.
Intelligent Data Analysis, 2022, 26(2): 427 -445.
[30] CHIANG W L, LIU X, SI S, et al. Cluster-GCN: an efficient
algorithm for training deep and large graph convolutional networks.
Proceedings of the 25th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining (KDD '19), 2019, Aug 4 -
8, Anchorage, AK, USA. New York, NY, USA: ACM, 2019:
257 -266.
[31] LIU M, GAO H, JI S. Towards deeper graph neural networks.
Proceedings of the 26th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining (KDD'20), 2020, Jul 6 -
10, Virtual Event, CA, USA. New York, NY, USA: ACM,
2020: 338 -348.
[32] GASTEIGER J, BOJCHEVSKI A, GÜNEMANN S. Predict then
propagate: graph neural networks meet personalized PageRank.
Proceedings of the 6th International Conference on Learning
Representations (ICLR'18), 2018, Apr 30 - May 3, Vancouver,
Canada. 2018: 1 -11.
[33] XU K, LI C, TIAN Y, et al. Representation learning on graphs
with jumping knowledge networks. Proceedings of the 35th
International Conference on Machine Learning (ICML'18), 2018,
Jul 10 -15, Stockholm, Sweden. Red Hook, NY, USA: Curran
Associates, 2018: 5453 -5462.
[34] ZHAO J, DONG Y, TANG J, et al. Generalizing graph
convolutional networks via heat kernel. Proceedings of the 9th
International Conference on Learning Representations(ICLR'21),
2021, May 4 -8, Vienna, Austria. 2021: 553 -562.
[35] SEN P, GETOOR L. Link-based classification. London, UK:
Springer, 2005: 189 -207.
[36] SEN P, NAMATA G, BILGIC M, et al. Collective classification
in network data. AI Magazine, 2008, 29(3): 93 -93.
[37] TANG J, QU M, WANG M Z, et al. LINE: Large-scale
information network embedding. Proceedings of the 24th
International Conference on World Wide Web (WWW'15), 2015,
May 18 -22, Florence, Italy. Geneva, Switzerland: International
World Wide Web Conferences Steering Committee, 2015: 1067 -
1077.
[38] VELICKOVI'C P, FEDUS W, HAMILTON W L, et al. Deep
graph infomax. Proceedings of the 7th International Conference on
Learning Representations ( ICLR'19), 2019, May 6 - 9, New
Orleans, LA, USA. 2019: 1 -17.
[39] WU F L, ZHANG Y T, SOUZA A H, et al. Simplifying graph
convolutional networks. Proceedings of the 36th International
Conference on Machine Learning (ICML'19), 2019, Jun 10 -13,
Long Beach, CA, USA. Red Hook, NY, USA: Curran
Associates, 2019: 6861 -6871.

Graph convolutional network combined with random walks and graph attention network for node classification

Graph convolutional network combined with random walks and graph attention network for node classification

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价