A Review on Graph Theory in Deep Learning
DOI:
https://doi.org/10.56286/wm2yfj62Keywords:
Graph Neural Networks (GNNs), Deep learning, Graph TheoryAbstract
Graph theory has gained popularity as a flexible tool in machine learning to capture complex correlations between objects in non-Euclidean structured data. Graph Neural Networks (GNNs) have proved to be effective deep learning methods to learn node and graph-level representations using message passing, attention and convolution applied to graph structures. This review classifies more than 39 GNN variants into five structural families: Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Sample and Aggregate (SAGE), Graph Isomorphism Networks (GINs) and attention-augmented or hybrid models. In addition to the recent models AA-HGNN, DySAT, EGNN, DyHAN, Hyper-GNN, and MGH, which are built to include heterogeneity, scalability, dynamic graphs, and geometric invariance. These models acknowledge the increasing diversity of GNN architectures adapted to real-life requirements in such areas as recommendation systems, bioinformatics, and social networks. Additionally, the review lists important challenges to GNN development, which included overfitting, explainability, and multi-modal learning, and provides directions in the future. To illustrate the development and use of GNN models in numerous directions, this paper provides a graph-centric structure through which researchers can learn how the GNN models are evolving and how it is used in several directions.
Downloads
References
Wu, S., Sun, F., Zhang, W., Xie, X., & Cui, B. (2022). Graph Neural Networks in Recommender Systems: A Survey. https://dl.acm.org/doi/pdf/10.1145/3535101.
Al-Jawadi, O. M., & Al-Shumam, A. A. (2020). A review on the relationship between computer engineering, discrete-math, and graph theory. Tikrit Journal of Engineering Sciences, 27(3), 38-44. https://doi.org/10.25130/tjes.27.3.05.
Pouyanfar, S., Sadiq, S., Yan, Y., Tian, H., Tao, Y., & others. (2018). A survey on deep learning: Algorithms, techniques, and applications. ACM Computing Surveys, 51(5), 1-36. https://proceedings.mlr.press/v198/zhu22a/zhu22a.pdf.
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2020). A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1), 4-24. https://doi.org/10.1109/TNNLS.2020.2978386.
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020). Graph neural networks: A review of methods and applications. AI Open, 1, 57-81. https://doi.org/10.1016/j.aiopen.2021.01.001.
Xu, K., et al. (2019). "How Powerful are Graph Neural Networks?" International Conference on Learning Representations (ICLR). https://arxiv.org/pdf/1810.00826.
Zhang, X. M., Liang, L., Liu, L., & Tang, M. J. (2021). Graph neural networks and their current applications in bioinformatics. Frontiers in Genetics, 12, 690049. https://oaktrust.library.tamu.edu/items/0a282300-8ac3-486c-9967-2aa49c54d706.
Chen, C., Wu, Y., Dai, Q., Zhou, H.-Y., Xu, M., Yang, S., Han, X., & Yu, Y. (2024). A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 12345-12367. https://arxiv.org/pdf/2209.13232.
Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., & Leskovec, J. (2018). "Graph Convolutional Neural Networks for Web-Scale Recommender Systems." Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 974-983. https://dl.acm.org/doi/pdf/10.1145/3219819.3219947.
Nguyen, M., Le, H., Pham, T., Nguyen, T., & Tran, T. (2021). Graph Convolutional Networks for Drug Response Prediction. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 18(6), 2131-2140. https://www.biorxiv.org/content/10.1101/2020.04.07.030908v2.full.pdf.
Lin, X., Quan, Z., Wang, Z. J., Ma, T., & Zeng, X. (2020). KGNN: Knowledge Graph Neural Network for Drug-Drug Interaction Prediction. Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2020. Retrieved from https://xuanlin1991.github.io/files/publications/ijcai20.pdf.
Kipf, T. N., & Welling, M. (2017). "Semi-Supervised Classification with Graph Convolutional Networks." International Conference on Learning Representations (ICLR). https://arxiv.org/pdf/1609.02907
Gao, H., Wang, Z., & Ji, S. (2018). Large-Scale Learnable Graph Convolutional Networks. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 1416–1424. https://dl.acm.org/doi/pdf/10.1145/3219819.3219947
Veli?kovi?, P., et al. (2018). "Graph Attention Networks." International Conference on Learning Representations (ICLR). https://arxiv.org/pdf/1710.10903.
Li, Y., Tarlow, D., Brockschmidt, M., & Zemel, R. (2016). Gated Graph Sequence Neural Networks. In International Conference on Learning Representations https://arxiv.org/pdf/1511.05493.
Ren, Y., Wang, B., Zhang, J., & Chang, Y. (2021, January 27). Adversarial Active Learning based Heterogeneous Graph Neural Network for Fake News Detection (arXiv:2101.11206). arXiv. https://doi.org/10.48550/arXiv.2101.11206.
Sankar, A., Wu, Y., Gou, L., Zhang, W., & Yang, H. (2020, February). DySAT: Deep Neural Representation Learning on Dynamic Graphs via Self?Attention Networks. In Proceedings of the 13th ACM International Conference on Web Search and Data Mining (WSDM ’20) (pp. 519–527). ACM. https://doi.org/10.1145/3336191.3371845 .
Karczewski, R., Souza, A. H., & Garg, V. (2024). On the Generalization of Equivariant Graph Neural Networks. In Proceedings of the 41st International Conference on Machine Learning (ICML) (Vol. 235, pp. 23159 23186). PMLR. https://proceedings.mlr.press/v235/karczewski24a.html?utm_source=chatgpt.com
Martirano, L., Ienco, D., Interdonato, R., & Tagarelli, A. (2024). DyHANE: Dynamic heterogeneous attributed network embedding through experience node replay. Applied Network Science, 9(1), Article 30. https://pmc.ncbi.nlm.nih.gov/articles/PMC7148053/?utm_source=chatgpt.com . .
Feng, Y., You, H., Zhang, Z., Ji, R., & Gao, Y. (2019, July). Hypergraph neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 3558–3565. https://doi.org/10.1609/aaai.v33i01.33013558.
Cai, L., Chen, Z., Luo, C., Gui, J., Ni, J., Li, D., & Chen, H. (2021). "Structural temporal graph neural networks for anomaly detection in dynamic graphs". In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM '21) (pp. 1–10). ACM. https://doi.org/10.1145/3459637.3481955.
Guan, C., Wang, X., Chen, H., Zhang, Z., & Zhu, W. (2022). Large-Scale Graph Neural Architecture Search. Proceedings of the 39th International Conference on Machine Learning, 162, 7968–7981. https://proceedings.mlr.press/v162/guan22d/guan22d.pdf
Ma, X., Wu, J., Xue, S., Yang, J., Zhou, C., Sheng, Q. Z., Xiong, H., & Akoglu, L. (2021). A Comprehensive Survey on Graph Anomaly Detection with Deep Learning. IEEE Transactions on Knowledge and Data Engineering. https://arxiv.org/pdf/2106.07178.
Wang, J., Zhang, S., Xiao, Y., & Song, R. (2022). A review on graph neural network methods in financial applications. Journal of Data Science, 20(2), 111–134. https://doi.org/10.6339/22-JDS1047
Boyaci, O. (2022). Graph Neural Network-Based Cybersecurity of Smart Grids. Texas A&M University. Retrieved from https://oaktrust.library.tamu.edu/items/0a282300-8ac3-486c-9967-2aa49c54d706.
Khaleel, T. A., & Al-Shumam, A. A. (2020). A study of graph theory applications in IT security. Iraqi Journal of Science, 61(10), 2774-2789. https://doi.org/10.24996/ijs.2020.61.10.17.
Al-Jawadi, O. M. (2023). Enhancing drivers' attention by a smart binary matching machine to avoid accidents. AL-Rafdain Engineering Journal, 28(2), 261-283. https://doi.org/10.33899/rengj.2023.138781.1240.
Liang, F., Qian, C., Yu, W., Griffith, D., & Golmie, N. (2022). Survey of Graph Neural Networks and Applications. Wireless Communications and Mobile Computing, 2022. https://onlinelibrary.wiley.com/doi/epdf/10.1155/2022/9261537
Hamilton, W. L., Ying, R., & Leskovec, J. (2017).Inductive Representation Learning on Large Graphs. Advances in Neural Information Processing Systems (NeurIPS).
https://proceedings.neurips.cc/paper/2017/file/5dd9db5e033da9c6fb5ba83c7a7ebea9-Paper.pdf
Ma, G., Ahmed, N. K., Willke, T. L., & Yu, P. S. (2021). Deep graph similarity learning: A survey. Data Mining and Knowledge Discovery, 35(3), 688-725.https://link.springer.com/content/pdf/10.1007/s10618-020-00733-5.pdf.
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., ... & Leskovec, J. (2021). OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs. Proceedings of the Neural Information Processing Systems (NeurIPS) 2021 Datasets and Benchmarks Track. https://arxiv.org/pdf/2103.09430.
Hasan, S. Q. (2023). Shallow model and deep learning model for features extraction of images. NTU Journal of Engineering and Technology (NTU-JET), 2(3). https://doi.org/10.56286/ntujet.v2i3.449.
Abdulatif, F., & Ali, A. (2025). Review of classification medical images using ensemble learning. NTU Journal of Pure Sciences, 3(4). https://doi.org/10.56286/10q1f229
Li, H., & Chen, Y. (2023). A Survey on Graph Neural Networks for Knowledge Graph Completion. IEEE Transactions on Neural Networks and Learning Systems.https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10077454
Yang, Y., Liu, J., Zhao, L., & Yin, Y. (2024). Human-Computer Interaction Based on ASGCN Displacement Graph Neural Networks. Informatica, 48(1), 1-16. Retrieved from https://informatica.si.
Wang, X., He, X., Wang, M., Feng, F., & Chua, T. S. (2019). Neural graph collaborative filtering. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 165-174.https://arxiv.org/pdf/1905.08108
Gao, C., Zheng, Y., Li, N., Li, Y., Qin, Y., Piao, J., ... & others. (2023). A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems. https://dl.acm.org/doi/pdf/10.1145/3568022.
Al-Nima, R. R. O., Hasan, S. Q., & Mahmood, S. E. (2023). Utilizing fingerphotos with deep learning techniques to recognize individuals. NTU Journal of Engineering and Technology (NTU-JET), 2(1). https://doi.org/10.56286/ntujet.v2i1.318.
Mafla, A., Dey, S., Biten, A. F., Gomez, L., & Karatzas, D. (2021). Multi-modal reasoning graph for scene-text based fine-grained image classification and retrieval. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 4023–4033. https://doi.org/10.1109/WACV48630.2021.00407
Yao, L., et al. (2020). "Graph Neural Networks for Natural Language Processing." IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/TNNLS.2020.2978386.
Rizvi, S. T. A., & Imran, M. (2024). Text Classification using Graph Convolutional Networks: A Comprehensive Survey. arXiv preprint arXiv:2410.09399. https://arxiv.org/pdf/2410.09399
Yang, Y., Ren, Z., Zhang, X., Li, H., Wang, Y., & Wang, Y. (2021). Learning dynamics via graph neural networks for human pose estimation and tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, 1214–1223. https://openaccess.thecvf.com/content/CVPR2021/papers/Yang_Learning_Dynamics_via_Graph_Neural_Networks_for_Human_Pose_Estimation_CVPR_2021_paper.pdf.
Ameen, R. H. M., Basheer, N. M., & Younis, A. K. (2023). A survey: Breast cancer classification by using machine learning techniques. NTU Journal of Engineering and Technology (NTU-JET), 2(1). https://doi.org/10.56286/ntujet.v2i1.367.
Ameen, R. H. M., Basheer, N. M., & Younis, A. K. (2023). A survey: Breast cancer classification by using machine learning techniques. NTU Journal of Engineering and Technology (NTU-JET), 2(1). https://doi.org/10.56286/ntujet.v2i1.367.
Downloads
Published
License
Copyright (c) 2026 Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0).
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to Cite
