Long-term urban traffic flow forecasting based on feature fusion and S-T transformer

doi:10.19682/j.cnki.1005-8885.2025.0002

The Journal of China Universities of Posts and Telecommunications ›› 2025, Vol. 32 ›› Issue (1): 61-74.doi: 10.19682/j.cnki.1005-8885.2025.0002

Previous Articles Next Articles

Long-term urban traffic flow forecasting based on feature fusion and S-T transformer

School of Computer and Communication, Lanzhou University of Technology, Lanzhou 730050, China

Received:2024-03-25 Revised:2024-09-16 Online:2025-02-28 Published:2025-02-28
Contact: XIJUN ZHANG E-mail:xyy_2402@163.com
Supported by:
the National Natural Science Foundation of China;the National Natural Science Foundation of China;the Higher Educational Innovation Foundation Project of Gansu Province of China;The Key Project of Natural Science Foundation of Gansu Province;The Science and Technology Program of Gansu Province

Abstract

Abstract:

As a fundamental component of intelligent transportation systems, existing urban traffic flow forecasting models tend to overlook the spatio-temporal and long-term time-dependent patterns that characterize transportation networks. Among these, the long sequence time-series forecasting ( LSTF) model is susceptible to the issue of gradient disappearance, which can be attributed to the influence of a multitude of intricate factors. Accordingly, in this paper, the standpoint of multi-feature fusion was studied, and a traffic flow forecasting network model based on feature fusion and spatio-temporal transformer ( S-T transformer) ( STFFN) was proposed. The model combined predictive recurrent neural network ( Pred RNN) and S-T transformer to dynamically capture the spatio-temporal dependence and long-term time-dependence of traffic flow, thereby achieving a certain degree of model interpretability. A novel gated residual network-2 ( GRN-2) was proposed to investigate the potential relationship between multivariate features and target values. Furthermore, a hybrid quantile loss function was devised to alleviate the gradient disappearance in LSTF problems effectively. In extensive real experiments, the rationality and effectiveness of each network of the model were demonstrated, and the superior forecasting performance was verified in comparison to existing benchmark models.

Key words: traffic flow forecasting, multi-feature fusion, transformer, interpretability

References

[1] LIPPI M, BERTINI M, FRASCONI P. Short-term traffic flow forecasting: an experimental comparison of time-series analysis and supervised learning. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(2): 871 - 882.

[2] VLAHOGIANNI E I, KARLAFTIS M G, GOLIAS J C. Short-term traffic forecasting: where we are and where we’re going. Transportation Research Part C: Emerging Technologies, 2014, 43(1): 3 - 19.

[3] KUMAR S V, VANAJAKSHI L. Short-term traffic flow prediction using seasonal ARIMA model with limited input data. European Transport Research Review, 2015, 7: Article 21.

[4] TAN M C, WONG S C, XU J M, et al. An aggregation approach to short-term traffic flow prediction. IEEE Transactions on Intelligent Transportation Systems, 2009, 10(1): 60 - 69.

[5] CAI L R, CHEN Q, CAI W H, et al. SVRGSA: a hybrid learning based model for short-term traffic flow forecasting. IET Intelligent Transport Systems, 2019, 13(9): 1348 - 1355.

[6] ZHANG J B, ZHENG Y, QI D K. Deep spatio-temporal residual networks for citywide crowd flows prediction. Proceedings of the 31st AAAI Conference on Artificial Intelligence ( AAAI’17 ), 2017, Feb 4 - 9, San Francisco, CA, USA. Palo Alto, CA, USA: Association for the Advancement of Artificial Intelligence ( AAAI), 2017, 1655 - 1661.

[7] YU H Y, WU Z H, WANG S Q, et al. Spatiotemporal recurrent convolutional networks for traffic prediction in transportation networks. Sensors, 2017, 17(7): Article 1501.

[8] MA C X, DAI G W, ZHOU J B. Short-term traffic flow prediction for urban road sections based on time series analysis and LSTM_ BILSTM method. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 5615 - 5624.

[9] WU Y K, TAN H C. Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework. arXiv Preprint, arXiv: 1612. 01022, 2016.

[10] YUAN H T, LI G L, BAO Z F, et al. An effective joint prediction model for travel demands and traffic flows. Proceedings of the IEEE 37th International Conference on Data Engineering ( ICDE’21), 2021, Apr 19 - 22, Chania, Greece. Piscataway, NJ, USA: IEEE, 2021: 348 - 359.

[11] PAN Z Y, LIANG Y X, WANG W F, et al. Urban traffic prediction from spatio-temporal data using deep meta learning. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining ( KDD’19), 2019, Aug 4 - 8, Anchorage, AK, USA. New York, NY, USA: ACM, 2019: 1720 - 1730.

[12] SHI X L, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 28: Proceedings of the 28th International Conference on Neural Information Processing Systems ( NIPS’15 ): Vol 1, 2015, Dec 7 - 12, Montreal, Canada. Cambridge, MA, USA: MIT Press, 2015: 802 - 810.

[13] ZHANG C Y, PATRAS P. Long-term mobile traffic forecasting using deep spatio-temporal neural networks. Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing ( Mobihoc’18), 2018, Jun 26 - 29, Los Angeles, CA, USA. New York, NY, USA: ACM, 2018: 231 - 240.

[14] GREFF K, SRIVASTAVA R K, KOUTNÍK J, et al. LSTM: a search space odyssey. IEEE Transactions on Neural Networks and Learning systems, 2017, 28(10): 2222 - 2232.

[15 ] CHEN C L, LIU Y B, CHEN L, et al. Bidirectional spatial- temporal adaptive transformer for urban traffic flow forecasting. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(10): 6913 - 6925.

[16] GAL Y L, GHAHRAMANI Z. A theoretically grounded application of dropout in recurrent neural networks. Advances in Neural Information Processing Systems 29: Proceedings of the 30th International Conference on Neural Information Processing Systems ( NIPS’16), 2016, Dec 5 - 10, Barcelona, Spain. Red Hook, NY, USA: Curran Associates Inc, 2016, 1027 - 1035.

[17] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units ( ELUs). arXiv Preprint, arXiv:1511. 07289, 2015.

[18] BA J L, KIROS J R, HINTON G E. Layer normalization. arXiv Preprint, arXiv:1607. 06450, 2016.

[19] Dauphin Y N, Fan A, Auli M, et al. Language modeling with gated convolutional networks. PMLR 70: Proceedings of the 34th International Conference on Machine Learning ( ICML’17), 2017, Aug 6 - 11, Sydney, Australia. Brookline, MA, USA: Microtome Publishing, 2017: 933 - 941.

[20] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Advances in Neural Information Processing Systems 30: Proceedings of the 31st International Conference on Neural Information Processing Systems ( NIPS’17 ), 2017, Dec 4 - 9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 6000 - 6010

[21] LI S, JIN X, XUAN Y, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems 32: Proceedings of the 33rd International Conference on Neural Information Processing Systems ( NIPS’19 ), Vancouver, Canada, 2019, Dec 8 - 14, Red Hook, NY, USA: Curran Associates Inc, 2019: 5243 - 5253

[22] ZHANG X J, YU G J, SHANG J Y, et al. Short-term traffic flow prediction with residual graph attention network. Engineering Letters, 2022, 30(4): 1230 - 1236

[ 23] WANG Y B, WU H X, ZHANG J J, et al. PredRNN: a recurrent neural network for spatiotemporal predictive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2208 - 2225.

[24] YU H F, RAO N, DHILLON I S. Temporal regularized matrix factorization for high-dimensional time series prediction. Advances in Neural Information Processing Systems 29: Proceedings of the 30th International Conference on Neural Information Processing Systems ( NIPS’17), Barcelona, Spain. 2016, Dec 5 - 10, Red Hook, NY, USA: Curran Associates Inc, 2016: 847 - 85.

[25] SALINAS D, FLUNKERT V, GASTHAUS J, et al. DeepAR: probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 2020, 36(3): 1181 - 1191.

[26] LI N, ZENG X W. Time series prediction based on EMD and neural network. Journal of Xi’an University of Posts and Telecommunications, 2007, 12(1): 51 - 54 ( in Chinese).

[27] ZHANG X J, ZHANG G N, ZHANG X L, et al. Medium and long-term traffic flow prediction based on ACNN-Trans model. Medium and long-term traffic flow prediction based on ACNN- Trans model. The Journal of China Universities of Posts and Telecommunications, 2023, 30(04): 21 - 32.

[28] WANG X L, GUAN X F, CAO J, et al. Forecast network-wide traffic states for multiple steps ahead: a deep learning approach considering dynamic non-local spatial correlation and non- stationary temporal dependency. Transportation Research Part C: Emerging Technologies, 2020, 119: Article 102763.

[29 ] LI Y G, YU R, SHAHABI C, et al. Diffusion convolutional recurrent neural network: data-driven traffic forecasting. arXiv Preprint, arXiv:1707. 01926, 2017.

[30] BAI L, YAO L N, LI C, et al. Adaptive graph convolutional recurrent network for traffic forecasting. Advances in Neural Information Processing Systems 33: Proceedings of the 34th International Conference on Neural Information Processing Systems ( NIPS’20), 2020, Dec 6 - 12, Vancouver, Canada. Red Hook, NY, USA: Curran Associates Inc, 2020: 17804 - 17815.

[31] WU Z H, PAN S R, LONG G D, et al. Graph WaveNet for deep spatial-temporal graph modeling. Proceedings of the 28th International Joint Conference on Artificial Intelligence ( JCAI’19), 2019, Aug 10 - 16, Macao, China. Palo Alto, CA, USA: Association for the Advancement of Artificial Intelligence ( AAAI), 2019: 1907 - 1913

[32] YU B, YIN H T, ZHU Z X. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. Proceedings of the 27th International Joint Conference on Artificial Intelligence ( IJCAI’18), 2018, Jul 13 - 19, Stockholm, Sweden. Palo Alto, CA, USA: Association for the Advancement of Artificial Intelligence ( AAAI), 2017: 3634 - 3640.

[33] GUO S N, LIN Y F, FENG N, et al. Attention based spatial- temporal graph convolutional networks for traffic flow forecasting. Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence ( AAAI’19 / IAAI’19 / EAAI’19), 2019, Jan 27 - Feb 1, Honolulu, HI, USA. Palo Alto, CA, USA: Association for the Advancement of Artificial Intelligence ( AAAI), 2019: 922 - 929.

[34] XU M X, DAI W R, LIU C M, et al. Spatial-temporal transformer networks for traffic flow forecasting. arXiv Preprint, arXiv:2001. 02908, 2020.

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Long-term urban traffic flow forecasting based on feature fusion and S-T transformer

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended Articles

Metrics

Comments