Short-term load forecasting model based on gated recurrent unit and multi-head attention

doi:10.19682/j.cnki.1005-8885.2022.1019

The Journal of China Universities of Posts and Telecommunications ›› 2023, Vol. 30 ›› Issue (3): 25-31.doi: 10.19682/j.cnki.1005-8885.2022.1019

• Artificial intelligence • Previous Articles Next Articles

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang

1. School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China 2. School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

Received:2021-09-24 Revised:2022-04-10 Online:2023-06-30 Published:2023-06-30
Contact: Zhang Linghua E-mail:zhanglh@njupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61771258).

Abstract

Abstract: Short-term load forecasting (STLF) plays a crucial role in the smart grid. However, it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load. In this paper, an STLF model based on gated recurrent unit and multi-head attention (GRU-MA) is proposed to address the aforementioned problems. The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit (GRU) and exploits multi-head attention (MA) to learn the decisive features and long-term dependencies. Additionally, the proposed model is compared with the supportvector regression (SVR) model, the recurrent neural network and multi-head attention (RNN-MA) model, the long short-term memory and multi-head attention (LSTM-MA ) model, the GRU model, and the temporal convolutional network (TCN) model using the public dataset of the Global Energy Forecasting Competition 2014 (GEFCOM2014). The results demonstrate that the GRU-MA model has the best prediction accuracy.

Key words: deep learning, short-term load forecasting (STLF), gated recurrent unit (GRU), multi-head attention (MA)

CLC Number:

TP183

Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang. Short-term load forecasting model based on gated recurrent unit and multi-head attention[J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(3): 25-31.

References

1. FANG X, MISRA S, XUE G L, et al. Smart grid —The new and improved power grid: A survey. IEEE Communications Surveys & Tutorials, 2012, 14(4): 944-980

2. GASPARIN A, LUKOVIC S, ALIPPI C. Deep learning for time series forecasting: The electric load case. CAAI Transactions on Intelligence Technology, 2022, 7(1): 1-25

3. HUANG C M, HUANG C J, WANG M L. A particle swarm optimization to identifying the ARMAX model for short-term load forecasting. IEEE Transactions on Power Systems, 2005, 20(2): 1126-1133

4. HAGAN M T, BEHR S M. The time series approach to short-term load forecasting. IEEE Power Engineering Review, 1987, 7(8): 56-57

5. GUO Y C, NIU D X, CHEN Y X. Support vector machine model in electricity load forecasting. Proceedings of the 2006 International Conference on Machine Learning and Cybernetics, 2006, Aug 13-16, Dalian, China. Piscataway, NJ, USA: IEEE, 2006: 2892-2896

6. BUI V, NGUYEN V H, PHAM T L, et al. RNN-based deep learning for one-hour ahead load forecasting. Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC’20), 2020, Jun 26-28, Tianjin, China. Piscataway, NJ, USA: IEEE, 2020: 587-589

7. KONG W C, DONG Z Y, JIA Y W, et al. Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Transactions on Smart Grid, 2019, 10(1): 841-851

8. GAO X Y, WANG Y, GAO Y, et al. Short-term load forecasting model of GRU network based on deep learning framework. Proceedings of the 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2’18), 2018, Oct 20-22, Beijing, China. Piscataway, NJ, USA: IEEE, 2018: 4p

9. CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv Preprint, arXiv:1412.3555, 2014

10. Du H Z, Qian J G. Hierarchical gated convolutional networks with multi-head attention for text classification. Proceedings of the 5th International Conference on Systems and Informatics (ICSAI’18), 2018, Nov 10-12, Nanjing, China. Piscataway, NJ, USA: IEEE, 2018: 1170-1175

11. ZHENG J X, CHEN X Y, YU K, et al. Short-term power load forecasting of residential community based on GRU neural network. Proceedings of the 2018 International Conference on Power System Technology (POWERCON’18), 2018, Nov 6-8, Guangzhou, China. Piscataway, NJ, USA: IEEE, 2018: 4862-4868

12. BAHDANAU D, CHO K H, BENGIO Y. Neural machine translation by jointly learning to align and translate. Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), 2015, May 7-9, San Diego, CA, USA. 2015: 15p

13. WANG H M, ZHAO Y, TAN S. Short-term load forecasting of power system based on time convolutional network. Proceedings of the 8th International Symposium on Next Generation Electronics (ISNE’19), 2019, Oct 9-10, Zhengzhou, China. Piscataway, NJ, USA: IEEE, 2019: 3p

14. BIANCHI F M, MAIORINO E, KAMPFFMEYER M C, et al. Recurrent neural networks for short-term load forecasting: An overview and comparative analysis. Berlin, Germany: Springer, 2017

15. VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 6000-6010

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

[1]	Du Rong, Chen Shudong, Li Weiwei, Zhang Xueting, Wang Xianhui, Ge Jin. Data augmentation via joint multi-scale CNN and multi-channel attention for bumblebee image generation [J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(3): 32-40.
[2]	DUAN Lian. Low-light image enhancement algorithm using a residual network with semantic information [J]. The Journal of China Universities of Posts and Telecommunications, 2022, 29(2): 52-62.
[3]	. TCL: a taxi trajectory prediction model combining time and space features [J]. The Journal of China Universities of Posts and Telecommunications, 2021, 28(3): 63-75.
[4]	Ji Yimu, Li Ke, Liu Shangdong, Liu Qiang, Yao Haichang, Li Kui. Collaborative filtering recommendation algorithm based on interactive data classification [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(5): 1-12.
[5]	Yang Jianjian, Zhang Qiang, Wang Xiaolin, Du Yibo, Wang Chao, Wu Miao. Research on equipment fault diagnosis method based on random stochastic adaptive particle swarm optimization [J]. The Journal of China Universities of Posts and Telecommunications, 2020, 27(4): 17-25.
[6]	Pang Hao, Bu Yunyun, Wang Cong, Xiao Hui. Automatic detection of breast nodule in the ultrasound images using CNN [J]. The Journal of China Universities of Posts and Telecommunications, 2019, 26(2): 9-16.
[7]	He Jin, Wang Cong, Chen Zhao. Pulmonary tuberculosis detection model of chest X-ray images using convolutional neural network [J]. JOURNAL OF CHINA UNIVERSITIES OF POSTS AND TELECOM, 2018, 25(6): 1-6.

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 7

Recommended Articles

Metrics

Comments