Short-term load forecasting model based on gated recurrent unit and multi-head attention

doi:10.19682/j.cnki.1005-8885.2022.1019

中国邮电高校学报(英文) ›› 2023, Vol. 30 ›› Issue (3): 25-31.doi: 10.19682/j.cnki.1005-8885.2022.1019

• Artificial Intelligence • 上一篇下一篇

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang

1. School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China 2. School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

收稿日期:2021-09-24 修回日期:2022-04-10 出版日期:2023-06-30 发布日期:2023-06-30
通讯作者: 张玲华 E-mail:zhanglh@njupt.edu.cn
基金资助:
This work was supported by the National Natural Science Foundation of China (61771258).

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang

1. School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China 2. School of Internet of Things, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

Received:2021-09-24 Revised:2022-04-10 Online:2023-06-30 Published:2023-06-30
Contact: Zhang Linghua E-mail:zhanglh@njupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61771258).

摘要/Abstract

摘要：

Short-term load forecasting (STLF) plays a crucial role in the smart grid. However, it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load. In this paper, an STLF model based on gated recurrent unit and multi-head attention (GRU-MA) is proposed to address the aforementioned problems. The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit (GRU) and exploits multi-head attention (MA) to learn the decisive features and long-term dependencies. Additionally, the proposed model is compared with the supportvector regression (SVR) model, the recurrent neural network and multi-head attention (RNN-MA) model, the long short-term memory and multi-head attention (LSTM-MA ) model, the GRU model, and the temporal convolutional network (TCN) model using the public dataset of the Global Energy Forecasting Competition 2014 (GEFCOM2014). The results demonstrate that the GRU-MA model has the best prediction accuracy.

关键词:

deep learning, short-term load forecasting (STLF), gated recurrent unit (GRU), multi-head attention (MA)

Abstract: Short-term load forecasting (STLF) plays a crucial role in the smart grid. However, it is challenging to capture the long-time dependence and the nonlinear relationship due to the comprehensive fluctuations of the electrical load. In this paper, an STLF model based on gated recurrent unit and multi-head attention (GRU-MA) is proposed to address the aforementioned problems. The proposed model accommodates the time series and nonlinear relationship of load data through gated recurrent unit (GRU) and exploits multi-head attention (MA) to learn the decisive features and long-term dependencies. Additionally, the proposed model is compared with the supportvector regression (SVR) model, the recurrent neural network and multi-head attention (RNN-MA) model, the long short-term memory and multi-head attention (LSTM-MA ) model, the GRU model, and the temporal convolutional network (TCN) model using the public dataset of the Global Energy Forecasting Competition 2014 (GEFCOM2014). The results demonstrate that the GRU-MA model has the best prediction accuracy.

Key words: deep learning, short-term load forecasting (STLF), gated recurrent unit (GRU), multi-head attention (MA)

中图分类号:

TP183

Li Hao, Zhang Linghua, Tong Cheng, Zhou Chenyang. Short-term load forecasting model based on gated recurrent unit and multi-head attention[J]. The Journal of China Universities of Posts and Telecommunications, 2023, 30(3): 25-31.

参考文献

1. FANG X, MISRA S, XUE G L, et al. Smart grid —The new and improved power grid: A survey. IEEE Communications Surveys & Tutorials, 2012, 14(4): 944-980

2. GASPARIN A, LUKOVIC S, ALIPPI C. Deep learning for time series forecasting: The electric load case. CAAI Transactions on Intelligence Technology, 2022, 7(1): 1-25

3. HUANG C M, HUANG C J, WANG M L. A particle swarm optimization to identifying the ARMAX model for short-term load forecasting. IEEE Transactions on Power Systems, 2005, 20(2): 1126-1133

4. HAGAN M T, BEHR S M. The time series approach to short-term load forecasting. IEEE Power Engineering Review, 1987, 7(8): 56-57

5. GUO Y C, NIU D X, CHEN Y X. Support vector machine model in electricity load forecasting. Proceedings of the 2006 International Conference on Machine Learning and Cybernetics, 2006, Aug 13-16, Dalian, China. Piscataway, NJ, USA: IEEE, 2006: 2892-2896

6. BUI V, NGUYEN V H, PHAM T L, et al. RNN-based deep learning for one-hour ahead load forecasting. Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC’20), 2020, Jun 26-28, Tianjin, China. Piscataway, NJ, USA: IEEE, 2020: 587-589

7. KONG W C, DONG Z Y, JIA Y W, et al. Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Transactions on Smart Grid, 2019, 10(1): 841-851

8. GAO X Y, WANG Y, GAO Y, et al. Short-term load forecasting model of GRU network based on deep learning framework. Proceedings of the 2nd IEEE Conference on Energy Internet and Energy System Integration (EI2’18), 2018, Oct 20-22, Beijing, China. Piscataway, NJ, USA: IEEE, 2018: 4p

9. CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv Preprint, arXiv:1412.3555, 2014

10. Du H Z, Qian J G. Hierarchical gated convolutional networks with multi-head attention for text classification. Proceedings of the 5th International Conference on Systems and Informatics (ICSAI’18), 2018, Nov 10-12, Nanjing, China. Piscataway, NJ, USA: IEEE, 2018: 1170-1175

11. ZHENG J X, CHEN X Y, YU K, et al. Short-term power load forecasting of residential community based on GRU neural network. Proceedings of the 2018 International Conference on Power System Technology (POWERCON’18), 2018, Nov 6-8, Guangzhou, China. Piscataway, NJ, USA: IEEE, 2018: 4862-4868

12. BAHDANAU D, CHO K H, BENGIO Y. Neural machine translation by jointly learning to align and translate. Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), 2015, May 7-9, San Diego, CA, USA. 2015: 15p

13. WANG H M, ZHAO Y, TAN S. Short-term load forecasting of power system based on time convolutional network. Proceedings of the 8th International Symposium on Next Generation Electronics (ISNE’19), 2019, Oct 9-10, Zhengzhou, China. Piscataway, NJ, USA: IEEE, 2019: 3p

14. BIANCHI F M, MAIORINO E, KAMPFFMEYER M C, et al. Recurrent neural networks for short-term load forecasting: An overview and comparative analysis. Berlin, Germany: Springer, 2017

15. VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17), 2017, Dec 4-9, Long Beach, CA, USA. Red Hook, NY, USA: Curran Associates Inc, 2017: 6000-6010

[1]	Du Rong, Chen Shudong, Li Weiwei, Zhang Xueting, Wang Xianhui, Ge Jin. Data augmentation via joint multi-scale CNN and multi-channel attention for bumblebee image generation [J]. 中国邮电高校学报(英文版), 2023, 30(3): 32-40.
[2]	段炼唐贵进. Low-light image enhancement algorithm using a residual network with semantic information[J]. 中国邮电高校学报(英文版), 2022, 29(2): 52-62.
[3]	焦继超陈新平管孟赵亚鑫. TCL: a taxi trajectory prediction model combining time and space features [J]. 中国邮电高校学报(英文版), 2021, 28(3): 63-75.
[4]	季一木, 李可, 刘尚东, 刘强, 尧海昌, 李奎. Collaborative filtering recommendation algorithm based on interactive data classification[J]. 中国邮电高校学报(英文版), 2020, 27(5): 1-12.
[5]	杨健健张强王晓林杜毅博王超吴淼. Research on equipment fault diagnosis method based on random stochastic adaptive particle swarm optimization [J]. 中国邮电高校学报(英文版), 2020, 27(4): 17-25.
[6]	Pang Hao, Bu Yunyun, Wang Cong, Xiao Hui. Automatic detection of breast nodule in the ultrasound images using CNN[J]. 中国邮电高校学报(英文版), 2019, 26(2): 9-16.
[7]	He Jin, Wang Cong, Chen Zhao. Pulmonary tuberculosis detection model of chest X-ray images using convolutional neural network[J]. 中国邮电高校学报(英文版), 2018, 25(6): 1-6.

Short-term load forecasting model based on gated recurrent unit and multi-head attention

Short-term load forecasting model based on gated recurrent unit and multi-head attention

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价