Analysis of prediction performance in wavelet minimum 
complexity echo state network

doi:10.1016/S1005-8885(13)60070-9

中国邮电高校学报(英文) ›› 2013, Vol. 20 ›› Issue (4): 59-66.doi: 10.1016/S1005-8885(13)60070-9

Analysis of prediction performance in wavelet minimum complexity echo state network

冯辰,崔鸿雁,刘韵洁

1. Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China 2. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

收稿日期:2012-05-07 修回日期:2012-09-14 出版日期:2013-08-31 发布日期:2013-08-30
通讯作者: 崔鸿雁 E-mail:cuihy@bupt.edu.cn
基金资助:
This work was supported by the National Natural Science Foundation of China (61201153), the National Basic Research Program of China (2012CB315805), and the National Key Science and Technology Projects (2010ZX03004-002-02).

Analysis of prediction performance in wavelet minimum complexity echo state network

1. Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China 2. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2012-05-07 Revised:2012-09-14 Online:2013-08-31 Published:2013-08-30
Contact: Cui Hongyan E-mail:cuihy@bupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61201153), the National Basic Research Program of China (2012CB315805), and the National Key Science and Technology Projects (2010ZX03004-002-02).

摘要/Abstract

摘要： Echo state network (ESN) has become one of the most popular recurrent neural networks (RNN) for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model (MCESN) was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model (WMCESN) to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol (TCP) and user datagram protocol (UDP) dataset to prove that our model can deal with the data packet bit prediction problem well.

关键词: wavelet minimum complexity echo state network, echo state network, wavelet parameter matrix, practical applicability

Abstract: Echo state network (ESN) has become one of the most popular recurrent neural networks (RNN) for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model (MCESN) was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model (WMCESN) to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol (TCP) and user datagram protocol (UDP) dataset to prove that our model can deal with the data packet bit prediction problem well.

Key words: wavelet minimum complexity echo state network, echo state network, wavelet parameter matrix, practical applicability

CUI Hong-yan, FENG Chen, LIU Yun-jie. Analysis of prediction performance in wavelet minimum complexity echo state network [J]. Acta Metallurgica Sinica(English letters), 2013, 20(4): 59-66.

参考文献

1. Jaeger H. The ‘echo state’ approach to analyzing and training recurrent neural networks—with an Erratum note. GMD Tech Rep 148. St Augustin, Germany: German National Research Center for Information Technology, 2001

2. Jaeger H. Short term memory in echo state networks. GMD Tech Rep 152. St Augustin, Germany : German National Research Center for Information Technology, 2001

3. Jaeger H. A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ‘echo state network’ approach. GMD Tech Rep 159. St Augustin, Germany: German National Research Center for Information Technology, 2002

4. Skowronski M D, Harris J G. Noise-robust automatic speech recognition using a predictive echo state network. IEEE Transactions on Audio, Speech, and Language Processing, 2007, 15(5): 1724-1730

5. Tong M H, Bickett A D, Christiansen E M, et al. Learning grammatical structure with echo state networks. IEEE Transactions on Neural Networks, 2007, 20(3): 424-432

6. Lin X W, Yang Z H, Song Y X. Short-term stock price prediction based on echo state networks. Expert Systems with Applications, 2009, 26(3): 7313-7317

7. Rodan A, Tino P. Minimum complexity echo state network. IEEE Transactions on Neural Networks, 2011, 22(1): 131-144

8. Zhang Q, Benveniste A. Wavelet networks. IEEE Transactions on Neural Networks, 1992, 3(6): 889-898

9. Zhang J, Walter G G, Miao Y. Wavelet neural networks for function learning. IEEE Transactions on Signal Processing, 1995, 43(6): 1485-1497

10. Lu C H. Wavelet fuzzy neural networks for identi?cation and predictive control of dynamic systems. IEEE Transactions on Industrial Electronics, 2011, 58(7): 3046-3057

11. Zhang Q. Using wavelet network in nonparametric estimation. IEEE Transactions on Neural Network, 1997, 8(2): 227-236

12. Pati Y C, Krishnaprasad P S. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks, 1993, 4(1): 73-85

13. Wang S, Yang X J, Wei C J. Harnessing non-linearity by sigmoid-wavelet hybrid echo state networks (SWHESN). Proceedings of the 6th World Congress on Intelligent Control and Automation (WCICA’06), Jun 21-23, 2006, Dalian, China. Piscataway, NJ, USA: IEEE, 2006: 3014-3018

14. Jaeger H, Lukosevicius M, Popovici D, et al. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks, 2007, 20(3): 335-352

15. Henon M. A two-dimensional mapping with a strange attractor. Communications in Mathematical Physics, 1976, 50(1): 69-77

16. Slutzky M, Cvitanovic P, Mogul D. Manipulating epileptiform bursting in the rat hippocampus using chaos control and adaptive techniques. IEEE Transactions on Biomedical Engineering, 2003, 50(5): 559-570

17. Xue Y, Yang L, Haykin S. Decoupled echo state networks with lateral inhibition. Neural Networks, 2007, 20(3): 365-376

18. Liu X, Cui H Y, Zhou T J, et al. Performance evaluation of new echo state networks based on complex network. The Journal of China Universities of Posts and Telecommunications, 2011, 19(1): 87-93

19. Cui H Y, Liu X, Li L X. The architecture of dynamic reservoir in echo state network, chaos 22(3), 033127 (2012). http://dx.doi.org/10.1063/1.4746765

Analysis of prediction performance in wavelet minimum complexity echo state network

Analysis of prediction performance in wavelet minimum complexity echo state network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

编辑推荐

Metrics

本文评价