Analysis of prediction performance in wavelet minimum 
complexity echo state network

doi:10.1016/S1005-8885(13)60070-9

Acta Metallurgica Sinica(English letters) ›› 2013, Vol. 20 ›› Issue (4): 59-66.doi: 10.1016/S1005-8885(13)60070-9

• Networks • Previous Articles Next Articles

Analysis of prediction performance in wavelet minimum complexity echo state network

1. Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China 2. School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2012-05-07 Revised:2012-09-14 Online:2013-08-31 Published:2013-08-30
Contact: Cui Hongyan E-mail:cuihy@bupt.edu.cn
Supported by:
This work was supported by the National Natural Science Foundation of China (61201153), the National Basic Research Program of China (2012CB315805), and the National Key Science and Technology Projects (2010ZX03004-002-02).

Abstract

Abstract: Echo state network (ESN) has become one of the most popular recurrent neural networks (RNN) for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model (MCESN) was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model (WMCESN) to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol (TCP) and user datagram protocol (UDP) dataset to prove that our model can deal with the data packet bit prediction problem well.

Key words: wavelet minimum complexity echo state network, echo state network, wavelet parameter matrix, practical applicability

CUI Hong-yan, FENG Chen, LIU Yun-jie. Analysis of prediction performance in wavelet minimum complexity echo state network [J]. Acta Metallurgica Sinica(English letters), 2013, 20(4): 59-66.

References

1. Jaeger H. The ‘echo state’ approach to analyzing and training recurrent neural networks—with an Erratum note. GMD Tech Rep 148. St Augustin, Germany: German National Research Center for Information Technology, 2001

2. Jaeger H. Short term memory in echo state networks. GMD Tech Rep 152. St Augustin, Germany : German National Research Center for Information Technology, 2001

3. Jaeger H. A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the ‘echo state network’ approach. GMD Tech Rep 159. St Augustin, Germany: German National Research Center for Information Technology, 2002

4. Skowronski M D, Harris J G. Noise-robust automatic speech recognition using a predictive echo state network. IEEE Transactions on Audio, Speech, and Language Processing, 2007, 15(5): 1724-1730

5. Tong M H, Bickett A D, Christiansen E M, et al. Learning grammatical structure with echo state networks. IEEE Transactions on Neural Networks, 2007, 20(3): 424-432

6. Lin X W, Yang Z H, Song Y X. Short-term stock price prediction based on echo state networks. Expert Systems with Applications, 2009, 26(3): 7313-7317

7. Rodan A, Tino P. Minimum complexity echo state network. IEEE Transactions on Neural Networks, 2011, 22(1): 131-144

8. Zhang Q, Benveniste A. Wavelet networks. IEEE Transactions on Neural Networks, 1992, 3(6): 889-898

9. Zhang J, Walter G G, Miao Y. Wavelet neural networks for function learning. IEEE Transactions on Signal Processing, 1995, 43(6): 1485-1497

10. Lu C H. Wavelet fuzzy neural networks for identi?cation and predictive control of dynamic systems. IEEE Transactions on Industrial Electronics, 2011, 58(7): 3046-3057

11. Zhang Q. Using wavelet network in nonparametric estimation. IEEE Transactions on Neural Network, 1997, 8(2): 227-236

12. Pati Y C, Krishnaprasad P S. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks, 1993, 4(1): 73-85

13. Wang S, Yang X J, Wei C J. Harnessing non-linearity by sigmoid-wavelet hybrid echo state networks (SWHESN). Proceedings of the 6th World Congress on Intelligent Control and Automation (WCICA’06), Jun 21-23, 2006, Dalian, China. Piscataway, NJ, USA: IEEE, 2006: 3014-3018

14. Jaeger H, Lukosevicius M, Popovici D, et al. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks, 2007, 20(3): 335-352

15. Henon M. A two-dimensional mapping with a strange attractor. Communications in Mathematical Physics, 1976, 50(1): 69-77

16. Slutzky M, Cvitanovic P, Mogul D. Manipulating epileptiform bursting in the rat hippocampus using chaos control and adaptive techniques. IEEE Transactions on Biomedical Engineering, 2003, 50(5): 559-570

17. Xue Y, Yang L, Haykin S. Decoupled echo state networks with lateral inhibition. Neural Networks, 2007, 20(3): 365-376

18. Liu X, Cui H Y, Zhou T J, et al. Performance evaluation of new echo state networks based on complex network. The Journal of China Universities of Posts and Telecommunications, 2011, 19(1): 87-93

19. Cui H Y, Liu X, Li L X. The architecture of dynamic reservoir in echo state network, chaos 22(3), 033127 (2012). http://dx.doi.org/10.1063/1.4746765

Metrics

Comments

Copyright © 2020 The Journal of China Universities of Posts and Telecommunications
　 Adress: P.O. Box 231,Beijing University of Posts and Telecommunications,10 Xi Tucheng Road,Beijing 100876,P.R.China　Post Code: 100081
Tel：86-010-62282493　Fax： 86-010-62283461　E-mail: jchupt@bupt.edu.cn
Support by: Beijing Magtech Co.Ltd

Analysis of prediction performance in wavelet minimum complexity echo state network

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 0

Recommended Articles

Metrics

Comments