A minimum complexity interaction echo state network | Neural Computing and Applications Skip to main content

Advertisement

Springer Nature Link
Log in

A minimum complexity interaction echo state network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Simple cycle reservoir is a classic work in reservoir structure design, and has good performance in tasks such as discrete dynamical system prediction and time series classification. However, the overly simple reservoir structure weakens its ability to model the complex systems such as chaotic systems. A minimum complexity interaction echo state network (MCI-ESN) is proposed in this paper to overcome the shortcomings of simple cycle reservoir. MCI-ESN consists of two identical simple cycle reservoirs which are interconnected by only two neurons for reducing the connection redundancy and improve connection efficiency. A sufficient condition is given to guarantee that the MCI-ESN model has the echo state property. Several numerical experiments, including multivariable chaotic time series prediction and time series classification, are used to verify the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Notes

  1. Publicly available at https://www.physionet.org/physiobank/database/mitdb/.

References

  1. Jaeger H (2001) The “echo state” approach to analysing and training recurrent neural networks. German National Research Center for Information Technology, GMD Report 148

  2. Maass W, Natschläger T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14:2531–2560

    Article  PubMed  Google Scholar 

  3. Jaeger H, Hass H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communications. Science 304(5667):78–80

    Article  ADS  CAS  PubMed  Google Scholar 

  4. Appeltant L, Soriano M, Van der Sande G et al (2011) Information processing using a single dynamical node as complex system. Nat Commun 2:468

    Article  ADS  CAS  PubMed  Google Scholar 

  5. Gauthier DJ, Bollt E, Griffith A, Barbosa W (2021) Next generation reservoir computing. Nature Commun 12:5564

    Article  ADS  CAS  Google Scholar 

  6. Pearlmutter BA (1995) Gradient calculations for dynamic recurrent neural networks: a survey. IEEE Trans Neural Netw 6(5):1212–1228

    Article  CAS  PubMed  Google Scholar 

  7. Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: Proceedings of the 30th international conference on machine learning, pp. 1310–1318

  8. Lun SX, Yao XS, Qi HY, Hu HF (2015) A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing 159:58–66

    Article  Google Scholar 

  9. Han M, Xu ML (2018) Laplacian echo state network for multivariate time series prediction. IEEE Trans Neural Netw Learn Syst 29:238–244

    Article  MathSciNet  PubMed  Google Scholar 

  10. Yang CL, Qiao JF, Wang L, Zhu XX (2019) Dynamical regularized echo state network for time series prediction. Neural Comput Appl 31:6781–6794

    Article  Google Scholar 

  11. Wang J, Su Z, Qiao JF, Yang CL (2021) Design of sparse Bayesian echo state network for time series prediction. Neural Comput Appl 33:7089–7102

    Article  Google Scholar 

  12. Pan Y, Wang J (2012) Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks. IEEE Trans Industr Electron 59(8):3089–3101

    Article  Google Scholar 

  13. Skowronski MD, Harris JG (2007) Noise-robust automatic speech recognition using a predictive echo state network. IEEE Trans Audio Speech Lang Process 15:1724–1730

    Article  Google Scholar 

  14. Duan HB, Wang XH (2016) Echo state networks with orthogonal pigeon-inspired optimization for image restoration. IEEE Trans Neural Netw Learn Syst 27:2413–2425

    Article  MathSciNet  PubMed  Google Scholar 

  15. Ozturk MC, Xu D, Príncipe JC (2007) Analysis and design of echo state networks. Neural Comput 19:111–138

    Article  PubMed  Google Scholar 

  16. Strauss T, Wustlich W, Labahn R (2012) Design strategies for weight matrices of echo state networks. Neural Comput 24(12):3246–3276

    Article  MathSciNet  PubMed  Google Scholar 

  17. Rodan A, Tino P (2011) Minimum complexity echo state network. IEEE Trans Neural Netw 22(1):131–144

    Article  PubMed  Google Scholar 

  18. Rodan A, Tino P (2012) Simple deterministically constructed cycle reservoirs with regular jumps. Neural Comput 24(7):1822–1852

    Article  MathSciNet  PubMed  Google Scholar 

  19. Bianchi FM, Scardapane S, Lkse S, Jenssen R (2021) Reservoir computing approaches for representation and classification of multivariate time series. IEEE Trans Neural Netw Learn Syst 32(5):2169–2179

    Article  PubMed  Google Scholar 

  20. Zhang H, Zheng M, Zhang Y et al (2021) Application of ESN prediction model based on compressed sensing in stock market. Commun Nonlinear Sci Numer Simul 101:105857

    Article  MathSciNet  Google Scholar 

  21. Wang ZJ, Zhao H, Zheng MW et al (2023) A novel time series prediction method based on pooling compressed sensing echo state network and its application in stock market. Neural Netw 164:216–227

    Article  PubMed  Google Scholar 

  22. Jordanou JP, Antonelo EA, Camponogara E, Gildin E (2023) Investigation of proper orthogonal decomposition for echo state networks. Neurocomputing 548:126395

    Article  Google Scholar 

  23. Xue L, Yang L, Haykin S (2007) Decoupled echo state networks with lateral inhibition. Neural Netw 20(3):365–376

    Article  PubMed  Google Scholar 

  24. Qiao J, Li F, Han H et al (2016) Growing echo-state network with multiple subreservoirs. IEEE Trans Neural Netw Learn Syst 28(2):1–14

    Google Scholar 

  25. Li Q, Wu Z, Ling R et al (2020) Multi-reservoir echo state computing for solar irradiance prediction: a fast yet efficient deep learning approach. Appl Soft Comput 95(10648):1

    CAS  Google Scholar 

  26. Gallicchio C, Micheli A, Pedrelli L (2017) Deep reservoir computing: a critical experimental analysis. Neurocomputing 268(11):87–99

    Article  Google Scholar 

  27. Gallicchio C, Micheli A, Pedrelli L (2018) Design of deep echo state networks. Neural Netw 108:33–47

    Article  PubMed  Google Scholar 

  28. Gao Z (2011) A novel four-dimensional hyperchaotic system and its analysis. J Wuhan Univ 57(3):201–204

    MathSciNet  Google Scholar 

  29. Lukosevicius M, Jaeger H, Schrauwen B (2012) Reservoir computing trends. KI Kunstl Intell 26(4):365–371

    Article  Google Scholar 

  30. Moody GB, Mark RG (2001) The impact of the MIT-BIH arrhythmia database. IEEE Eng Med Biol Mag 20(3):45–50

    Article  CAS  PubMed  Google Scholar 

  31. Lukoeviius M, Uselis A (2020) Efficient implementations of echo state network cross-validation. Cogn Comput. https://doi.org/10.1007/s12559-021-09849-2

    Article  Google Scholar 

  32. Chen H, Tino P, Rodan A, Yao X (2014) Learning in the model space for cognitive fault diagnosis. IEEE Trans Neural Netw Learn Syst 25(1):124–136

    Article  PubMed  Google Scholar 

  33. Juang CF, Yeh YT (2017) Multiobjective evolution of biped robot gaits using advanced continuous ant-colony optimized recurrent neural networks. IEEE Trans Cybern 48(6):1910–1922

    Article  PubMed  Google Scholar 

  34. Wiens J, Horvitz E, Guttag JV (2012) Patient risk stratification for hospital-associated C. diff as a time-series classification task. In: Advances in neural information processing systems, pp 467–475

  35. Delgado M, Cuellar MP, Pegalajar MC (2008) Multiobjective hybrid optimization and training of recurrent neural networks. IEEE Trans Syst Man Cybern Part B-Cybern 38(2):381–403

    Article  Google Scholar 

  36. Bianchi F M, Scardapane S, Lkse S, Jenssen R (2018) Bidirectional deep-readout echo state networks. In: Proceedings of ESANN

  37. Gong Z, Chen H, Yuan B, Yao X (2019) Multiobjective learning in the model space for time series classification. IEEE Trans Cybern 49(3):918–932

    Article  PubMed  Google Scholar 

  38. Chen Y, Keogh E, Hu B, Begum N, Bagnall A, Mueen A, Batista G (2015) The UCR time series classification archive. www.cs.ucr.edu/∼eamonn/ time series data/

  39. Aswolinskiy W, René FR, Steil J (2016) Time series classification in reservoir- and model-space: a comparison. Artif Neural Netw Pattern Recognit 9896:197–208

    Article  Google Scholar 

  40. Baydogan MG, Runger G (2015) Time series representation and similarity based on local autopatterns. Data Min Knowl Disc 30:476–509

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Scientific Foundation of China (NSFC) under grant number 12072128.

Funding

This work was supported by the National Natural Scientific Foundation of China (NSFC) under Grant Number 12072128.

Author information

Authors and Affiliations

  1. College of Mathematics, Jilin University, 2699 Qianjin Street, Changchun, 130012, People’s Republic of China

    Jianming Liu & Xu Xu

  2. School of Science, Engineering & Design, Teesside University, Middleborough, UK

    Eric Li

Authors
  1. Jianming Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JL: Conceptualization, Methodology, Writing-Original draft preparation, Software, Editing; XX: Conceptualization, Methodology, Writing-Original draft preparation, Software, Editing; EL: Writing-Reviewing, Validation. All the authors approved the final article.

Corresponding author

Correspondence to Xu Xu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, J., Xu, X. & Li, E. A minimum complexity interaction echo state network. Neural Comput & Applic 36, 4013–4026 (2024). https://doi.org/10.1007/s00521-023-09271-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-09271-9

Keywords

Search

Navigation