Deep Learning with ConvNet Predicts Imagery Tasks Through EEG | Neural Processing Letters Skip to main content
Springer Nature Link
Log in

Deep Learning with ConvNet Predicts Imagery Tasks Through EEG

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Deep learning with convolutional neural networks (ConvNets) has dramatically improved the learning capabilities of computer vision applications just through considering raw data without any prior feature extraction. Nowadays, there is a rising curiosity in interpreting and analyzing electroencephalography (EEG) dynamics with ConvNets. Our study focused on ConvNets of different structures, the efficiency of multiple machine learning algorithms with optimization on ConvNets, constructing for predicting imagined left and right movements on a subject-independent basis through raw EEG data. We adapted novel lower-upper triangularization based extreme learning machines (LuELM) to the ConvNet architecture. Results showed that recently advanced methods in machine learning field, i.e. adaptive moments and batch normalization together with dropout strategy, improved ConvNets predicting ability, outperforming that of conventional fully-connected neural networks with widely-used spectral features. The proposed prediction model achieved improvements in classification performances with the rates of 90.33%, 91.00%, and 89.67% for accuracy, recall, and specificity, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Acharya UR, Oh SL, Hagiwara Y, Tan JH, Adeli H (2018) Deep convolutional neural network for the automated detection and diagnosis of seizure using EEG signals. Comp Biol Med. https://doi.org/10.1016/j.compbiomed.2017.09.017

    Article  Google Scholar 

  2. Alomari MH, Baniyounes AM, Awada EA (2014) EEG-based classification of imagined fists movements using machine learning and wavelet transform analysis. Int J Adv Electron Electr Eng 23:83–87

    Google Scholar 

  3. Altan G, Kutlu Y, Allahverdi N (2016) Deep Belief Networks Based Brain Activity Classification Using EEG from Slow Cortical Potentials in Stroke. Int J Appl Mathemat Electron Comput 4:205–205

    Article  Google Scholar 

  4. Bai L, Yu T, Li Y (2015) A brain computer interface-based explorer. J Neurosci Methods 244:2–7

    Article  Google Scholar 

  5. Carlson T, Millan JDR (2013) Brain-controlled wheelchairs: a robotic architecture. IEEE Robotics Automat Magazine 20(1):65–73

    Article  Google Scholar 

  6. Cecotti H, Gräser A (2011) Convolutional neural networks for P300 detection with application to brain-computer interfaces. IEEE Transact Pattern Anal Machine Intell. https://doi.org/10.1109/TPAMI.2010.125

    Article  Google Scholar 

  7. Cohen MX (2014) Analyzing neural time series data: theory and practice. MIT press, Cambridge

    Book  Google Scholar 

  8. Dos Santos, C., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pp. 69–78 (2014)

  9. Filho CA, Attux R, Castellano G (2017) EEG sensorimotor rhythms’ variation and functional connectivity measures during motor imagery: Linear relations and classification approaches. PeerJ. https://doi.org/10.7717/peerj.3983

    Article  Google Scholar 

  10. Fukushima K, Miyake S. (1982) Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In: Competition and cooperation in neural nets, pp. 267–285. Springer

  11. Goldberger AL, Amaral LA, Glass L, Hausdorff JM, Ivanov PC, Mark RG, Mietus JE, Moody GB, Peng CK, Stanley HE (2000) PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals. Circulation 101(23):e215–e220

    Article  Google Scholar 

  12. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press, Newyork

    MATH  Google Scholar 

  13. Gregory, K., George, K., George, H.: An EEG pre-processing technique for the fast recognition of motor imagery movements. In: 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 90–94. IEEE (2016)

  14. Alamori H, M., AbuBaker, A., Turani, A., M., A. (2014) EEG Mouse: A Machine Learning-Based Brain Computer Interface. Int J Adv Comput Sci Appl. doi.org/10.14569/ijacsa.2014.050428

  15. H., M., Samaha, A., AlKamha, K. (2013) Automated Classification of L/R Hand Movement EEG Signals using Advanced Feature Extraction and Machine Learning. International Journal of Advanced Computer Science and Applications

  16. Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep Neural Networks for Acoustic Modeling in Speech Recognition. Ieee Sig Process Magazine 29(6):82–97. https://doi.org/10.1109/MSP.2012.2205597

    Article  Google Scholar 

  17. Hinton G, Srivastava N, Swersky K (2012) Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. Cited on 14

  18. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: Theory and applications. Neurocomputing. https://doi.org/10.1016/j.neucom.2005.12.126

    Article  Google Scholar 

  19. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  20. Kim J, Kim J, Jang GJ, Lee M (2017) Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Netw. https://doi.org/10.1016/j.neunet.2016.12.002

    Article  Google Scholar 

  21. Kim Y, Ryu J, Kim KK, Took CC, Mandic DP, Park C (2016) Motor Imagery Classification Using Mu and Beta Rhythms of EEG with Strong Uncorrelating Transform Based Complex Common Spatial Patterns. Computat Intell Neurosci. https://doi.org/10.1155/2016/1489692

    Article  Google Scholar 

  22. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  23. Kolsch, A., Afzal, M.Z., Ebbecke, M., Liwicki, M.: Real-Time Document Image Classification Using Deep CNN and Extreme Learning Machines. In: Proceedings of the International Conference on Document Analysis and Recognition, ICDAR (2018). https://doi.org/10.1109/ICDAR.2017.217

  24. Kutlu Y, Yayık A, Yildirim E, Yildirim S (2019) LU triangularization extreme learning machine in EEG cognitive task classification. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3142-1

    Article  Google Scholar 

  25. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Others: Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  26. Li J, Yu ZL, Gu Z, Wu W, Li Y, Jin L (2018) A hybrid network for ERP detection and analysis based on restricted boltzmann machine. IEEE Transact Neural Syst Rehabilit Eng. https://doi.org/10.1109/TNSRE.2018.2803066

    Article  Google Scholar 

  27. Liu M, Wu W, Gu Z, Yu Z, Qi FF, Li Y (2018) Deep learning based on Batch Normalization for P300 signal detection. Neurocomputing. https://doi.org/10.1016/j.neucom.2017.08.039

    Article  Google Scholar 

  28. Ma, X., Qiu, S., Du, C., Xing, J., He, H.: Improving EEG-Based Motor Imagery Classification via Spatial and Temporal Recurrent Neural Networks. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS (2018). https://doi.org/10.1109/EMBC.2018.8512590

  29. Major TC, Conrad JM (2017) The effects of pre-filtering and individualizing components for electroencephalography neural network classification. Conf Proc IEEE SOUTHEASTCON. https://doi.org/10.1109/SECON.2017.7925289

    Article  Google Scholar 

  30. Meisheri, H., Ramrao, N., Mitra, S.: Multiclass common spatial pattern for EEG based brain computer interface with adaptive learning classifier. arXiv preprint arXiv:1802.09046 (2018)

  31. Mousavi Z, Yousefi Rezaii T, Sheykhivand S, Farzamnia A, Razavi SN (2019) Deep convolutional neural network for classification of sleep stages from single-channel EEG signals. J Neurosci Methods. https://doi.org/10.1016/j.jneumeth.2019.108312

    Article  Google Scholar 

  32. Pang S, Yang X (2016) Deep Convolutional Extreme Learning Machine and Its Application in Handwritten Digit Classification. Comput Intell Neurosci. https://doi.org/10.1155/2016/3049632

    Article  Google Scholar 

  33. Pinheiro, O.R., Alves, L.R., Romero, M.F., De Souza, J.R.: Wheelchair simulator game for training people with severe disabilities. In: TISHW 2016 - 1st International Conference on Technology and Innovation in Sports, Health and Wellbeing, Proceedings (2016). https://doi.org/10.1109/TISHW.2016.7847792

  34. Qi F, Li Y, Wu W (2015) RSTFC: A novel algorithm for spatio-temporal filtering and classification of single-trial EEG. IEEE Transact Neural Netw Learning Syst. https://doi.org/10.1109/TNNLS.2015.2402694

    Article  MathSciNet  Google Scholar 

  35. Raghu S, Sriraam N, Temel Y, Rao SV, Kubben PL (2020) EEG based multi-class seizure type classification using convolutional neural network and transfer learning. Neural Netw. https://doi.org/10.1016/j.neunet.2020.01.017

    Article  Google Scholar 

  36. San-Segundo R, Gil-Martín M, D’Haro-Enríquez LF, Pardo JM (2019) Classification of epileptic EEG recordings using signal transforms and convolutional neural networks. Comput Biol Med. https://doi.org/10.1016/j.compbiomed.2019.04.031

    Article  Google Scholar 

  37. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR (2004) BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Transact Biomed Eng 51(6):1034–1043

    Article  Google Scholar 

  38. Schirrmeister RT, Springenberg JT, Fiederer LDJ, Glasstetter M, Eggensperger K, Tangermann M, Hutter F, Burgard W, Ball T (2017) Deep learning with convolutional neural networks for EEG decoding and visualization. Human Brain Mapp. https://doi.org/10.1002/hbm.23730

    Article  Google Scholar 

  39. Shenoy, H.V., Vinod, A.P., Guan, C.: Shrinkage estimator based regularization for EEG motor imagery classification. In: 2015 10th International Conference on Information, Communications and Signal Processing, ICICS 2015 (2016). https://doi.org/10.1109/ICICS.2015.7459836

  40. Sita, J., Nair, G.J.: Feature extraction and classification of EEG signals for mapping motor area of the brain. In: 2013 International Conference on Control Communication and Computing, ICCC 2013 (2013). https://doi.org/10.1109/ICCC.2013.6731699

  41. Sleight J, Pillai P, Mohan S (2009) Classification of executed and imagined motor movement EEG signals. University of Michigan, Ann Arbor Michigan, pp 1–10

    Google Scholar 

  42. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Machine Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  43. Sun Y, Lo FP, Lo B (2019) EEG-based user identification system using 1D-convolutional long short-term memory neural networks. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.01.080

    Article  Google Scholar 

  44. Wang YT, Wang Y, Jung TP (2011) A cell-phone-based brain-computer interface for communication in daily life. Journal of neural engineering 8(2):25,018

  45. Welch P (1967) The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Transact Audio Electroacoustics 15(2):70–73

    Article  Google Scholar 

  46. Wu W, Chen Z, Gao X, Li Y, Brown EN, Gao S (2015) Probabilistic common spatial patterns for multichannel EEG analysis. IEEE Transact Pattern Anal Machine Intell. https://doi.org/10.1109/TPAMI.2014.2330598

    Article  Google Scholar 

  47. Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing. https://doi.org/10.1016/j.neucom.2014.03.077

    Article  Google Scholar 

  48. Zakaria, M.H.F., Mansor, W., Lee, K.Y.: Time-frequency analysis of executed and imagined motor movement EEG signals for neuro-based home appliance system. In: TENCON 2017-2017 IEEE Region 10 Conference, pp. 1657–1660. IEEE (2017)

  49. Zeng, Y., Xu, X., Fang, Y., Zhao, K.: Traffic sign recognition using extreme learning classifier with deep convolutional features. The 2015 international conference on intelligence science and big data engineering (IScIDE 2015), Suzhou, China (2015). https://doi.org/10.1007/978-3-319-23989-7_28

  50. Zhang, D., Yao, L., Zhang, X., Wang, S., Chen, W., Boots, R.: EEG-based intention recognition from spatio-temporal representations via cascade and parallel convolutional recurrent neural networks. arXiv preprint arXiv:1708.06578 (2017)

  51. Zhang J, Yao R, Ge W, Gao J (2020) Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG. Comp Methods Programs Biomed. https://doi.org/10.1016/j.cmpb.2019.105089

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Iskenderun Technical University, Hatay, Turkey

    Gokhan Altan & Yakup Kutlu

  2. Huawei R&D Center, Istanbul, Turkey

    Apdullah Yayık

Authors
  1. Gokhan Altan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Apdullah Yayık
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yakup Kutlu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gokhan Altan.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Altan, G., Yayık, A. & Kutlu, Y. Deep Learning with ConvNet Predicts Imagery Tasks Through EEG. Neural Process Lett 53, 2917–2932 (2021). https://doi.org/10.1007/s11063-021-10533-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10533-7

Keywords

Search

Navigation