Context-Dependent Computations in Spiking Neural Networks with Apical Modulation | SpringerLink
Skip to main content
Springer Nature Link
Log in

Context-Dependent Computations in Spiking Neural Networks with Apical Modulation

  • Conference paper
  • First Online:
Artificial Neural Networks and Machine Learning – ICANN 2023 (ICANN 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14254))

Included in the following conference series:

Abstract

Neocortical pyramidal neurons integrate two distinct streams of information. Bottom-up information arrives at their basal dendrites, and resulting neuronal activity is modulated by top-down input that targets the apical tufts of these neurons and provides context information. Although this integration is essential for cortical computations, its relevance for the computations in spiking neural networks has so far not been investigated. In this article, we propose a simple spiking neuron model for pyramidal cells. The model consists of a basal and an apical compartment, where the latter modulates activity of the former in a multiplicative manner. We show that this model captures the experimentally observed properties of top-down modulated activity of cortical pyramidal neurons. We evaluated recurrently connected networks of such neurons in a series of context-dependent computation tasks. Our results show that the resulting novel spiking neural network model can significantly enhance spike-based context-dependent computations.

Supported by the European Community’s Horizon 2020 FET-Open Programme, grant number 899265, ADOPD and by the CHIST-ERA grant CHIST-ERA-18-ACAI-004, Austrian Science Fund (FWF) proj. nb. I 4670-N (project SMALL).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 9380
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 11725
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Amir, A., et al.: A low power, fully event-based gesture recognition system. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7243–7252 (2017)

    Google Scholar 

  2. Antic, S.D., Zhou, W.L., Moore, A.R., Short, S.M., Ikonomu, K.D.: The decade of the dendritic NMDA spike. J. Neurosci. Res. 88(14), 2991–3001 (2010)

    Article  Google Scholar 

  3. Asgari, H., Maybodi, B.M.N., Payvand, M., Azghadi, M.R.: Low-energy and fast spiking neural network for context-dependent learning on FPGA. IEEE Trans. Circuits Syst. II Express Briefs 67(11), 2697–2701 (2020)

    Google Scholar 

  4. Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., Maass, W.: Long short-term memory and learning-to-learn in networks of spiking neurons. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  5. Beniaguev, D., Segev, I., London, M.: Single cortical neurons as deep artificial neural networks. Neuron 109(17), 2727–2739 (2021)

    Article  Google Scholar 

  6. Chen, G., Scherr, F., Maass, W.: A data-based large-scale model for primary visual cortex enables brain-like robust and versatile visual processing. Sci. Adv. 8(44), eabq7592 (2022)

    Google Scholar 

  7. Cramer, B., Stradmann, Y., Schemmel, J., Zenke, F.: The Heidelberg spiking data sets for the systematic evaluation of spiking neural networks. IEEE Trans. Neural Netw. Learn. Syst. 33(7), 2744–2757 (2020)

    Article  Google Scholar 

  8. Davies, M., et al.: Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1), 82–99 (2018)

    Article  Google Scholar 

  9. Frenkel, C., Indiveri, G.: ReckOn: a 28 nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales. In: 2022 IEEE ISSCC, vol. 65, pp. 1–3. IEEE (2022)

    Google Scholar 

  10. Gerstner, W., Kistler, W.M., Naud, R., Paninski, L.: Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, Cambridge (2014)

    Google Scholar 

  11. Gilbert, C.D., Li, W.: Top-down influences on visual processing. Nat. Rev. Neurosci. 14(5), 350–363 (2013)

    Article  Google Scholar 

  12. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  13. Jayakumar, S.M., et al.: Multiplicative interactions and where to find them. In: ICLR (2020)

    Google Scholar 

  14. Kaiser, J., Billaudelle, S., Müller, E., Tetzlaff, C., Schemmel, J., Schmitt, S.: Emulating dendritic computing paradigms on analog neuromorphic hardware. Neuroscience 489, 290–300 (2022)

    Article  Google Scholar 

  15. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)

    Google Scholar 

  16. Kreiman, G., Serre, T.: Beyond the feedforward sweep: feedback computations in the visual cortex. Ann. N. Y. Acad. Sci. 1464(1), 222–241 (2020)

    Article  Google Scholar 

  17. Larkum, M.E., Senn, W., Lüscher, H.R.: Top-down dendritic input increases the gain of layer 5 pyramidal neurons. Cereb. Cortex 14(10), 1059–1070 (2004)

    Article  Google Scholar 

  18. Larkum, M.E., Wu, J., Duverdin, S.A., Gidon, A.: The guide to dendritic spikes of the mammalian cortex in vitro and in vivo 489, 15–33 (2022)

    Google Scholar 

  19. Liang, D., Indiveri, G.: A neuromorphic computational primitive for robust context-dependent decision making and context-dependent stochastic computation. IEEE Trans. Circuits Syst. II: Express Briefs 66(5), 843–847 (2019)

    Google Scholar 

  20. Limbacher, T., Özdenizci, O., Legenstein, R.: Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity. arXiv preprint arXiv:2205.11276 (2022)

  21. Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)

    Article  Google Scholar 

  22. Merolla, P.A., et al.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)

    Article  Google Scholar 

  23. Moradi, S., Qiao, N., Stefanini, F., Indiveri, G.: A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs). IEEE Trans. Biomed. Circuits Syst. 12(1), 106–122 (2017)

    Article  Google Scholar 

  24. Painkras, E., et al.: Spinnaker: a 1-w 18-core system-on-chip for massively-parallel neural network simulation. IEEE J. Solid-State Circuits 48(8), 1943–1953 (2013)

    Article  Google Scholar 

  25. Paugam-Moisy, H., Bohte, S.M.: Computing with spiking neuron networks. Rozenberg, G., Bäck, T., Kok, J.N. (eds.) Handbook of Natural Computing, vol. 1, pp. 1–47 (2012). https://doi.org/10.1007/978-3-540-92910-9_10

  26. Pei, J., et al.: Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature 572(7767), 106–111 (2019)

    Article  Google Scholar 

  27. Quaresima, A., Fitz, H., Duarte, R., Broek, D.V.D., Hagoort, P., Petersson, K.M.: The tripod neuron: a minimal structural reduction of the dendritic tree. J. Physiol. (2022)

    Google Scholar 

  28. Rossbroich, J., Gygax, J., Zenke, F.: Fluctuation-driven initialization for spiking neural network training. Neuromorphic Comput. Eng. 2(4), 044016 (2022)

    Article  Google Scholar 

  29. Schemmel, J., Kriener, L., Müller, P., Meier, K.: An accelerated analog neuromorphic hardware system emulating NMDA-and calcium-based non-linear dendrites. In: IJCNN, pp. 2217–2226. IEEE (2017)

    Google Scholar 

  30. Schuman, B., Dellal, S., Prönneke, A., Machold, R., Rudy, B.: Neocortical layer 1: an elegant solution to top-down and bottom-up integration. Annu. Rev. Neurosci. 44, 221–252 (2021)

    Article  Google Scholar 

  31. Shrestha, S.B., Orchard, G.: Slayer: spike layer error reassignment in time. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  32. Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., Masquelier, T., Maida, A.: Deep learning in spiking neural networks. Neural Netw. 111, 47–63 (2019)

    Article  Google Scholar 

  33. Tripathy, S.J., Savitskaya, J., Burton, S.D., Urban, N.N., Gerkin, R.C.: NeuroElectro: a window to the world’s neuron electrophysiology data. Front. Neuroinform. 8, 40 (2014)

    Article  Google Scholar 

  34. Ujfalussy, B.B., Makara, J.K., Lengyel, M., Branco, T.: Global and multiplexed dendritic computations under in vivo-like conditions. Neuron 100(3), 579–592 (2018)

    Article  Google Scholar 

  35. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  36. Wybo, W.A., Jordan, J., Ellenberger, B., Marti Mengual, U., Nevian, T., Senn, W.: Data-driven reduction of dendritic morphologies with preserved dendro-somatic responses. eLife 10, e60936 (2021)

    Article  Google Scholar 

  37. Yang, S., Wang, J., Deng, B., Azghadi, M.R., Linares-Barranco, B.: Neuromorphic context-dependent learning framework with fault-tolerant spike routing. IEEE Trans. Neural Netw. Learn. Syst. 33(12), 7126–7140 (2021)

    Article  Google Scholar 

  38. Zenke, F., Vogels, T.P.: The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks. Neural Comput. 33(4), 899–925 (2021)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Theoretical Computer Science, Graz University of Technology, Graz, Austria

    Romain Ferrand, Maximilian Baronig, Thomas Limbacher & Robert Legenstein

Authors
  1. Romain Ferrand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maximilian Baronig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Limbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Legenstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert Legenstein .

Editor information

Editors and Affiliations

  1. Democritus University of Thrace, Xanthi, Greece

    Lazaros Iliadis

  2. Democritus University of Thrace, Xanthi, Greece

    Antonios Papaleonidas

  3. Lancaster University, Lancaster, UK

    Plamen Angelov

  4. Teesside University, Middlesbrough, UK

    Chrisina Jayne

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ferrand, R., Baronig, M., Limbacher, T., Legenstein, R. (2023). Context-Dependent Computations in Spiking Neural Networks with Apical Modulation. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14254. Springer, Cham. https://doi.org/10.1007/978-3-031-44207-0_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44207-0_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44206-3

  • Online ISBN: 978-3-031-44207-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation