Layer-Wise Optimization of Contextual Neural Networks with Dynamic Field of Aggregation | SpringerLink
Skip to main content
Springer Nature Link
Log in

Layer-Wise Optimization of Contextual Neural Networks with Dynamic Field of Aggregation

  • Conference paper
  • First Online:
Intelligent Information and Database Systems (ACIIDS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13758))

Included in the following conference series:

  • 780 Accesses

Abstract

This paper includes a presentation of experiments performed on Contextual Neural Networks with a dynamic field of view. It is checked how their properties can be affected by the usage of not-uniform numbers of groups in different layers of contextual neurons. Basic classification properties and activity of connections are reported based on simulations with H2O machine learning server and Generalized Backpropagation algorithm. Results are obtained for data sets with a high number of attributes (gene expression of bone marrow cancer and myeloid leukemia) as well as for standard problems from UCI Machine Learning Repository. Results indicate that layer-wise selection of numbers of connection groups can have a positive influence on the behavior of Contextual Neural Networks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 12583
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 15729
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Nasser, I.M., Abu-Naser, S.S.: Lung cancer detection using artificial neural network. Int. J. Eng. Inform. Syst. (IJEAIS) 3(3), 17–23 (2019)

    Google Scholar 

  2. Suleymanova, I., et al.: A deep convolutional neural network approach for astrocyte detection. Sci. Rep. 8(12878), 1–7 (2018)

    Google Scholar 

  3. Wang, Z.H., Horng, G.J., Hsu, T.H., Chen, C.C., Jong, G.J.: A novel facial thermal feature extraction method for non-contact healthcare system. IEEE Access 8, 86545–86553 (2020)

    Article  Google Scholar 

  4. Tsai, Y.C., et al.: FineNet: a joint convolutional and recurrent neural network model to forecast and recom-mend anomalous financial items. In: Proceedings of the 13th ACM Conference on Recommender Systems RecSys’19. ACM New York, NY, USA, pp. 536–537 (2019)

    Google Scholar 

  5. Mendez, K.M., Broadhurst, D.I., Reinke, S.N.: The application of artificial neural networks in metabolomics: a historical perspective. Metabolomics 15(11), 1–14 (2019). https://doi.org/10.1007/s11306-019-1608-0

    Article  Google Scholar 

  6. Cabrera-Vives, G., Reyes, I., Förster, F., Estévez, P.A., Maureira, J.: Supernovae detection by using convolutional neural networks. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 251–258 (2016). https://doi.org/10.1109/IJCNN.2016.7727206

  7. Rosu, C., Bacu, V.: Asteroid image classification using convolutional neural networks. In: 2021 IEEE 17th International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 3–10 (2021). https://doi.org/10.1109/ICCP53602.2021.9733484

  8. Chen, S., Zhang, S., Shang, J., Chen, B., Zheng, N.: Brain-inspired cognitive model with attention for self-driving cars. IEEE Trans. Cognitive Dev. Syst. 11(1), 13–25 (2019)

    Article  Google Scholar 

  9. Cheng, L., Jiang, F., Wang, Z., Li, J.: Multi-constrained real-time entry guidance using deep neural networks. IEEE Trans. Aerosp. Electron. Syst. 5(1), 325–340 (2021). https://doi.org/10.1109/taes.2020.3015321

    Article  Google Scholar 

  10. Cheng, L., Wang, Z., Jiang, F., Zhou, C.: Real-time optimal control for spacecraft orbit transfer via multiscale deep neural networks. IEEE Trans. Aerosp. Electron. Syst. 55(5), 2436–2450 (2019). https://doi.org/10.1109/TAES.2018.2889571

    Article  Google Scholar 

  11. Bakhadirovna, M.M., Azatovich S.M., Ulugbek Otkir Ugli, B.M.: Study of neural networks in telecommunication systems. In: 2021 International Conference on Information Science and Communications Technologies (ICISCT), pp. 1–4 (2021). https://doi.org/10.1109/ICISCT52966.2021.9670198

  12. Guest, D., Cranmer, K., Whiteson, D.: Deep learning and its application to LHC physics. Annu. Rev. Nucl. Part. Sci. 68, 1–22 (2018)

    Article  Google Scholar 

  13. Liu, L., Zheng, Y., Tang, D., Yuan, Y., Fan, C., Zhou, K.: Automatic skin binding for production characters with deep graph networks. ACM Trans. Graph. (SIGGRAPH) 38(4), 1–12 (2019)

    Article  Google Scholar 

  14. Gao, D., Li, X., Dong, Y., Peers, P., Xu, K., Tong, X.: Deep Inverse Rendering for High-resolution SVBRDF Estimation from an Arbitrary Number of Images. ACM Trans. Graph. (SIGGRAPH) 38(4), 1–15 (2019)

    Article  Google Scholar 

  15. Higgins, I., et al.: β-VAE: Learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Repr., ICLR 2017, vol. 2, no. 5, pp. 1–22 (2017)

    Google Scholar 

  16. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: International Conference on Learning Representations, ICLR 2018, pp. 1–26 (2018)

    Google Scholar 

  17. Huang, X., Tan, H., Lin, G., Tian, Y.: A LSTM-based bidirectional translation model for optimizing rare words and terminologies. In: 2018 IEEE International Conference on Artificial Intelligence and Big Data (ICAIBD). IEEE, China, pp. 5077–5086 (2018)

    Google Scholar 

  18. Athiwaratkun, B., Stokes, J.W.: Malware classification with LSTM and GRU language models and a character-level CNN. In: Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2482–2486. IEEE, USA (2017)

    Google Scholar 

  19. Gong, K., et al.: Iterative PET image reconstruction using convolutional neural network representation. IEEE Trans. Med. Imaging 38(3), 675–685 (2019)

    Article  Google Scholar 

  20. Munkhdalai, L., Park, K.-H., Batbaatar, E., Theera-Umpon, N., Ho Ryu, K.: Deep learning-based demand forecasting for Korean postal delivery service. IEEE Access 8, 188135–188145 (2020)

    Article  Google Scholar 

  21. Dozono, H., Niina, G., Araki, S.: Convolutional self organizing map. In: 2016 IEEE International Conference on Computational Science and Computational Intelligence (CSCI), IEEE, pp. 767–771 (2016)

    Google Scholar 

  22. Amato, F., et al.,: Multilayer perceptron: an intelligent model for classification and intrusion detection. In: 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA), pp. 686–691. IEEE, Taipei, Taiwan (2017)

    Google Scholar 

  23. Huk, M.: Backpropagation generalized delta rule for the selective attention Sigma-if artificial neural network. Int. J. App. Math. Comp. Sci. 22, 449–459 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Huk, M.: Notes on the generalized backpropagation algorithm for contextual neural networks with conditional aggregation functions. J. Intell. Fuzzy Syst. 32, 1365–1376 (2017)

    Article  Google Scholar 

  25. Huk, M.: Stochastic optimization of contextual neural networks with RMSprop. In: Nguyen, N.T., Jearanaitanakij, K., Selamat, A., Trawiński, B., Chittayasothorn, S. (eds.) ACIIDS 2020. LNCS (LNAI), vol. 12034, pp. 343–352. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42058-1_29

    Chapter  Google Scholar 

  26. Burnell, E.D., Wołk, K., Waliczek, K., Kern, R.: The impact of constant field of attention on properties of contextual neural networks. In: Nguyen, N.T., Trawinski, B., et al. (eds.) 12th Asian Conference on Intelligent Information and Database Systems, ACIIDS 2020, vol. 12034, pp. 364–375. LNAI, Springer (2020)

    Google Scholar 

  27. Huk, M.: Non-uniform initialization of inputs groupings in contextual neural networks. In: Nguyen, N.T., Gaol, F.L., Hong, T.-P., Trawiński, B. (eds.) ACIIDS 2019. LNCS (LNAI), vol. 11432, pp. 420–428. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14802-7_36

    Chapter  Google Scholar 

  28. Huk, M.: Training contextual neural networks with rectifier activation functions: Role and adoption of sorting methods. J. Intell. Fuzzy Syst. 37(6), 7493–7502 (2019)

    Article  Google Scholar 

  29. Huk, M.: Weights ordering during training of contextual neural networks with generalized error backpropagation: importance and selection of sorting algorithms. In: Nguyen, N.T., Hoang, D.H., Hong, T.-P., Pham, H., Trawiński, B. (eds.) ACIIDS 2018. LNCS (LNAI), vol. 10752, pp. 200–211. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75420-8_19

    Chapter  Google Scholar 

  30. Szczepanik, M., et al.: Multiple classifier error probability for multi-class problems. Eksploatacja i Niezawodnosc - Maintenance Reliab. 51(3), 12–16 (2011). https://doi.org/10.17531/ein

    Article  Google Scholar 

  31. Huk, M.: Measuring computational awareness in contextual neural networks. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, pp. 002254–002259 (2016). https://doi.org/10.1109/SMC.2016.7844574

  32. Szczepanik, M., Jóźwiak, I.: Data management for fingerprint recognition algorithm based on characteristic points’ groups. New Trends Databases Inform. Syst. Found. Comp. Decis. Sci. 38(2), 123–130 (2013)

    Google Scholar 

  33. Szczepanik, M., Jóźwiak, I.: Fingerprint recognition based on minutes groups using directing attention algorithms. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012. LNCS (LNAI), vol. 7268, pp. 347–354. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29350-4_42

    Chapter  Google Scholar 

  34. Huk, M., Pietraszko, J.: Contextual neural-network based spectrum prediction for cognitive radio. In: 4th International Conference on Future Generation Communication Technology (FGCT 2015). IEEE Computer Society, London, UK, pp. 1–5 (2015)

    Google Scholar 

  35. Kwiatkowski, J., et al.: Context-sensitive text mining with fitness leveling genetic algorithm. In: 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), Gdynia, Poland, pp. 1–6, 2015, Electronic Publication (2015). ISBN: 978-1-4799-8321-6

    Google Scholar 

  36. Huk, M.: Using context-aware environment for elderly abuse prevention. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, T.-P. (eds.) ACIIDS 2016. LNCS (LNAI), vol. 9622, pp. 567–574. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49390-8_55

    Chapter  Google Scholar 

  37. Huk, M.: Context-related data processing with artificial neural networks for higher reliability of telerehabilitation systems. In: 17th International Conference on E-health Networking, Application and Services (HealthCom), pp. 217–221. IEEE Computer Society, Boston, USA (2015)

    Google Scholar 

  38. Privitera, C.M., Azzariti, M., Stark, L.W.: Locating regions-of-interest for the Mars Rover expedition. Int. J. Remote Sensing 21, 3327–3347 (2000)

    Article  Google Scholar 

  39. UCI Machine Learning Repository: http://archive.ics.uci.edu/ml

  40. Golub, T.R., et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)

    Article  Google Scholar 

  41. Glosser, C., Piermarocchi, C., Shanker, B.: Analysis of dense quantum dot systems using a self-consistent Maxwell-Bloch framework. In: Proceedings of 2016 IEEE Int. Symposium on Antennas and Propagation (USNC-URSI), Puerto Rico, pp. 1323–1324. IEEE (2016)

    Google Scholar 

  42. Huk, M.: Measuring the effectiveness of hidden context usage by machine learning methods under conditions of increased entropy of noise. In: 2017 3rd IEEE International Con-ference on Cybernetics (CYBCONF), pp. 1–6. Exeter (2017)

    Google Scholar 

  43. Mikusova, M., et al.: Towards layer-wise optimization of contextual neural networks with constant field of aggregation. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds.) ACIIDS 2021. LNCS (LNAI), vol. 12672, pp. 743–753. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-73280-6_59

    Chapter  Google Scholar 

  44. H2O.ai documentation: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/index.html

Download references

Author information

Authors and Affiliations

  1. Wroclaw University of Science and Technology, Wroclaw, Poland

    Marcin Jodłowiec

  2. Politechnica University Timisoara, Timisoara, Romania

    Adriana Albu

  3. Esently, Rzeszów, Poland

    Krzysztof Wołk

  4. Can Tho University, Can Tho, Vietnam

    Nguyen Thai-Nghe

  5. Information Technologies Department, CERN, Geneva, Switzerland

    Adrian Karasiński

Authors
  1. Marcin Jodłowiec
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adriana Albu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Krzysztof Wołk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nguyen Thai-Nghe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adrian Karasiński
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcin Jodłowiec .

Editor information

Editors and Affiliations

  1. Wrocław University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Vietnam National University, Ho Chi Minh City, Ho Chi Minh City, Vietnam

    Tien Khoa Tran

  3. Al-Farabi Kazakh National University, Almaty, Kazakhstan

    Ualsher Tukayev

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

  5. Wrocław University of Science and Technology, Wrocław, Poland

    Bogdan Trawiński

  6. University of Newcastle, Newcastle, NSW, Australia

    Edward Szczerbicki

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jodłowiec, M., Albu, A., Wołk, K., Thai-Nghe, N., Karasiński, A. (2022). Layer-Wise Optimization of Contextual Neural Networks with Dynamic Field of Aggregation. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiński, B., Szczerbicki, E. (eds) Intelligent Information and Database Systems. ACIIDS 2022. Lecture Notes in Computer Science(), vol 13758. Springer, Cham. https://doi.org/10.1007/978-3-031-21967-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-21967-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-21966-5

  • Online ISBN: 978-3-031-21967-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation