Implementation and evaluation of a container management platform on Docker: Hadoop deployment as an example | Cluster Computing Skip to main content

Advertisement

Springer Nature Link
Log in

Implementation and evaluation of a container management platform on Docker: Hadoop deployment as an example

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In recent years, virtualization is one of the key technologies of next-generation data centers. However, the problem of virtualization technology is that each instance needs to run a client operating system and a lot of applications. Therefore, it might generate a heavy load and affect the system efficiency and performance. In this work, the performance evaluation of three environments (bare-metal, Docker containers, and virtual machines) is investigated to understand the differences between the characteristics of each environment. Also, we addressed whether container-based virtualization can solve the problems of traditional virtualization. In addition, we combined Docker with OpenStack to implement a container management platform. Finally, we took Hadoop deployment as an example to verify whether Docker can solve the deployment problem and save time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Buyya, R., Vecchiola, C., Thamarai Selvi, S.: Chapter 3: virtualization. In: Mastering Cloud Computing. Morgan Kaufmann, Burlington (2013)

  2. Liao, X., Jin, H., Yu, S., Zhang, Y.: A novel memory allocation scheme for memory energy reduction in virtualization environment. J. Comput. Syst. Sci. 81(1), 3–15 (2015)

    Article  MathSciNet  Google Scholar 

  3. Dong, Y., Zhang, X., Dai, J., Guan, H.: HYVI: a hybrid virtualization solution balancing performance and manageability. IEEE Trans. Parallel Distrib. Syst. 25(9), 2332–2341 (2014)

    Article  Google Scholar 

  4. Pfaff, B., Pettit, J., Koponen, T. Shenker, S.: Extending networking into the virtualization layer. In: 8th ACM Workshop on Hot Topics in Networks (HotNets-VIII), New York City, NY, October 2009 (2009)

  5. Yang, C.-T., Liu, J.-C., Chen, S.-T., Huang, K.-L.: Virtual machine management system based on the power saving algorithm in cloud. J. Netw. Comput. Appl. 80, 165–180 (2017)

    Article  Google Scholar 

  6. Yang, C.-T., Chen, S.-T., Liu, J.-C., Chan, Y.-W., Chen, C.-C., Verma, V.K.: An energy-efficient cloud system with novel dynamic resource allocation methods. J. Supercomput. 75(8), 4408–4429 (2019)

    Article  Google Scholar 

  7. Yang, C.-T., Wan, T.-Y.: Implementation of an energy saving cloud infrastructure with virtual machine power usage monitoring and live migration on OpenStack. Computing 102(6), 1547–1566 (2020)

    Article  Google Scholar 

  8. Špaček, F., Sohlich, R., Dulík, T.: Docker as platform for assignments evaluation. Energy Procedia 100, 1665–1671 (2015)

    Article  Google Scholar 

  9. Liu, D., Zhao, L.: The research and implementation of cloud computing platform based on Docker. In: 2014 11th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 475–478 (2014)

  10. Felter, W., Ferreira, A., Rajamony, R., Rubio, J.: An updated performance comparison of virtual machines and Linux containers. In: 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 171–172 (2015)

  11. Nakagawa, G., Oikawa, S.: Behavior-based memory resource management for container-based virtualization. In: Proceedings—4th International Conference on Applied Computing and Information Technology, 3rd International Conference on Computational Science/Intelligence and Applied Informatics, 1st International Conference on Big Data, Cloud Computing, Data Science and Engineering, ACIT-CSII-BCD 2016, pp. 213–217 (2017)

  12. Soltesz, S., Pötzl, H., Fiuczynski, M.E., Bavier, A., Peterson, L.: Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In: Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, pp. 275–287 (2007)

  13. Saraladevi, B., Pazhaniraja, N., Victer Paul, P., Saleem Basha, M.S., Dhavachelvan, P.: Big data and Hadoop—a study in security perspective. Procedia Comput. Sci. 50, 598–601 (2015)

    Article  Google Scholar 

  14. Kačeniauskas, A., Pacevič, R., Starikovičius, V., Maknickas, A., Staškūnienė, M., Davidavičius, G.: Development of cloud services for patient-specific simulations of blood flows through aortic valves. Adv. Eng. Softw. 103, 57–64 (2017)

    Article  Google Scholar 

  15. Jlassi, A., Martineau, P.: Benchmarking Hadoop performance in the cloud—an in depth study of resource management and energy consumption. In: The 6th International Conference on Cloud Computing and Services Science, Rome, Italy, (2016)

  16. Li, Z., Li, H., Wang, X., Li, K.: A generic cloud platform for engineering optimization based on OpenStack. Adv. Eng. Softw. 75, 42–57 (2014)

    Article  Google Scholar 

  17. Yamato, Y., Muroi, M., Tanaka, K., Uchimura, M.: Development of template management technology for easy deployment of virtual resources on OpenStack. J. Cloud Comput. 3(1), 1–12 (2014)

    Article  Google Scholar 

  18. Yamato, Y., Nishizawa, Y., Muroi, M., Tanaka, K.: Development of resource management server for production IaaS services based on OpenStack. J. Inf. Process. 23(1), 58–66 (2015)

    Google Scholar 

  19. Watada, J., Roy, A., Kadikar, R., Pham, H., Xu, B.: Emerging trends, techniques and open issues of containerization: a review. IEEE Access 7, 152443–152472 (2019)

    Article  Google Scholar 

  20. Pahl, C., Brogi, A., Soldani, J., Jamshidi, P.: Cloud container technologies: a state-of-the-art review. IEEE Trans. Cloud Comput. 7(3), 677–692 (2019)

    Article  Google Scholar 

  21. Potdar, A.M., Narayan, D.G., Kengond, S., Mulla, M.M.: Performance evaluation of Docker container and virtual machine. Procedia Comput. Sci. 171, 1419–1428 (2000)

    Article  Google Scholar 

  22. Yadav, R.R., Sousa, E.T.G., Callou, G.R.A.: Performance comparison between virtual machines and Docker containers. IEEE Lat. Am. Trans. 16(8), 2282–2288 (2018)

    Article  Google Scholar 

  23. Lingayat, A., Badre, R.R., Gupta, A.K.: Performance evaluation for deploying Docker containers on baremetal and virtual machine. In: 2018 3rd International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, pp. 1019–1023 (2018)

  24. Zhang, Q., Liu, L, Pu, C., Dou, Q., Wu, L., Zhou, W.: A comparative study of containers and virtual machines in Big Data environment. In: IEEE CLOUD, pp. 178–185 (2018)

  25. Shirinbab, S., Lundberg, L., Casalicchio, E.: Performance evaluation of container and virtual machine running cassandra workload. In: 3rd International Conference of Cloud Computing Technologies and Applications (CloudTech), Rabat, 2017, pp. 1–8 (2017)

  26. Barik, R.K., Lenka, R.K., Rao, K.R., Ghose, D.: Performance analysis of virtual machines and containers in cloud computing. In: 2016 International Conference on Computing, Communication and Automation (ICCCA), Noida, pp. 1204–1210 (2016)

  27. Tay, Y.C., Gaurav, K., Karkun, P.: A performance comparison of containers and virtual machines in workload migration context. In: 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW), Atlanta, GA, pp. 61–66 (2017)

  28. Salah, T., Zemerly, M.J., Yeun, C.Y., Al-Qutayri, M., Al-Hammadi, Y.: Performance comparison between container-based and VM-based services. In: 2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), Paris, pp. 185–190 (2017)

Download references

Acknowledgements

This work was supported by the Ministry of Science and Technology (MOST), Taiwan, under Grant Nos. 110-2622-E-029-003-3 and 109-2221-E-029-020. In addition, this work was also funded in part by The National Applied Research Laboratories (NARLabs), Taiwan, under Grant No. 03108F1106 and 03109F1106. We are grateful to the National Center for High-performance Computing for computer time and facilities.

Author information

Authors and Affiliations

  1. Department of M-Commerce and Multimedia Applications, Asia University, Taichung, 41354, Taiwan, ROC

    Wen-Chung Shih

  2. Department of Computer Science, Tunghai University, No. 1727, Sec. 4, Taiwan Boulevard, Taichung, 407224, Taiwan, ROC

    Chao-Tung Yang & Chun-I Chiang

  3. Research Center for Smart Sustainable Circular Economy, Tunghai University, No. 1727, Sec. 4, Taiwan Boulevard, Taichung, 407224, Taiwan, ROC

    Chao-Tung Yang

  4. School of Computing Science, Newcastle University, Newcastle, UK

    Rajiv Ranjan

Authors
  1. Wen-Chung Shih
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Chao-Tung Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Rajiv Ranjan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Chun-I Chiang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Chao-Tung Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shih, WC., Yang, CT., Ranjan, R. et al. Implementation and evaluation of a container management platform on Docker: Hadoop deployment as an example. Cluster Comput 24, 3421–3430 (2021). https://doi.org/10.1007/s10586-021-03337-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03337-w

Keywords