Online Dynamic Container Rescheduling for Improved Application Service Time | Journal of Network and Systems Management Skip to main content
Springer Nature Link
Log in

Online Dynamic Container Rescheduling for Improved Application Service Time

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Despite their maturity and robustness, container orchestration platforms still suffer from some limitations. One of those concerns the lack of runtime adaptability of the scheduler to the overall cluster status as (i) it instantiates containers with local optimization in mind i.e. it only considers the container-specific predefined requirements which may lead to a sub-optimal overall cluster state and (ii) it does not reshuffle the deployed containers at runtime based on container observed behavior and interdependencies. These limitations become even more apparent within volatile load contexts. This work proposes an autonomous and dynamic rescheduling system that aims at improving application service time by co-locating highly interdependent containers for network delay reduction. To this extend, two distinct combinatorial optimization heuristics, Simulated Annealing and Particle Swarm Optimization, are evaluated and compared on their respective effectiveness and efficiency as well as on their relative performance towards the optimal solution obtained by Integer Linear Programming. Additionally, the impact of the proposed system on application service time is validated by means of two complementary use-cases, an event-based IoT data-hub platform and a web-based e-commerce app, with an average improvement of the end-to-end service time of 21.6% and 13.1% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. The intuition is that highly interacting containers spend more time in communication when placed across different nodes, thereby leading to a degradation in the observed response time.

  2. Moreover, a current trend is the hybridization of methods in the direction of the integration of trajectory methods in population-based ones [30].

  3. ilog.cplex.CpxException: CPLEX Error 1001: Out of memory.

  4. java.lang.OutOfMemoryError: Java heap space - reported by the program used to feed CPLEX with the variables and constraints.

  5. For scoping reasons, this work only considers hard constraints. The impact of soft constraints is considered as a candidate topic for future extensions of this work (see Sect. 5).

  6. Data isolation is internally ensured through the concept of scopes which represent logical data sets with configurable perimeter. A scope can be understood as a labelling mechanism aiming at isolating data between different contexts of use. Data access APIs for data ingestion, querying and streaming all require the scope to be mentioned.

References

  1. da Silva, V.G., Kirikova, M., Alksnis, G.: Containers for virtualization: an overview. Appl. Comput. Syst. 23(1), 21–27 (2018)

    Article  Google Scholar 

  2. Docker. Docker, Inc.: Docker Website. https://www.docker.com. [Online] (2022)

  3. LXC/LXD. The Linux Foundation: LXC Website. https://linuxcontainers.org. [Online] (2022)

  4. Podman. Red Hat, Inc.: Podman Website. https://podman.io/. [Online] (2022)

  5. containerd. The Cloud Native Computing foundation: containerd Website. https://containerd.io. [Online] (2022)

  6. Mesos. The Apache Software Foundation: Mesos Website. http://mesos.apache.org. [Online] (2022)

  7. Docker. Docker, Inc.: DockerSwarm Website. https://docs.docker.com/engine/swarm/. [Online] (2022)

  8. Kubernetes. The Cloud Native Computing Foundation: Kubernetes Website. https://kubernetes.io. [Online] (2022)

  9. Flexera: RightScale 2019 State of the Cloud Report from Flexera. Available at https://resources.flexera.com/web/media/documents/rightscale-2019-state-of-the-cloud-report-from-flexera.pdf (2020/05/20) (2019)

  10. Pinedo, M.L.: Scheduling. Springer, New York (2016)

    Book  MATH  Google Scholar 

  11. Bittencourt, L.F., Goldman, A., Madeira, E.R.M., da Fonseca, N.L.S., Sakellariou, R.: Scheduling in distributed systems: a cloud computing perspective. Comput. Sci. Rev. 30, 31–54 (2018). https://doi.org/10.1016/j.cosrev.2018.08.002

    Article  Google Scholar 

  12. Söylemez, M., Tekinerdogan, B., Tarhan, A.K.: Challenges and solution directions of microservice architectures: a systematic literature review. Appl. Sci. 12(11), 5507 (2022)

    Article  Google Scholar 

  13. Beloglazov, A., Buyya, R.: Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurr. Comput.: Pract. Exp. 24(13), 1397–1420 (2012). https://doi.org/10.1002/cpe.1867

    Article  Google Scholar 

  14. Mahdhi, T., Mezni, H.: A prediction-based vm consolidation approach in iaas cloud data centers. J. Syst. Softw. 146, 263–285 (2018). https://doi.org/10.1016/j.jss.2018.09.083

    Article  Google Scholar 

  15. Wang, J.V., Cheng, C.-T., Tse, C.K.: Thermal-aware vm consolidation mechanism with outage avoidance. Softw.: Pract. Exp. 49(5), 906–920 (2019)

    Google Scholar 

  16. Zhao, D., Mohamed, M., Ludwig, H.: Locality-aware scheduling for containers in cloud computing. IEEE Trans. Cloud Comput. 8(2), 635–646 (2020). https://doi.org/10.1109/TCC.2018.2794344

    Article  Google Scholar 

  17. Filip, I.-D., Pop, F., Serbanescu, C., Choi, C.: Microservices scheduling model over heterogeneous cloud-edge environments as support for iot applications. IEEE Int. Things J. 5(4), 2672–2681 (2018). https://doi.org/10.1109/JIOT.2018.2792940

    Article  Google Scholar 

  18. Nanda, S., Hacker, T.J.: Racc: Resource-aware container consolidation using a deep learning approach. In: Proceedings of the First Workshop on Machine Learning for Computing Systems. MLCS’18. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3217871.3217876

  19. Wen, Z., Lin, T., Yang, R., Ji, S., Ranjan, R., Romanovsky, A., Lin, C., Xu, J.: Ga-par: dependable microservice orchestration framework for geo-distributed clouds. IEEE Trans. Parallel Distrib. Syst. 31(1), 129–143 (2020). https://doi.org/10.1109/TPDS.2019.2929389

    Article  Google Scholar 

  20. Guerrero, C., Lera, I., Juiz, C.: Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications. J. Supercomput. 74(7), 2956–2983 (2018)

    Article  Google Scholar 

  21. Piraghaj, S.F., Dastjerdi, A.V., Calheiros, R.N., Buyya, R.: A framework and algorithm for energy efficient container consolidation in cloud data centers. In: 2015 IEEE International Conference on Data Science and Data Intensive Systems, pp. 368–375 (2015)https://doi.org/10.1109/DSDIS.2015.67

  22. Rattihalli, G.: Exploring potential for resource request right-sizing via estimation and container migration in apache mesos. In: 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), pp. 59–64 (2018)https://doi.org/10.1109/UCC-Companion.2018.00035

  23. Bulej, L., Bureš, T., Hnětynka, P., Khalyeyev, D.: Self-adaptive k8s cloud controller for time-sensitive applications. In: 2021 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 166–169 (2021) https://doi.org/10.1109/SEAA53835.2021.00029

  24. Rodriguez, M., Buyya, R.: Container orchestration with cost-efficient autoscaling in cloud computing environments. In: Handbook of Research on Multimedia Cyber Security, IGI global, Melbourne pp. 190–213. (2020) https://doi.org/10.4018/978-1-7998-2701-6.ch010

  25. Zhou, R., Li, Z., Wu, C.: An efficient online placement scheme for cloud container clusters. IEEE J. Select. Areas Commun. 37(5), 1046–1058 (2019). https://doi.org/10.1109/JSAC.2019.2906745

    Article  Google Scholar 

  26. Wojciechowski, L., Opasiak, K., Latusek, J., Wereski, M., Morales, V., Kim, T., Hong, M.: Netmarks: Network metrics-aware kubernetes scheduler powered by service mesh. In: IEEE INFOCOM 2021 - IEEE Conference on Computer Communications, pp. 1–9 (2021). https://doi.org/10.1109/INFOCOM42981.2021.9488670

  27. Marchese, A., Tomarchio, O.: Network-aware container placement in cloud-edge kubernetes clusters. In: 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid), pp. 859–865 (2022). https://doi.org/10.1109/CCGrid54584.2022.00102

  28. Joseph, C.T., Chandrasekaran, K.: Nature-inspired resource management and dynamic rescheduling of microservices in cloud datacenters. Concurr. Comput.: Pract. Exp. 33(17), 6290 (2021)

    Article  Google Scholar 

  29. Koopmans, T.C., Beckmann, M.: Assignment problems and the location of economic activities. Econometrica: J. Econometric Soc. 25, 53–76 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  30. Blum, C., Roli, A.: Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput. Surv. 35, 268–308 (2001). https://doi.org/10.1145/937503.937505

    Article  Google Scholar 

  31. Glover, F.W.: Future paths for integer programming and links to artificial intelligence. Comput. Oper. Res. 13, 533–549 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  32. Baum, E.B.: Towards practical ‘neural’ computation for combinatorial optimization problems. In: AIP Conference Proceedings, vol. 151, pp. 53–58 (1986). American Institute of Physics

  33. Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Operat. Res. 24(11), 1097–1100 (1997). https://doi.org/10.1016/S0305-0548(97)00031-2

    Article  MathSciNet  MATH  Google Scholar 

  34. Feo, T.A., Resende, M.G.C.: A probabilistic heuristic for a computationally difficult set covering problem. Operat. Res. Lett. 8(2), 67–71 (1989). https://doi.org/10.1016/0167-6377(89)90002-3

    Article  MathSciNet  MATH  Google Scholar 

  35. Kirkpatrick, S., Gelatt, C., Vecchi, M.: Optimization by simulated annealing. Science 220, 671–680 (1983). https://doi.org/10.1126/science.220.4598.671

    Article  MathSciNet  MATH  Google Scholar 

  36. Cerny, V.: Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J. Optimiz. Theory Appl. 45, 41–51 (1985). https://doi.org/10.1007/BF00940812

    Article  MathSciNet  MATH  Google Scholar 

  37. Holland, J.: Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  38. Bozorg-Haddad, O., Afshar, A., Mariño, M.: Honey-bees mating optimization (hbmo) algorithm: a new heuristic approach for water resources optimization. Water Res. Manag. 20, 661–680 (2006). https://doi.org/10.1007/s11269-005-9001-3

    Article  Google Scholar 

  39. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN’95 - International Conference on Neural Networks, vol. 4, pp. 1942–19484 (1995). https://doi.org/10.1109/ICNN.1995.488968

  40. Dorigo, M., Birattari, M., Stutzle, T.: Ant colony optimization. IEEE Comput. Intell. Mag. 1(4), 28–39 (2006). https://doi.org/10.1109/MCI.2006.329691

    Article  Google Scholar 

  41. Koulamas, C., Antony, S., Jaen, R.: A survey of simulated annealing applications to operations research problems. Omega 22(1), 41–56 (1994). https://doi.org/10.1016/0305-0483(94)90006-X

    Article  Google Scholar 

  42. Connolly, D.T.: An improved annealing scheme for the qap. Eur. J. Operat. Res. 46(1), 93–100 (1990). https://doi.org/10.1016/0377-2217(90)90301-Q

    Article  MathSciNet  MATH  Google Scholar 

  43. Fidanova, S.: Simulated annealing for grid scheduling problem. In: IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing (JVA’06), pp. 41–45 (2006). https://doi.org/10.1109/JVA.2006.44

  44. Sengupta, S., Basak, S., Peters, R.A.: Particle swarm optimization: A survey of historical and recent developments with hybridization perspectives. Machine Lear. Knowledge Extract. 1(1), 157–191 (2019). https://doi.org/10.3390/make1010010

    Article  Google Scholar 

  45. Salman, A., Ahmad, I., Al-Madani, S.: Particle swarm optimization for task assignment problem. Microprocess. Microsyst. 26(8), 363–371 (2002). https://doi.org/10.1016/S0141-9331(02)00053-4

    Article  Google Scholar 

  46. Zhang, L., Chen, Y., Sun, R., Jing, S., Yang, B.: A task scheduling algorithm based on pso for grid computing. Int. J. Comput. Intell. Res. 4(1), 37–43 (2008)

    Google Scholar 

  47. Bertsimas, D., Tsitsiklis, J.: Simulated Annealing. Stat. Sci. 8(1), 10–15 (1993). https://doi.org/10.1214/ss/1177011077

    Article  MATH  Google Scholar 

  48. Ellison Geltman, K.: The Simulated Annealing Algorithm. http://katrinaeg.com/simulated-annealing.html (2014)

  49. Rezaee Jordehi, A., Jasni, J.: Particle swarm optimisation for discrete optimisation problems: a review. Artif. Intell. Rev. (2014). https://doi.org/10.1007/s10462-012-9373-8

    Article  Google Scholar 

  50. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360), pp. 69–73 (1998). https://doi.org/10.1109/ICEC.1998.699146

  51. Bansal, J.C., Singh, P.K., Saraswat, M., Verma, A., Jadon, S.S., Abraham, A.: Inertia weight strategies in particle swarm optimization. In: 2011 Third World Congress on Nature and Biologically Inspired Computing, pp. 633–640 (2011). https://doi.org/10.1109/NaBIC.2011.6089659

  52. García, F., Moreno-Pérez, J.: Jumping frogs optimization: a new swarm method for discrete optimization. Technical report, Grupo de Computación Inteligente, Departamento de Estadística, I.O. y C., Instituto Universitario de Desarrollo Regional, University of La Laguna, Tenerife, Spain (January 2008)

  53. Balaji, S., Revathi, N.: A new approach for solving set covering problem using jumping particle swarm optimization method. Nat. Comput. 15, 503–517 (2015). https://doi.org/10.1007/s11047-015-9509-2

    Article  MathSciNet  MATH  Google Scholar 

  54. Gutiérrez, J., Landa-Silva, D., Moreno-Pérez, J.: Exploring feasible and infeasible regions in the vehicle routing problem with time windows using a multi-objective particle swarm optimization approach. In: Proceedings of the International Workshop on Nature Inspired Cooperatives Strategies for Optimization, NICSO, pp. 103–114 (2008). https://doi.org/10.1007/978-3-642-03211-0_9

  55. Consoli, S., Moreno-Pérez, J., Darby-Dowman, K., Mladenovic, N.: Discrete particle swarm optimization for the minimum labelling steiner tree problem. Nat. Comput. 9, 29–46 (2010). https://doi.org/10.1007/s11047-009-9137-9

    Article  MathSciNet  MATH  Google Scholar 

  56. Hahn, D.: A Day in the Life of a Netflix Engineer. https://youtu.be/-mL3zT1iIKw?t=931. AWS re:Invent 2015 - Las Vegas - Accessed: 2022-07-08 (2015)

  57. Vogels, W.: Real-time graph of microservice dependencies at amazon.com in 2008. https://twitter.com/Werner/status/741673514567143424. CTO Amazon - Accessed: 2022-07-08 (2008)

  58. Tewolde, G.S., Hanna, D.M., Haskell, R.E.: Enhancing performance of pso with automatic parameter tuning technique. In: 2009 IEEE Swarm Intelligence Symposium, pp. 67–73 (2009). https://doi.org/10.1109/SIS.2009.4937846

  59. Park, M.-W., Kim, Y.-D.: A systematic procedure for setting parameters in simulated annealing algorithms. Comput. Operat. Res. 25(3), 207–217 (1998)

    Article  MATH  Google Scholar 

  60. Santos, J., Wauters, T., Volckaert, B., De Turck, F.: Resource provisioning in fog computing: from theory to practice. Sensors 19, 2238 (2019). https://doi.org/10.3390/s19102238

    Article  Google Scholar 

  61. SIGs, K.: Descheduler for Kubernetes. https://github.com/kubernetes-sigs/descheduler. Accessed: 2022-09-19 (2022)

  62. Wikipedia: Control theory. https://en.wikipedia.org/wiki/Control_theory. Accessed: 2022-09-20 (2022)

  63. Kubernetes: Kubernetes Controllers. https://kubernetes.io/docs/concepts/architecture/controller/. Accessed: 2022-09-20 (2022)

  64. pixielabs.ai: Pixie Overview. https://docs.pixielabs.ai/about-pixie/what-is-pixie. Accessed: 2022-09-22 (2022)

  65. altexsoft.com: What is Data Hub: Purpose, Architecture Patterns, and Existing Solutions Overview. https://www.altexsoft.com/blog/data-hub/. Accessed: 2022-09-12 (2021)

  66. cloudera.com: CLOUDERA DATA PLATFORM Data Hub, A comprehensive cloud-based Edge-to-AI analytics service. https://www.cloudera.com/products/data-hub.html. Accessed: 2022-09-12 (2022)

  67. softwareag.com: Cumulocity IoT DataHub overview. https://cumulocity.com/guides/datahub/datahub-overview/. Accessed: 2022-09-12 (2022)

  68. microsoft.com: Azure IoT Hub: Connect, monitor and manage billions of IoT assets. https://azure.microsoft.com/en-us/services/iot-hub/. Accessed: 2022-09-12 (2022)

  69. amazon.com: AWS IoT Core: Easily and securely connect devices to the cloud. https://aws.amazon.com/iot-core/. Accessed: 2022-09-12 (2022)

  70. google.com: Google Cloud IoT Core: A fully managed service to easily and securely connect, manage, and ingest data from globally dispersed devices. https://cloud.google.com/iot-core. Accessed: 2022-09-12 (2022)

  71. Bracke, V., Sebrechts, M., Moons, B., Hoebeke, J., De Turck, F., Volckaert, B.: Design and evaluation of a scalable internet of things backend for smart ports. Software: Practice and Experience 51(7), 1557–1579 (2021). https://doi.org/10.1002/spe.2973. https://onlinelibrary.wiley.com/doi/pdf/10.1002/spe.2973

  72. google.com: GoogleCloudPlatform microservices-demo: Online Boutique. https://github.com/GoogleCloudPlatform/microservices-demo. Accessed: 2022-09-12 (2022)

Download references

Acknowledgements

José Santos is funded by the Research Foundation Flanders (FWO), grant number 1299323N.

Funding

José Santos is funded by the Research Foundation Flanders (FWO), grant number 1299323N.

Author information

Authors and Affiliations

  1. IDLab, Department of Information Technology, Ghent University - imec, Technologiepark-Zwijnaarde 126, B-9052, Ghent, Belgium

    Vincent Bracke, Gillis Werrebrouck, José Santos, Tim Wauters, Filip De Turck & Bruno Volckaert

Authors
  1. Vincent Bracke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gillis Werrebrouck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tim Wauters
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Filip De Turck
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bruno Volckaert
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

VB: substantially contributed to the conception and design of the work, to the acquisition, analysis and interpretation of data and to the creation of new software used in the work. He drafted the work and substantively revised it. GW: substantially contributed to the design of the work, as well as to the creation of new software used in the work. He drafted the work. JS and TW: substantially contributed to the analysis and interpretation of data. They substantively revised the work. FDT and BVL: substantially contributed to the conception of the work, as well as to the analysis and interpretation of data. They drafted the work and substantively revised it. All authors have approved the submitted version.

Corresponding author

Correspondence to Vincent Bracke.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethical Approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bracke, V., Werrebrouck, G., Santos, J. et al. Online Dynamic Container Rescheduling for Improved Application Service Time. J Netw Syst Manage 31, 80 (2023). https://doi.org/10.1007/s10922-023-09766-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-023-09766-9

Keywords

Search

Navigation