Abstract
The fast processing speeds of the current generation of supercomputers provide a great convenience to scientists dealing with extremely large data sets. The next generation of exascale supercomputers could provide accurate simulation results for the automobile industry, aerospace industry, and even nuclear fusion reactors for the very first time. However, the energy cost of super-computing is extremely high, with a total electricity bill of 9 million dollars per year. Thus, conserving energy and increasing the energy efficiency of supercomputers have become critical in recent years. Many researchers have studied this problem and are trying to conserve energy by incorporating the dynamic voltage frequency scaling technique into their methods. However, this approach is limited, especially when the workload is high. In this paper, we developed a power-aware job scheduler by applying a rule-based control method and taking into consideration real-world power and speedup profiles to improve power efficiency while adhering to predetermined power constraints. The intensive simulation results showed that our proposed method is able to achieve the maximum utilization of computing resources as compared to baseline scheduling algorithms while keeping the energy cost under the threshold. Moreover, by introducing a power performance factor based on the real-world power and speedup profiles, we are able to increase the power efficiency by up to 75%.
Similar content being viewed by others
References
Niu S, Zhai J, Ma X, Liu M, Zhai Y, Chen W, Zheng W (2013) Employing checkpoint to improve job scheduling in large-scale systems. In: Job Scheduling Strategies for Parallel Processing. Springer, pp 36–55
Supercomputer ’titans’ face huge energy costs. http://www.livescience.com/18072-rise-titans-exascale-supercomputers-leap-power-hurdle.html. Accessed 23 Jan 2012
Reducing energy consumption and cost in the data center. http://www.datacenterknowledge.com/archives/2014/12/11/reducing-energy-consumption-cost-data-center/. Accessed 11 Dec 2014
America’s data centers consuming and wasting growing amounts of energy. http://www.nrdc.org/energy/data-center-efficiency-assessment.asp. Accessed 6 Feb 2015
Wang Y, Lu P (2013) DDS: a deadlock detection-based scheduling algorithm for workflow computations in HPC systems with storage constraints. Parallel Comput 39(8):291–305
Chan H-L, Chan W-T, Lam T-W, Lee L-K, Mak K-S, Wong PW (2007) Energy efficient online deadline scheduling. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, pp 795–804
Wang X, Wang Y, Zhu H (2012) Energy-efficient task scheduling model based on mapreduce for cloud computing using genetic algorithm. J Comput 7(12):2962–2970
Deore SS, Patil AN (2013) Energy-efficient job scheduling and allocation scheme for virtual machines in private clouds. Energy 5(1):56–60
Varsamopoulos G, Banerjee A, Gupta SK (2009) Energy efficiency of thermal-aware job scheduling algorithms under various cooling models. In: Contemporary Computing. Springer, pp 568–580
Wang L-X (1999) A course in fuzzy systems. Prentice-Hall press, Upper Saddle River
Shaocheng T, Changying L, Yongming L (2009) Fuzzy adaptive observer backstepping control for mimo nonlinear systems. Fuzzy Sets Syst 160(19):2755–2775
Tong S, Li Y (2009) Observer-based fuzzy adaptive control for strict-feedback nonlinear systems. Fuzzy Sets Syst 160(12):1749–1764
The san diego supercomputer center (sdsc) sp2 log. http://www.cs.huji.ac.il/labs/parallel/workload/l_sdsc_sp2/index.html. Accessed 6 Feb 2015
Logs of real parallel workloads from production systems. http://www.cs.huji.ac.il/labs/parallel/workload/logs.html. Accessed 6 Feb 2015
The anl intrepid log. http://www.cs.huji.ac.il/labs/parallel/workload/l_anl_int/index.html. Accessed 6 Feb 2015
The intel netbatch logs. http://www.cs.huji.ac.il/labs/parallel/workload/l_intel_netbatch/index.html. Accessed 6 Feb 2015
Jackson D, Snell Q, Clement M (2001) Core algorithms of the maui scheduler. In: Job Scheduling Strategies for Parallel Processing. Springer, pp 87–102
Soner S, Ozturan C (2013) An auction based slurm scheduler for heterogeneous supercomputers and its comparative performance study. Technical report, PRACE. http://www.prace-project.eu/IMG/pdf/wp59_an_auction_based_slurm_scheduler_heterogeneous_supercomputers_and_its_comparative_study.pdf. Accessed 6 Dec 2013
Schroeder B, Harchol-Balter M (2004) Evaluation of task assignment policies for supercomputing servers: the case for load unbalancing and fairness. Clust Comput 7(2):151–161
Chandio AA, Bilal K, Tziritas N, Yu Z, Jiang Q, Khan SU, Xu C-Z (2014) A comparative study on resource allocation and energy efficient job scheduling strategies in large-scale parallel computing systems. Clust Comput 17(4):1349–1367
Mämmelä O, Majanen M, Basmadjian R, De Meer H, Giesler A, Homberg W (2012) Energy-aware job scheduler for high-performance computing. Comput Sci Res Dev 27(4):265–275
Liu C, Qin X, Kulkarni S, Wang C, Li S, Manzanares A, Baskiyar S (2008) Distributed energy-efficient scheduling for data-intensive applications with deadline constraints on data grids. In: Performance, Computing and Communications Conference, 2008. IPCCC 2008. IEEE International. IEEE, pp 26–33
Zhang Y, Duan L, Li B, Peng L, Sadagopan S (2014) Energy efficient job scheduling in single-isa heterogeneous chip-multiprocessors. In: 2014 15th International Symposium on Quality Electronic Design (ISQED). IEEE, pp 660–666
Chen J-J, Kuo T-W (2005) Energy-efficient scheduling of periodic real-time tasks over homogeneous multiprocessors. PARC 1:30–35
Fox K, Im S, Moseley B (2013) Energy efficient scheduling of parallelizable jobs. In: Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithms. SIAM, pp 948–957
Khuller S, Li J, Saha B (2010) Energy efficient scheduling via partial shutdown. In: Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, pp 1360–1372
Zadeh LA (2008) Is there a need for fuzzy logic? Inf Sci 178(13):2751–2779
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Wang, J., Han, D. & Wang, R. A new rule-based power-aware job scheduler for supercomputers. J Supercomput 74, 2508–2527 (2018). https://doi.org/10.1007/s11227-018-2281-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-018-2281-1