Abstract
The past several years have seen a rapid increase in the number and type of public cloud computing hardware configurations and pricing options offered to customers. In addition public cloud providers have also expanded the number and type of storage options and established incremental price points for storage and network transmission of outbound data from the cloud facility. This has greatly complicated the analysis to determine the most economical option for moving general purpose applications to the cloud. This paper investigates whether this economic analysis for moving general purpose applications to the public cloud can be extended to more computationally intensive HPC type computations. Using an HPC baseline hardware configuration for comparison, the total cost of operations for several HPC private and public cloud providers are analyzed. The analysis shows under what operational conditions the public cloud option may be a more cost effective alternative for HPC type applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The ‘M’ in M4.10xLarge denotes the General Purpose intention of the instance and ‘4’ indicates that the processors are current generation processors. Each virtual CPU is a hyperthread of an Intel Xeon core.
- 2.
The spot pricing is the cheapest way one can secure access to cloud resources. However if another user initiates a reservation instance in the midst of the spot reservation, the spot reservation gets preempted for the reserved instance and the usage will not be charged to the spot reservation user.
- 3.
For instance the most powerful Compute Intensive instance provided by Amazon is C4.8xLarge which has 32 virtual cores and 60 GB of RAM.
- 4.
A virtual CPU is equivalent to a single hyperthread on a 2.6 GHz Intel Xeon E5 (Sandy Bridge), 2.5 GHz Intel Xeon E5 v2(Ivy Bridge), or 2.3 GHz Intel Xeon E5 v3 (Haswell) depending on the processor which makes up the instance.
- 5.
The Private Cloud does not have a fee for the data transfer over the network.
- 6.
This potential economic strategy may work if the user’s job only involves computations and is not dependent on staging large quantities of data in a particular region and zone before submitting a bid price for access.
References
Chen, Y., Sion, R.: To cloud or not to cloud? musings on costs and viability. In: Proceedings of the 2nd ACM Symposium on Cloud Computing, pp. 29:1–29:7 (2011)
Walker, E.: The real cost of a CPU hour. IEEE Comput. 42, 3541 (2009)
Zhai, Y., Liu, M., Zhai, J.: Cloud versus in-house cluster: evaluating amazon cluster compute instances for running MPI applications. In: State of the Practice Reports, pp. 11:1–11:10 (2011)
Gupta, A., Milojicic, D.: Evaluation of HPC applications on cloud. In: Fifth Open Cirrus Summit, pp. 22–26 (2011)
Ostermann, S., Iosup, A., Yigitbasi, N., Prodan, R., Fahringer, T., Epema, D.: A performance analysis of EC2 cloud computing services for scientific computing. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) CloudComp 2009. LNICSSTE, vol. 34, pp. 115–131. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12636-9_9
Ding, F., Mey, D., Wienke, S., Zhang, R., Li, L.: A study on today’s cloud environments for HPC applications. In: Helfert, M., Desprez, F., Ferguson, D., Leymann, F. (eds.) CLOSER 2013. CCIS, vol. 453, pp. 114–127. Springer, Cham (2014). doi:10.1007/978-3-319-11561-0_8
Brandt, J., Gentile, A., Mayo, J., Pebay, P., Roe, D., Thompson, D., Wong, M.: Resource monitoring and management with OVIS to enable HPC in cloud computing environments. In: IEEE International Symposium Parallel Distributed Processing, IPDPS 2009, pp. 1–8 (2009)
Gupta, A., Kal, L.V.: Towards efficient mapping, scheduling, and execution of HPC applications on platforms in cloud. In: Parallel and Distributed Processing Symposium Workshops Ph.D. Forum, pp. 2294–2297 (2013)
Gómez Sáez, S., Andrikopoulos, V., Hahn, M., Karastoyanova, D., Leymann, F., Skouradaki, M., Vukojevic-Haupt, K.: Performance and cost trade-off in IaaS environments: a scientific workflow simulation environment case study. In: Helfert, M., Méndez Muñoz, V., Ferguson, D. (eds.) CLOSER 2015. CCIS, vol. 581, pp. 153–170. Springer, Cham (2016). doi:10.1007/978-3-319-29582-4_9
Saez, S., Andrikopoulos, V., Hahn, M., Karastoyanova, D., Leymann, F., Skouradaki, M., Vukojevic-Haupt, K.: Performance and cost evaluation for the migration of a scientific workflow infrastructure to the cloud. In: Proceedings of the 5th International Conference on Cloud Computing and Service Science, CLOSER 2015, p. 110. SciTePress (2015)
Coghlan, S., Yelick, K., Draney, B., Canon, R.S: The Magellan report on cloud computing. In: Office of Advanced Scientific Computing Research (ASCR), US Department of Energy (2011). http://science.energy.gov/~/media/ascr/pdf/programdocuments/docsMagellan_Final_Report.pdf
Vouk, M., Sills, E., Dreher, P.: Integration of high-performance computing into cloud computing services. In: Handbook of Cloud Computing, pp. 255–276 (2010). Chap. 11
Amazon High Performance Computing (2016). https://aws.amazon.com/hpc/
Google Compute Engine (2016). https://cloud.google.com/compute/
Microsoft Azure (2016). https://azure.microsoft.com/en-us/
Microsoft Big Compute: HPC & Batch (2016). https://azure.microsoft.com/en-us/solutions/big-compute/
Vouk, M.: Cloud computing issues, research and implementations. J. Comput. Inf. Technol. 16(4), 235–246 (2008)
Dreher, P., Vouk, M., Sills, E., Averitt, S.: Evidence for a cost effective cloud computing implementation based upon the NC state virtual computing laboratory model. In: Advances in Parallel Computing, High Speed and Large Scale Scientific Computing, vol. 18, pp. 236–250 (2009)
Schaffer, H.E., Averitt, S.F., Hoit, M.I., Peeler, A., Sills, E.D., Vouk, M.A.: NCSUs virtual computing laboratory: a cloud computing solution. In: IEEE Computer, pp. 94–97 (2009)
Apache VCL (2016). https://vcl.apache.org/
Amazon EC2 Spot Instances. http://aws.amazon.com/ec2/spot-instances/
Zhang, Q., Gurses, E., Boutaba, R., and Xiao, J., Dynamic resource allocation for spot markets in clouds. In: Workshop on Hot Topics in Management of Internet, Cloud, and Enterprise Networks and Services Hot-ICE (2011)
Chen, J., Wang, C., Zhou, B.B., Sun, L., Lee, Y.C., Zomaya, A.Y.: Tradeoffs between profit and customer satisfaction for service provisioning in the cloud. In: HPDC (2011)
Mazzucco, M., Dumas, M.: Achieving performance and availability guarantees with spot instances. In: IEEE International Conference on High Performance Computing and Communications, pp. 296–303 (2011)
Mattess, M., Vecchiola, C., Buyya, R.: Managing peak loads by leasing cloud infrastructure services from a spot market. In: IEEE International Conference on High Performance Computing and Communications, pp. 180–188 (2010)
Yi, S., Andrzejak, A., Kondo, D.: Monetary cost-aware checkpointing and migration on amazon cloud spot instances. IEEE Trans. Serv. Comput. 5(4), 512–524 (2012)
Agmon Ben-Yehuda, O., Ben-Yehuda, M., Schuster, A., Tsafrir, D.: Deconstructing amazon ec2 spot instance pricing. In: 2011 IEEE Third International Conference on Cloud Computing Technology and Science (CloudCom), pp. 304–311 (2011)
Bonacquisto, P., Di Modica, G., Petralia, G., Tomarchio, O.: Dynamic pricing in cloud markets: evaluation of procurement auctions. In: CLOSER 2014. CCIS, vol. 512, pp. 31–46 (2015)
GitHub Repository for Boto Python Library (2016). https://github.com/boto/boto
Amazon SDK for Python to access Amazon public data (2016). https://aws.amazon.com/sdk-for-python/
Bhatia, K.: The data science of AWS Spot Pricing (2015). https://medium.com/cloud-uprising/the-data-science-of-aws-spot-pricing-8bed655caed2#.f9w14i4iq
Acknowledgments
This work is supported in part through NSF grant 0910767, 1318564, 1330553, the U.S. Army Research Office (ARO) grant W911NF-08-1-0105 managed by the NCSU Science of Security Initiative and the Science of Security Lablet, by the IBM Share University Research and Fellowships program funding, and the Argonne Leadership Computing Facility at Argonne National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under contract DE-AC02-06CH11357. One of us (Patrick Dreher) gratefully acknowledges support with an IBM Faculty award.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Dreher, P., Nair, D., Sills, E., Vouk, M. (2017). Cost Analysis Comparing HPC Public Versus Private Cloud Computing. In: Helfert, M., Ferguson, D., Méndez Muñoz, V., Cardoso, J. (eds) Cloud Computing and Services Science. CLOSER 2016. Communications in Computer and Information Science, vol 740. Springer, Cham. https://doi.org/10.1007/978-3-319-62594-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-62594-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-62593-5
Online ISBN: 978-3-319-62594-2
eBook Packages: Computer ScienceComputer Science (R0)