Abstract
Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF’s ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, R., Masanet, E., Nordman, B., Tschudi, B., Shehabi, A., Stanley, J., Koomey, J., Sartor, D., Chan, P., Loper, J., Capana, S., Hedman, B., Duff, R., Haines, E., Sass, D., Fanara, A.: Report to Congress on Server and Data Center Energy Efficiency: Public Law 109–431 (2007)
Whitney, J., Delforge, P.: Scaling Up Energy Efficiency Across the Data Center Industry: Evaluating Key Drivers and Barriers. Data Center Efficiency Assessment (2014)
von Kistowski, J., Herbst, N., Kounev, S.: Modeling variations in load intensity over time. In: Proceedings of the third International Workshop on Large Scale Testing, LT 2014, pp. 1–4 (2014)
Chiosi, M., Clarke, D., Willis, P., Reid, A., Feger, J., Bugenhagen, M., Khan, W., Fargano, M., Cui, C., Deng, H., Benitez, J., Michel, U., Damker, H., Ogaki, K., Matsuzaki, T., Fukui, M., Shimano, K., Delisle, D., Loudier, Q., Kolias, C., Guardini, I., Demaria, E., Miverva, R., Manzalini, A., López, D., Salguero, F., Ruhl, F., Sen, P.: Network functions virtualisation - introductory white paper. In: SDN and OpenFlow World Congress (2012)
Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)
Herbst, N., Kounev, S., Reussner, R.: Elasticity in cloud computing: what it is, and what it is not. In: Proceedings of the 10th International Conference on Autonomic Computing, ICAC, San Jose, California, USA (2013)
von Kistowski, J., A. Arnold, J., Huppler, K., Lange, K.-D., L. Henning, J., Cao, P.: How to build a benchmark. In: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering, ICPE, Austin, Texas, USA (2015)
Power and Performance Benchmark Methodology V2.1. Standard Performance Evaluation Corporation (SPEC), Gainesville, USA (2012)
Huppler, K.R.: Performance per watt - benchmarking ways to get more for less. In: Nambiar, R., Poess, M. (eds.) TPCTC 2012. LNCS, vol. 7755, pp. 60–74. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36727-4_5
Schall, D., Hoefner, V., Kern, M.: Towards an enhanced benchmark advocating energy-efficient systems. In: Nambiar, R., Poess, M. (eds.) TPCTC 2011. LNCS, vol. 7144, pp. 31–45. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32627-1_3
von Kistowski, J., Becket, J., Lange, K.-D., Block, H., A. Arnold, J., Kounev, S.: Energy efficiency of hierarchical server load distributions strategies. In: 2015 IEEE 23rd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunications Systems (MASCOTS), Atlanta, USA (2015)
von Kistowski, J., Block, H., Beckett, J., Spradling, C., Lange, K.-D., Kounev, S.: Variations in CPU power consumption. In: Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE 2016, Delft, The Netherlands, pp. 147–158 (2016)
von Kistowski, J., Block, H., Beckett, J., Lange, K.-D., A. Arnold, J., Kounev, S.: Analysis of the influences on server power consumption and energy efficiency for CPU-intensive workloads. In: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering, ICPE 2015, Austin, Texas, pp. 223–234 (2015)
Lange, S., Nguyen-Ngoc, A., Gebert, S., Zinner, T., Jarschel, M., Köpsel, A., Sune, M., Raumer, D., Gallenmüller, S., Calre, G., Tran-Gia, P.: Performance benchmarking of a software-based LTE SGW. In: 2015 11th International Conference on Network and Service Management (CNSM), Barcelona, Spain (2015)
Falkner, M., Leivadeas, A., Lambadaris, I., Kesidis, G.: Performance analysis of virtualized network functions on virtualized systems architectures. In: 2016 IEEE 21st International Workshop on Computer Aided Modelling and Design of Communication Links and Networks (CAMAD), Toronto, Canada (2016)
Bradner, S., McQuaid, J.: RFC 2544: Benchmarking Methodology for Network Interconnected Devices. IETF (1999)
Bradner, S., McQuaid, J.: RFC 6201: Device Reset Characterization. IETF (2011)
Bradner, S., Dubray, K., McQuaid, J., Morton, A.: RFC 6815: Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful. IETF (2012)
Rosa Ed., E., Szabo, R.: VNF Benchmarking Methodology (draft expired in September 2016). IETF (2016)
Morton, A.: Considerations for Benchmarking Virtual Network Functions and Their Infrastructure (draft expired in August 2015). IETF (2015)
Moens, H., De Turck, F.: VNF-P: a model for efficient placement for virtualized network functions. In: 2014 10th International Conference on Network and Service Management (CNSM), Rio de Janeiro, Brazil (2014)
Addis, B., Belabed, D., Bouet, M., Secci, S.: Virtual network functions placement and routing optimization. In: 2015 IEEE 4th International Conference on Cloud Networking (CloudNet), Niagara Falls, Canada (2015)
SPEC VIRT_SC 2013. The Standard Performance Evaluation Corporation (SPEC) (2013). https://www.spec.org/virt_sc2013/
Smith, W.D., Sebastian, S.: Virtualization Performance Insights from TPC-VMS. Transaction Processing Performance Council (2013)
Bond, A., Johnson, D., Kopczynski, G., Taheri, H.R.: Profiling the performance of virtualized databases with the TPCx-V benchmark. In: Nambiar, R., Poess, M. (eds.) TPCTC 2015. LNCS, vol. 9508, pp. 156–172. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31409-9_10
TPC-Energy Specification. Transaction Processing Performance Council, Version 1.5.0 (2012). http://www.tpc.org/tpc_documents_current_versions/pdf/tpc-energy_v1.5.0.pdf
SPECpower_ssj2008. The Standard Performance Evaluation Corporation (SPEC) (2007). https://www.spec.org/power_ssj2008/
Server Efficiency Rating Tool 2. The Standard Performance Evaluation Corporation (SPEC) (2017). https://www.spec.org/sert2/
Chauffeur™WDK 2. The Standard Performance Evaluation Corporation (SPEC) (2017). https://www.spec.org/chauffeur-wdk/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Schmitt, N., von Kistowski, J., Kounev, S. (2018). Towards a Scalability and Energy Efficiency Benchmark for VNF. In: Nambiar, R., Poess, M. (eds) Performance Evaluation and Benchmarking for the Analytics Era. TPCTC 2017. Lecture Notes in Computer Science(), vol 10661. Springer, Cham. https://doi.org/10.1007/978-3-319-72401-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-72401-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72400-3
Online ISBN: 978-3-319-72401-0
eBook Packages: Computer ScienceComputer Science (R0)