Abstract
Recently MapReduce has emerged as a popular model for supporting modern data-intensive computing. Hadoop, Apache open source project, has implemented MapReduce, which becomes the most popular tools for many programmers due to its ability to process parallel programming. However, to control the resource allocations and power efficient among the jobs, working on simulate the MapReduce is still important. So, there are a large number of simulating tools available to simulate distributed environment. In this paper, we conclude some typical simulator on MapReduce and analysis their work flow and defect to help the programmer of MapReduce to model the real execution process of MapReduce job.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ghemawat, S., Gobioff, H., Leung, S-T.: The Google file system. In: Proceedings of the 19th ACM SOSP, pp. 29–43, Dec 2003
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: Proceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI2004), pp. 137–150 (2004)
Apache Hadoop. http://hadoop.apache.org
Mumak.: Map-Reduce Simulator ASF JIRA. https://issues.apache.org/jira/browse/MAPREDUCE-728. Accessed 21 Apr 2010
Apache.: Rumen: a tool to extract job characterization data from jobtracker logs (2010). https://issues.apache.org/jira/browse/MAPREDUCE-751
Wang, G., Butt, A.R., Pandey, P., Gupta, K.: Using realistic simulation for performance analysis of MapReduce setups. In: Proceedings of the 1st ACM Workshop on Large-Scale System and Application Performance, Garching. ACM, Germany (2009), pp. 19–26
Verma, A., Cherkasova, L., Campbell, R.H.: PlayItAgain, SimMR!. In: Proceedings of the 2011 IEEE International Conference on Cluster Computing (Cluster’2011)
Hammoud, S., Li, M., Liu, Y., Alham, N., Liu, Z.: MRSim: a discrete event based MapReduce simulator. In: 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 6 (2010)
GridSim.: Get GridSim: A grid simulation toolkit at SourceForge.net. http://sourceforge.net/projects/gridsim/. Accessed 21 Apr 2010
Howell, F., McNab, R.: SimJava: A Discrete Event Simulation Library for Java. Simulation Series, vol. 30, 1998
Wang, G., Butt, A., Pandey, P., Gupta, K.: A simulation approach to evaluating design decisions in MapReduce setups. In: International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (2009)
Acknowledgement
This work is supported by the National High-Tech Research and Development Plan of China under Grant No. 2011AA01A204 and 2012AA01A306, National Key Technology R&D Program under grant No. 2011BAH04B03, National Natural Science Foundation of China under grant NO. 61202041, and the Fundamental Research Funds for the Central Universities under grant 08142007.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fan, Y., Wei, W., Gao, Y., Wu, W. (2014). Introduction and Analysis of Simulators of MapReduce. In: Yuan, Y., Wu, X., Lu, Y. (eds) Trustworthy Computing and Services. ISCTCS 2013. Communications in Computer and Information Science, vol 426. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43908-1_42
Download citation
DOI: https://doi.org/10.1007/978-3-662-43908-1_42
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43907-4
Online ISBN: 978-3-662-43908-1
eBook Packages: Computer ScienceComputer Science (R0)