{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T04:03:41Z","timestamp":1729137821488,"version":"3.27.0"},"reference-count":63,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2018,5,2]],"date-time":"2018-05-02T00:00:00Z","timestamp":1525219200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"SKAMPY: ultra-scalable multi-physics simulations for solidification processes in metals","award":["01 IH 15003 A"]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2019,7]]},"abstract":" Realistic simulations in engineering or in the materials sciences can consume enormous computing resources and thus require the use of massively parallel supercomputers. The probability of a failure increases both with the runtime and with the number of system components. For future exascale systems, it is therefore considered critical that strategies are developed to make software resilient against failures. In this article, we present a scalable, distributed, diskless, and resilient checkpointing scheme that can create and recover snapshots of a partitioned simulation domain. We demonstrate the efficiency and scalability of the checkpoint strategy for simulations with up to 40 billion computational cells executing on more than 400 billion floating point values. A checkpoint creation is shown to require only a few seconds and the new checkpointing scheme scales almost perfectly up to more than 260,\u200a000 (218<\/jats:sup>) processes. To recover from a diskless checkpoint during runtime, we realize the recovery algorithms using ULFM MPI. The checkpointing mechanism is fully integrated in a state-of-the-art high-performance multi-physics simulation framework. We demonstrate the efficiency and robustness of the method with a realistic phase-field simulation originating in the material sciences and with a lattice Boltzmann method implementation. <\/jats:p>","DOI":"10.1177\/1094342018767736","type":"journal-article","created":{"date-parts":[[2018,5,3]],"date-time":"2018-05-03T02:24:57Z","timestamp":1525314297000},"page":"571-589","update-policy":"http:\/\/dx.doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":17,"title":["A scalable and extensible checkpointing scheme for massively parallel simulations"],"prefix":"10.1177","volume":"33","author":[{"given":"Nils","family":"Kohl","sequence":"first","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"}]},{"given":"Johannes","family":"H\u00f6tzer","sequence":"additional","affiliation":[{"name":"Institute of Materials and Processes, Karlsruhe University of Applied Sciences, Karlsruhe, Germany"}]},{"given":"Florian","family":"Schornbaum","sequence":"additional","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"}]},{"given":"Martin","family":"Bauer","sequence":"additional","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-2794-9510","authenticated-orcid":false,"given":"Christian","family":"Godenschwager","sequence":"additional","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"}]},{"given":"Harald","family":"K\u00f6stler","sequence":"additional","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"}]},{"given":"Britta","family":"Nestler","sequence":"additional","affiliation":[{"name":"Institute of Materials and Processes, Karlsruhe University of Applied Sciences, Karlsruhe, Germany"},{"name":"Institute for Applied Materials, Karlsruhe Institute of Technology, Karlsruhe, Germany"}]},{"given":"Ulrich","family":"R\u00fcde","sequence":"additional","affiliation":[{"name":"Chair for System Simulation, Friedrich-Alexander University Erlangen-N\u00fcrnberg, Erlangen, Germany"},{"name":"Parallel Algorithms Project, CERFACS, Toulouse, France"}]}],"member":"179","published-online":{"date-parts":[[2018,5,2]]},"reference":[{"key":"bibr1-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2013.10.001"},{"key":"bibr2-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1142\/S0129183114410095"},{"key":"bibr3-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.compfluid.2015.12.010"},{"key":"bibr4-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocs.2015.02.006"},{"key":"bibr5-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807662"},{"key":"bibr6-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1080\/17445760.2015.1118478"},{"key":"bibr7-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/2063384.2063427"},{"volume-title":"Department of Electrical Engineering and Computer Science, University of Tennessee","year":"2012","author":"Bland W","key":"bibr8-1094342018767736"},{"key":"bibr62-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33518-1_24"},{"key":"bibr9-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1177\/1094342013488238"},{"key":"bibr10-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid.2015.51"},{"key":"bibr11-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.93.043302"},{"key":"bibr12-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2008.12.002"},{"issue":"1","key":"bibr13-1094342018767736","volume":"1","author":"Cappello F","year":"2014","journal-title":"Supercomputing Frontiers and Innovations"},{"key":"bibr14-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1975.218955"},{"key":"bibr15-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/1065944.1065973"},{"key":"bibr16-1094342018767736","first-page":"370","volume-title":"Proceedings of annual symposium on fault tolerant computing. IEEE Comput. Soc. Press","author":"Chiueh TC","year":"1996"},{"key":"bibr17-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.85.021602"},{"key":"bibr18-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2004.11.016"},{"key":"bibr19-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/s12666-012-0172-3"},{"key":"bibr20-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2014.122"},{"key":"bibr21-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69182-2_37"},{"key":"bibr22-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1177\/1094342010391989"},{"key":"bibr23-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/568522.568525"},{"key":"bibr24-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84882-409-6_8"},{"key":"bibr25-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2014.12.003"},{"key":"bibr26-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.camwa.2017.03.017"},{"key":"bibr27-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/2503210.2503273"},{"key":"bibr28-1094342018767736","doi-asserted-by":"publisher","DOI":"10.21914\/anziamj.v50i0.1445"},{"key":"bibr29-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/46\/1\/067"},{"key":"bibr30-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-20943-2"},{"volume-title":"High Performance Computing in Science and Engineering \u2013 Garching\/Munich 2016","year":"2016","author":"H\u00f6tzer J","key":"bibr31-1094342018767736"},{"key":"bibr32-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/TC.1984.1676475"},{"key":"bibr33-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1137\/15M1026122"},{"key":"bibr34-1094342018767736","unstructured":"H\u00f6tzer J (2017) Massiv-parallele und gro\u00dfskalige Phasenfeldsimulationen zur Untersuchung der Mikrostrukturentwicklung, Vol. 70. Karlsruhe: KIT Scientific Publishing. DOI: 10.5445\/IR\/1000069984."},{"key":"bibr35-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.actamat.2015.03.051"},{"issue":"3","key":"bibr36-1094342018767736","first-page":"235","volume":"96","author":"H\u00f6tzer J","year":"2016","journal-title":"Journal of the Indian Institute of Science"},{"key":"bibr37-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.actamat.2015.12.052"},{"key":"bibr38-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.actamat.2017.07.007"},{"key":"bibr63-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/s10853-015-9542-7"},{"key":"bibr39-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/s00450-009-0066-8"},{"key":"bibr40-1094342018767736","doi-asserted-by":"publisher","DOI":"10.17815\/jlsrf-1-18"},{"volume-title":"The Lattice Boltzmann Method","year":"2016","author":"Kr\u00fcger T","key":"bibr41-1094342018767736"},{"key":"bibr42-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-65993-5"},{"key":"bibr43-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2014.62"},{"key":"bibr44-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2010.18"},{"volume-title":"MPI: A Message-Passing Interface Standard","year":"2015","author":"MPI Forum","key":"bibr45-1094342018767736"},{"key":"bibr46-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-09766-4"},{"key":"bibr47-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/s40571-015-0047-6"},{"key":"bibr48-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-58667-0_1"},{"key":"bibr49-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.compfluid.2018.01.023"},{"key":"bibr50-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1137\/15M1035240"},{"key":"bibr51-1094342018767736","unstructured":"Schornbaum F, R\u00fcde U (2017) Extreme-scale block-structured adaptive mesh refinement. CoRR abs\/1704.06829. Available at: http:\/\/arxiv.org\/abs\/1704.06829. Submitted to SIAMSISC."},{"key":"bibr52-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2006.5"},{"key":"bibr53-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/EURMIC.1998.711832"},{"key":"bibr54-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1016\/j.actamat.2015.09.047"},{"key":"bibr55-1094342018767736","first-page":"29","volume-title":"Proceedings of the 2008 high availability and performance computing workshop","volume":"4","author":"Taerat N","year":"2008"},{"key":"bibr56-1094342018767736","unstructured":"Treaster M (2005) A survey of fault-tolerance and fault-recovery techniques in parallel systems. CoRR abs\/cs\/0501002. Available at: http:\/\/arxiv.org\/abs\/cs\/0501002."},{"key":"bibr57-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1177\/1094342013490972"},{"key":"bibr58-1094342018767736","unstructured":"Yao E, Chen M, Wang R, et al. (2011) A new and efficient algorithm-based fault tolerance scheme for a million way parallelism. CoRR. Available at: http:\/\/arxiv.org\/abs\/1106.4213."},{"key":"bibr59-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1145\/361147.361115"},{"key":"bibr60-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/DSNW.2012.6264677"},{"key":"bibr61-1094342018767736","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2011.83"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342018767736","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342018767736","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342018767736","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T03:25:47Z","timestamp":1729049147000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342018767736"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,5,2]]},"references-count":63,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,7]]}},"alternative-id":["10.1177\/1094342018767736"],"URL":"https:\/\/doi.org\/10.1177\/1094342018767736","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"type":"print","value":"1094-3420"},{"type":"electronic","value":"1741-2846"}],"subject":[],"published":{"date-parts":[[2018,5,2]]}}}