Abstract
In high-performance computing (HPC), using Linux containers can improve the convenience of system management and utilization, as well as facilitate the adoption of HPC DevOps and Clouds. However, utilizing containers with MPI applications poses some challenges in building portable MPI containers that can leverage various high-performance interconnect hardware available on different systems. A common solution is to replace the entire MPI stack in the container, but this may introduce compatibility issues between the host and container libraries. An alternative and more flexible solution is to inject host-shared libraries into the container and use them in conjunction with the container’s MPI stack, commonly referred to as a hybrid approach. In this paper, we propose a hybrid solution that uses the UCX communication framework as an intermediary between MPI and various high-end interconnects. This solution simplifies the container-building process while still achieving near-native communication performance. The flexibility of the proposed method was tested by performing a cross-version injection experiment. The results showed a good degree of forward/backward compatibility between versions, which can also refer to good portability as well. We evaluated the performance of our solution using communication micro-benchmarks and application benchmarks, and found that it achieved near-native communication performance. We also compared our solution with a similar solution using the Libfabric communication library, and confirmed that they have similar pros and cons with similar performance characteristics in most cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Balaji, P., et al.: PMI: a scalable parallel process-management interface for extreme-scale systems. In: Keller, R., Gabriel, E., Resch, M., Dongarra, J. (eds.) EuroMPI 2010. LNCS, vol. 6305, pp. 31–41. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15646-5_4
Castain, R.H., Solt, D., Hursey, J., Bouteiller, A.: PMIx: process management for exascale environments. In: Proceedings of the 24th European MPI Users’ Group Meeting, pp. 1–10 (2017)
Dongarra, J., Heroux, M.A., Luszczek, P.: High-performance conjugate-gradient benchmark: a new metric for ranking high-performance computing systems. Int. J. High Perform. Comput. Appl. 30(1), 3–10 (2016)
Grun, P., et al.: A brief introduction to the openfabrics interfaces-a new network API for maximizing high-performance application efficiency. In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, pp. 34–39. IEEE (2015)
Hursey, J.: Design considerations for building and running containerized MPI applications. In: 2020 2nd International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC), pp. 35–44. IEEE (2020)
Lindner, B., Petridis, L., Schulz, R., Smith, J.C.: Solvent-driven preferential association of lignin with regions of crystalline cellulose in molecular dynamics simulation. Biomacromol 14(10), 3390–3398 (2013). https://doi.org/10.1021/bm400442n. PMID: 23980921
Madonna, A., Aliaga, T.: Libfabric-based injection solutions for portable containerized MPI applications. In: 2022 IEEE/ACM 4th International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC), pp. 45–56. IEEE (2022)
PRACE - Partnership for Advanced Computing in Europe: Unified European Applications Benchmark Suite (2022). https://repository.prace-ri.eu/git/UEABS/ueabs/
Shamis, P., et al.: UCX: an open source framework for HPC network APIs and beyond. In: 2015 IEEE 23rd Annual Symposium on High-Performance Interconnects, pp. 40–43. IEEE (2015)
The Ohio State University: OSU Micro-Benchmarks 7.1 (2023). https://mvapich.cse.ohio-state.edu/benchmarks/
The Unified Communication X Library (2023). http://www.openucx.org
Van Der Spoel, D., Lindahl, E., Hess, B., Groenhof, G., Mark, A.E., Berendsen, H.J.: GROMACS: fast, flexible, and free. J. Comput. Chem. 26(16), 1701–1718 (2005)
Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple Linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Manatura, S. et al. (2024). Portable Containerized MPI Application Using UCX Replacement Method. In: Barolli, L. (eds) Advances on P2P, Parallel, Grid, Cloud and Internet Computing . 3PGCIC 2023. Lecture Notes on Data Engineering and Communications Technologies, vol 189. Springer, Cham. https://doi.org/10.1007/978-3-031-46970-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-031-46970-1_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46969-5
Online ISBN: 978-3-031-46970-1
eBook Packages: EngineeringEngineering (R0)