Abstract
Complex workflows play a critical role in accelerating scientific discovery. In many scientific domains, efficient workflow management can lead to faster scientific output and broader user groups. Workflows that can leverage resources across the boundary between cloud and HPC are a strong driver for the convergence of HPC and cloud. This study investigates the transition and deployment of a GPU-accelerated molecular docking workflow that was designed for HPC systems onto a cloud-native environment with Kubernetes and Apache Airflow. The case study focuses on state-of-of-the-art molecular docking software for drug discovery. We provide a DAG-based implementation in Apache Airflow and technical details for GPU-accelerated deployment. We evaluated the workflow using the SWEETLEAD bioinformatics dataset and executed it in a Cloud environment with heterogeneous computing resources. Our workflow can effectively overlap different stages when mapped onto different computing resources.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Adorf, C.S., Dodd, P.M., Ramasubramani, V., Glotzer, S.C.: Simple data and workflow management with the signac framework. Comput. Mater. Sci. 146, 220–229 (2018)
Beltre, A.M., Saha, P., Govindaraju, M., Younge, A., Grant, R.E.: Enabling HPC workloads on cloud infrastructure using Kubernetes container orchestration mechanisms. In: 2019 IEEE/ACM International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC), pp. 11–20. IEEE (2019)
Deelman, E., et al.: Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Sci. Program. 13(3), 219–237 (2005)
Deelman, E., Vahi, K., Rynge, M., Juve, G., Mayani, R., Da Silva, R.F.: Pegasus in the cloud: science automation through workflow technologies. IEEE Internet Comput. 20(1), 70–76 (2016)
Di Tommaso, P., Chatzou, M., Floden, E.W., Barja, P.P., Palumbo, E., Notredame, C.: Nextflow enables reproducible computational workflows. Nat. Biotechnol. 35(4), 316–319 (2017)
Hasham, K., et al.: CMS workflow execution using intelligent job scheduling and data access strategies. IEEE Trans. Nucl. Sci. 58(3), 1221–1232 (2011)
Jain, A., et al.: Fireworks: a dynamic workflow system designed for high-throughput applications. Concurr. Comput. Pract. Exp. 27(17), 5037–5059 (2015)
LeGrand, S., et al.: GPU-accelerated drug discovery with docking on the summit supercomputer: porting, optimization, and application to COVID-19 research. In: Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics. BCB ’20, ACM (2020)
Markidis, S., Gadioli, D., Vitali, E., Palermo, G.: Understanding the I/O impact on the performance of high-throughput molecular docking. In: 2021 IEEE/ACM Sixth International Parallel Data Systems Workshop (PDSW), pp. 9–14. IEEE (2021)
Merzky, A., Santcroos, M., Turilli, M., Jha, S.: Radical-pilot: scalable execution of heterogeneous and dynamic workloads on supercomputers. CoRR, abs/1512.08194 (2015)
Misale, C. et al. Towards Standard Kubernetes Scheduling Interfaces for Converged Computing. In: Nichols, J., et al. (eds.) Driving Scientific and Engineering Discoveries Through the Integration of Experiment, Big Data, and Modeling and Simulation. SMC 2021. CCIS, vol. 1512, pp. 310–326. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96498-6_18
Morris, G.M., et al.: Automated docking using a Lamarckian genetic algorithm and an empirical binding free energy function. J. Comput. Chem. 19(14), 1639–1662 (1998)
Novick, P.A., Ortiz, O.F., Poelman, J., Abdulhay, A.Y., Pande, V.S.: SWEETLEAD: an in silico database of approved drugs, regulated chemicals, and herbal isolates for computer-aided drug discovery. PLOS ONE 8(11) (2013)
Rogers, D.: ORNL large-scale docking workflow. https://code.ornl.gov/99R/launchad/-/tree/master
Rosa, M.J., Ralha, C.G., Holanda, M., Araujo, A.P.: Computational resource and cost prediction service for scientific workflows in federated clouds. Futur. Gener. Comput. Syst. 125, 844–858 (2021)
Saha, P., Beltre, A., Uminski, P., Govindaraju, M.: Evaluation of docker containers for scientific workloads in the cloud. In: Proceedings of the Practice and Experience on Advanced Research Computing, pp. 1–8 (2018)
Santos-Martins, D., Solis-Vasquez, L., Tillack, A.F., Sanner, M.F., Koch, A., Forli, S.: Accelerating AutoDock4 with GPUs and gradient-based local search. J. Chem. Theory Comput. 17(2), 1060–1073 (2021)
Schieffer, G., Peng, I.: Accelerating drug discovery in AutoDock-GPU with tensor cores. In: Euro-Par 2023: Parallel Processing: 29th International European Conference on Parallel and Distributed Computing (Euro-Par 2023), Proceedings. LNCS, vol. 14100, pp. 1–15. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-39698-4_41
Trifan, A., et al.: Intelligent resolution: integrating Cryo-EM with AI-driven multi-resolution simulations to observe the severe acute respiratory syndrome coronavirus-2 replication-transcription machinery in action. Int. J. High Perform. Comput. Appl. 36(5–6), 603–623 (2022)
Venkatraman, V., et al.: Drugsniffer: an open source workflow for virtually screening billions of molecules for binding affinity to protein targets. Front. Pharmacol. 13 (2022)
Wolstencroft, K., et al.: The taverna workflow suite: designing and executing workflows of web services on the desktop, web or in the cloud. Nucleic Acids Res. 41(W1), W557–W561 (2013)
Acknowledgment
This research is supported by the European Commission under the Horizon project OpenCUBE (GA-101092984).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Medeiros, D., Schieffer, G., Wahlgren, J., Peng, I. (2023). A GPU-Accelerated Molecular Docking Workflow with Kubernetes and Apache Airflow. In: Bienz, A., Weiland, M., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023. Lecture Notes in Computer Science, vol 13999. Springer, Cham. https://doi.org/10.1007/978-3-031-40843-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-40843-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-40842-7
Online ISBN: 978-3-031-40843-4
eBook Packages: Computer ScienceComputer Science (R0)