MDLoader: A Hybrid Model-driven Data Loader for Distributed Deep Neural Networks Training
In this work, we propose MD Loader, a hybrid in-memory data loader for distributed deep neural networks. MDLoader introduces a model-driven performance estimator to automatically switch between one-sided and collective communication at runtime.
- Research Organization:
- Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC05-00OR22725
- OSTI ID:
- 2428083
- Resource Relation:
- Conference: 2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) - San Francisco, California, United States of America - 5/27/2024 12:00:00 PM-5/31/2024 12:00:00 PM
- Country of Publication:
- United States
- Language:
- English
Similar Records
A gradient-based deep neural network model for simulating multiphase flow in porous media
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
Deep neural network enabled corrective source term approach to hybrid analysis and modeling
Journal Article
·
Tue May 10 00:00:00 EDT 2022
· Journal of Computational Physics
·
OSTI ID:2428083
+2 more
SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks
Conference
·
Wed Feb 28 00:00:00 EST 2018
·
OSTI ID:2428083
+5 more
Deep neural network enabled corrective source term approach to hybrid analysis and modeling
Journal Article
·
Fri Nov 26 00:00:00 EST 2021
· Neural Networks
·
OSTI ID:2428083
+1 more