@inproceedings{
unsal2023flextrain,
title={FlexTrain: A Dynamic Training Framework for Heterogeneous Devices Environments},
author={Mert Unsal and Ali Maatouk and Antonio De Domenico and Nicola Piovesan and Fadhel Ayed},
booktitle={Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@NeurIPS 2023)},
year={2023},
url={https://openreview.net/forum?id=ommktfjGRp}
}
As deep learning models become increasingly large, they pose significant challenges in heterogeneous devices environments. The size of deep learning models makes it difficult to deploy them on low-power or resource-constrained devices, leading to long inference times and high energy consumption. To address these challenges, we propose FlexTrain, a framework that accommodates the diverse storage and computational resources available on different devices during the training phase. FlexTrain enables efficient deployment of deep learning models, while respecting device constraints, minimizing communication costs, and ensuring seamless integration with diverse devices. We demonstrate the effectiveness of FlexTrain on the CIFAR-100 dataset, where a single global model trained with FlexTrain can be easily deployed on heterogeneous devices, saving training time and energy consumption. We also extend FlexTrain to the federated learning setting, showing that our approach outperforms standard federated learning benchmarks on both CIFAR-10 and CIFAR-100 datasets.
FlexTrain: A Dynamic Training Framework for Heterogeneous Devices Environments
Mert Unsal Ali Maatouk Antonio De Domenico Nicola Piovesan Fadhel Ayed