DeepSpark: Spark Based Deep Learning Supporting Asynchronous Updates and Caffe Compatibility
The increasing complexity of deep neural networks (DNNs) has made it challenging to exploit existing large scale data process pipelines for handling massive data and parameters involved in DNN training. Distributed computing platforms and GPGPU based acceleration provide a mainstream solution to this computational challenge. In this paper, we propose DeepSpark, a distributed and parallel deep learning framework that simultaneously exploits Apache Spark for large scale distributed data management and Caffe for GPU based acceleration. DeepSpark directly accepts Caffe input specifications, providing seamless compatibility with existing designs and network structures. To support parallel operations, DeepSpark automatically distributes workloads and parameters to Caffe running nodes using Spark and iteratively aggregates training results by a novel lock free asynchronous variant of the popular elastic averaging stochastic gradient descent (SGD) update scheme, effectively complementing the synchronized processing capabilities of Spark. DeepSpark is an on going project, and the current release is available at this http URL
ArXiv