An efficient parallel sorting algorithm for shared memory multiprocessors
Abstract A new parallel sorting algorithm, Parasort, based upon Quicksort and two-way merging and suitable for implementation on shared memory multiprocessors is presented. An asynchronous parallel partitioning algorithm is used to distribute the work evenly amongst the processors during merging to ensure an even load balance which is crucial if high efficiency is to be achieved. The implementation of this parallel sorting algorithm on the Balance 8000 multiprocessor exhibits good speedup rates approaching linearity when compared to sequential quicksort.
Recent works show that ordering of the training data affects the model performance for Neural Machine Translation. Several approaches involving dynamic data ordering and data sharding based on curriculum learning have been analysed for the their performance gains and faster convergence. In this work we propose to empirically study several ordering approaches for the training data based on different metrics and evaluate their impact on the model performance. Results from our study show that pre-fixing the ordering of the training data based on perplexity scores from a pre-trained model performs the best and outperforms the default approach of randomly shuffling the training data every epoch.