Neural sequence-to-sequence models are applied to automatic speech recognition with a great success [1]} and this model can be easily extended for the multilingual scenario in which the training data consists of multiple languages [2]}, [3]}, [4]}. These models are able to improve quality especially for the lower-resource languages since the acoustic representation can benefit from sharing with other languages. However, the common assumption in such multilingual works so far is that, the datasets for all languages are available before the training process, so that the model has access to the combination of the data for training. The standard batch-training process involves collecting data and sampling batches to the model often tries to balances the amount of data per language given to the model.
