RECURRENTNEURAL NETWORK REGULARIZATION
We present a simple regularization technique for Recurrent Neural Networks (RNNs) with Long Short Term Memory (LSTM) units. Dropout, t he most successful technique for regularizing neural networks, does n ot work well with RNNs and LSTMs. In this paper, we show how to correctly apply dropo ut t LSTMs, and show that it substantially reduces overfitting on a varie ty of tasks. These tasks include language modeling, speech recognition, image capt ion generation, and machine translation.