I want to train speech-to-text model on tensorflow.   Code at:https://github.com/bikramjitroy/speech-to-text/blob/master/TrainingUnitTest-CTCTrain.pyA sample audio.wav and label.txt file also present in repo.https://github.com/bikramjitroy/speech-to-textI am loading a wav file. Getting 20 mfcc feature per frame. Passing input to one layer RNN (256 hidden unit) and then added a fully connected layer and then using ctc_loss.The accuracy is not increasing even with 500 steps with one training example : Error = 70.932137   # Expecting it should go near zeroAccuracy = 0.62068963     # expecting this will go to zeroI have tried bi-directional rnn as well but that is diverging.What am I doing wrong in code?
Tensorflow speech-to-text training on single file with tf.nn.ctc_loss not converging
machine learning;python;tensorflow;rnn;machine translation
I have found the solution.Earlier I have a Dropout layer after single layer of RNN. I have changed the rnn_layer function as below. Removed the DropoutWrapper from the RNN layer - This helped me to overfitAdded 2 layers of LSTM cell. Single layer LSTM cell taking time toconverge but rather than making LSTM layer wide changed it to deep.This helped me achieve faster convergence than making network wider.Below is the code of rnn_layer.def rnn_layer(input_tensor, n_cell_units, dropout, seq_length, batch_size):    lstm_fw_cell = tf.contrib.rnn.BasicLSTMCell(2*n_cell_units, forget_bias=1.0, state_is_tuple=True)# Dropout layer dropped#    lstm_fw_cell = tf.contrib.rnn.DropoutWrapper(lstm_fw_cell, input_keep_prob=dropout, output_keep_prob=dropout)# Multi layer RNN    lstm_fw_cell = tf.contrib.rnn.MultiRNNCell([lstm_fw_cell] * 2, state_is_tuple=True)    outputs, output_states = tf.nn.dynamic_rnn(cell=lstm_fw_cell,                                               inputs=input_tensor,                                               dtype=tf.float32,                                               time_major=True,                                               sequence_length=seq_length)    return outputs