For reference, we made the same experiments with cross-entropy loss inside the pytorch-kaldi [1]}, [2]} framework. Here again, a layer of unidirectional UBRUs with backward recursion is able to compete with a fifth Li-GRU layer, both scoring an accuracy of 14.4\(\%\)  compared to 14.8\(\%\)  for four Li-GRU layers.
