Bi-LSTM: Bi-directional LSTMs (Bi-LSTM) were created as an improvement over the vanilla LSTMs in tasks where full sequences are present, and context is important [1]}. The idea behind Bi-LSTM is to present the information forward and backwards to two separate LSTM networks that are both connected to the same output layer. Having such bi-directional processing on the input information means that the network has, for each input (embedding or word), sequential and context information about it.
