To achieve this, we directly train an LSTM with a cosine similarity loss between the output embedding of the network and language representation of the ground truth completion. Such an approach integrates more tightly with the multi-choice filling the blanks task, and significantly outperforms the prior CNN+LSTM method <cite>[32]</cite> .