We proposed Direct Output Connection (DOC), a generalization method of MoS introduced by <cite>Yang et al. (2018)</cite> . DOC raises the expressive power of RNN language models and improves quality of the model.