As shown by this table, the output of AWD-LSTM is restricted to D 3 7 . In contrast, AWD-LSTM-MoS<cite> (Yang et al., 2018)</cite> and AWD-LSTM-DOC outputted matrices whose ranks equal the vocabulary size. This fact indicates that DOC (including MoS) can output the same matrix as the true distributions in view of a rank.