The current state of the art on modeling both PTB and WikiText 2 [8] datasets as reported in <cite>[4]</cite> shows little sensitivity to hyper parameters; sharing almost all hyper parameters values between both datasets.