In this work we assess the performance of the AWD-LSTM model <cite>[4]</cite> for language modeling to better understand how relevant the published hyper parameters may be for a codemixed corpus and to isolate which hyper parameters could be further tuned to improve performance.