Unfortunately there are many limitations to the model proposed by<cite> Collobert et al. (2011b)</cite> . First, it uses a simple feed-forward neural network, which restricts the use of context to a fixed sized window around each word -an approach that discards useful long-distance relations between words. Second, by depending solely on word embeddings, it is unable to exploit explicit character level features such as prefix and suffix, which could be useful especially with rare words where word embeddings are poorly trained. We seek to address these issues by proposing a more powerful neural network model.