HeLI, a Word-Based Backoff Method for Language Identification
T Jauhiainen, K Lindén, H Jauhiainen - Proceedings of the VarDial Workshop, 2016
... We collected from the Common Crawl 3 corpus all the web pages from the respective domains as in Table 2. When language models were created directly from the pages, the accuracy on the DSL development corpus was 49.86%, which was much ... 3http://commoncrawl.org/ ...