A Stylometric Inquiry into Hyperpartisan and Fake News
We report on a comparative style analysis of hyperpartisan (extremely one sided) news and fake news. A corpus of 1,627 articles from 9 political publishers, three each from the mainstream, the hyperpartisan left, and the hyperpartisan right, have been fact checked by professional journalists at BuzzFeed: 97% of the 299 fake news articles identified are also hyperpartisan. We show how a style analysis can distinguish hyperpartisan news from the mainstream (F1 0.78) and satire from both (F1 0.81) But stylometry is no silver bullet as style based fake news detection does not work (F1 0.46) We further reveal that left wing and right wing news share significantly more stylistic similarities than either does with the mainstream. This result is robust: it has been confirmed by three different modeling approaches, one of which employs Unmasking in a novel way. Applications of our results include partisanship detection and pre screening for semi automatic fake news detection.
ACL