Fscore = 99.2199% Precision = 99.6447% (41782/41931) Recall = 98.7988% (41782/42290) real 1m0.091s user 0m57.772s sys 0m0.947s

The scores didn’t impress us:

accuracy: 0.990596822671 AUC: 0.991292619197

StreamSVM outputs both binary predictions and probabilities, like this:

-1 0.810701 -1 0.920251 -1 0.89225

Note that these are probabilities for negative class, which may be misleading. It can be corrected in classifier.hpp, we sent the author a bug report. When you take these estimates into account, AUC matches VW!

AUC: 0.998972438313

Conclusion

Current version of Vowpal Wabbit outperforms both Liblinear and StreamSVM on webspam. That’s just one dataset, so it would be good to compare using a broader selection. On the other hand, the results show that one better take claims made in papers with a grain of salt, but you already knew that.

VW is also faster and allows using non-linear features like n-grams, which should score even better without any additional hassle. Its advantage is no surprise if you take into consideration that VW comes from a strong industrial lab, while the two other projects are of more academic quality.

We would like to thank Rajarshi Das and Peter Ritz, the main supporters of the Indiegogo campaign, and Gurjeet Singh, who was the first to contribute. You made this article possible.