<commit_before>Changes
=======

0.8.10

* word2vec load_word2vec_format uses less memory (Yves Raimond)
* save/load automatically single out large arrays + allow mmap
* HDP estimation on new documents (<NAME>)
* store labels in SvmLight corpus (Ritesh)
* fix word2vec binary load on Windows (<NAME>)
* replace numpy.svd with scipy.svd for more stability (<NAME>)

0.8.9

* use travis-ci for continuous integration
* auto-optimize LDA asymmetric prior (<NAME>)
* update for new word2vec binary format (<NAME>)
* doc rendering fix (<NAME>)
* better LDA perplexity logging
* fix Pyro thread leak in distributed algos (<NAME>)
* optimizations in word2vec (<NAME>)
* allow compressed input in LineSentence corpus (<NAME>)
* upgrade ez_setup, doc improvements, minor fixes etc.

0.8.8 (aka "word2vec release")

* python3 port by <NAME>: https://github.com/samantp/gensimPy3
* massive optimizations to word2vec (cython, BLAS, multithreading): ~20x-300x speedup
* new word2vec functionality (thx to <NAME>, PR #124)
* new CSV corpus class (thx to Zygmunt Zając)
* corpus serialization checks to prevent overwriting (by <NAME>, PR #125)
* add context manager support for older Python<=2.6 for gzip and bz2
* added unittests for word2vec

0.8.7<commit_msg>add josephcc to contributors<commit_after>Changes
=======

0.8.10

* better Wikipedia article parsing (<NAME>)
* word2vec load_word2vec_format uses less memory (Yves Raimond)
* save/load automatically single out large arrays + allow mmap
* HDP estimation on new documents (<NAME>)
* store labels in SvmLight corpus (Ritesh)
* fix word2vec binary load on Windows (<NAME>)
* replace numpy.svd with scipy.svd for more stability (<NAME>)

0.8.9

* use travis-ci for continuous integration
* auto-optimize LDA asymmetric prior (<NAME>)
* update for new word2vec binary format (<NAME>)
* doc rendering fix (<NAME>)
* better LDA perplexity logging
* fix Pyro thread leak in distributed algos (<NAME>)
* optimizations in word2vec (<NAME>)
* allow compressed input in LineSentence corpus (<NAME>)
* upgrade ez_setup, doc improvements, minor fixes etc.

0.8.8 (aka "word2vec release")

* python3 port by <NAME>: https://github.com/samantp/gensimPy3
* massive optimizations to word2vec (cython, BLAS, multithreading): ~20x-300x speedup
* new word2vec functionality (thx to Ghassen Hamrouni, PR #124)
* new CSV corpus class (thx to Zygmunt Zając)
* corpus serialization checks to prevent overwriting (by <NAME>, PR #125)
* add context manager support for older Python<=2.6 for gzip and bz2
* added unittests for word2vec

0.8.7
{'fraction_non_alphanumeric': 0.09957048028114018, 'fraction_numerical': 0.02967590784849668, 'mean_word_length': 5.028235294117647, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 27, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4281148', 'n_tokens_mistral': 845, 'n_tokens_neox': 791, 'n_words': 320}