<commit_before># word2gauss
Gaussian word embeddings

Python/Cython implementation of [<NAME> and <NAME>
<i>Word Representations via Gaussian Embedding</i>, ICLR 2015](http://arxiv.org/abs/1412.6623)
that represents each word as a multivariate Gaussian.
Scales to (relatively) large corpora using Cython extensions and threading
with asynchronous stochastic gradient descent (Adagrad).

## Getting started

### Installing
First install numpy, scipy and the packages in `requirements.txt`.
Then `sudo make install`.  `make test` runs the test suite.

NOTE: The performance sensitive parts of the code have been carefully
written in a way that allows gcc to auto-vectorize all the important loops.
Accordingly we recommend using gcc to compile and setting these
these flags for building:
```
export CFLAGS="-ftree-vectorizer-verbose=2 -O3 -ffast-math"
sudo -E bash -c "make install"
```

If you are using a Mac, gcc produces code approximately 2.5X faster then
the default clang compiler.  You can force the build to use gcc instead
of clang with:
```
# change these to the location of gcc -- note that /usr/bin/gcc is really
# clang in a default XCode installation
export CC=/usr/local/bin/gcc
export CXX=/usr/local/bin/g++
export CFLAGS="-ftree-vectorizer-verbose=2 -O3 -ffast-math"
sudo -E bash -c "make install"
```


### Code overview
The `GaussianEmbedding` class is the main workhorse for most tasks.  It
stores the model data, deals with serialization to/from files and
learns the parameters.  To allow embedding of non-word types like
hierarchies and entailment relations, `GaussianEmbedding` has
no knowledge of any vocabulary and operates only on
unique IDs.  Each ID is a `uint32` from `0 .. N-1` with `-1` signifying
an OOV token.

For learning word embeddings, the token - id mapping is off-loaded to a
`Vocabulary` class that translates streams of documents into `token_id` lists<commit_msg>Typos<commit_after># word2gauss
Gaussian word embeddings

Python/Cython implementation of [<NAME> and <NAME>
<i>Word Representations via Gaussian Embedding</i>, ICLR 2015](http://arxiv.org/abs/1412.6623)
that represents each word as a multivariate Gaussian.
Scales to (relatively) large corpora using Cython extensions and threading
with asynchronous stochastic gradient descent (Adagrad).

## Getting started

### Installing
First install numpy, scipy and the packages in `requirements.txt`.
Then `sudo make install`.  `make test` runs the test suite.

NOTE: The performance sensitive parts of the code have been carefully
written in a way that allows gcc to auto-vectorize all the important loops.
Accordingly we recommend using gcc to compile and setting these
flags for building:
```
export CFLAGS="-ftree-vectorizer-verbose=2 -O3 -ffast-math"
sudo -E bash -c "make install"
```

If you are using a Mac, gcc compiled code runs approximately 2.5X faster than
the default clang compiler.  You can force the build to use gcc instead
of clang with:
```
# change these to the location of gcc -- note that /usr/bin/gcc is really
# clang in a default XCode installation
export CC=/usr/local/bin/gcc
export CXX=/usr/local/bin/g++
export CFLAGS="-ftree-vectorizer-verbose=2 -O3 -ffast-math"
sudo -E bash -c "make install"
```


### Code overview
The `GaussianEmbedding` class is the main workhorse for most tasks.  It
stores the model data, deals with serialization to/from files and
learns the parameters.  To allow embedding of non-word types like
hierarchies and entailment relations, `GaussianEmbedding` has
no knowledge of any vocabulary and operates only on
unique IDs.  Each ID is a `uint32` from `0 .. N-1` with `-1` signifying
an OOV token.

For learning word embeddings, the token - id mapping is off-loaded to a
`Vocabulary` class that translates streams of documents into `token_id` lists
{'fraction_non_alphanumeric': 0.07738878652276915, 'fraction_numerical': 0.012634903922084759, 'mean_word_length': 4.547445255474453, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '584515', 'n_tokens_mistral': 1165, 'n_tokens_neox': 1049, 'n_words': 517}