<jupyter_start><jupyter_text>WordRank wrapper tutorial on Lee CorpusWordRank is a new word embedding algorithm which captures the semantic similarities in a text data well. See this [notebook](https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/Wordrank_comparisons.ipynb) for it's comparisons to other popular embedding models. This tutorial will serve as a guide to use the WordRank wrapper in gensim. You need to install [WordRank](https://bitbucket.org/shihaoji/wordrank) before proceeding with this tutorial. Train modelWe'll use [Lee corpus](https://github.com/RaRe-Technologies/gensim/blob/develop/gensim/test/test_data/lee.cor) for training which is already available in gensim. Now for Wordrank, two parameters `dump_period` and `iter` needs to be in sync as it dumps the embedding file with the start of next iteration. For example, if you want results after 10 iterations, you need to use `iter=11` and `dump_period` can be anything that gives mod 0 with resulting iteration, in this case 2 or 5.<jupyter_code>from gensim.models.wrappers import Wordrank

wr_path = 'wordrank' # path to Wordrank directory
out_dir = 'model' # name of output directory to save data to
data = '../../gensim/test/test_data/lee.cor' # sample corpus

model = Wordrank.train(wr_path, data, out_dir, iter=11, dump_period=5)<jupyter_output><empty_output><jupyter_text>Now, you can use any of the Keyed Vector function in gensim, on this model for further tasks. For example,<jupyter_code>model.most_similar('President')
model.similarity('President', 'military')<jupyter_output><empty_output><jupyter_text>As Wordrank provides two sets of embeddings, the word and context embedding, you can obtain their addition by setting ensemble parameter to 1 in the train method. Save and Load modelsIn case, you have trained the model yourself using demo scripts in Wordrank, you can then simply load the embedding files in gensim. Also, Wordrank doesn't return the embeddings sorted according to the word frequency in corpus, so you can use the sorted_vocab parameter in the load method. But for that, you need to provide the vocabulary file generated in the 'matrix.toy' directory(if you used default names in demo) where all the metadata is stored.<jupyter_code>wr_word_embedding = 'wordrank.words'
vocab_file = 'vocab.txt'

model = Wordrank.load_wordrank_model(wr_word_embedding, vocab_file, sorted_vocab=1)<jupyter_output><empty_output><jupyter_text>If you want to load the ensemble embedding, you similarly need to provide the context embedding file and set ensemble to 1 in `load_wordrank_model` method.<jupyter_code>wr_context_file = 'wordrank.contexts'
model = Wordrank.load_wordrank_model(wr_word_embedding, vocab_file, wr_context_file, sorted_vocab=1, ensemble=1)<jupyter_output><empty_output><jupyter_text>You can save these sorted embeddings using the standard gensim methods.<jupyter_code>from tempfile import mkstemp

fs, temp_path = mkstemp("gensim_temp")  # creates a temp file
model.save(temp_path)  # save the model<jupyter_output><empty_output><jupyter_text>Evaluating modelsNow that the embeddings are loaded in Word2Vec format and sorted according to the word frequencies in corpus, you can use the evaluations provided by gensim on this model.For example, it can be evaluated on following Word Analogies and Word Similarity benchmarks.<jupyter_code>word_analogies_file = 'datasets/questions-words.txt'
model.accuracy(word_analogies_file)
word_similarity_file = 'datasets/ws-353.txt'
model.evaluate_word_pairs(word_similarity_file)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07045009784735812, 'fraction_numerical': 0.005311713726586525, 'mean_word_length': 6.316973415132924, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '36557', 'license': 'MIT', 'n_tokens_mistral': 1076, 'n_tokens_neox': 1026, 'n_words': 390, 'path': 'gensim/docs/notebooks/WordRank_wrapper_quickstart.ipynb', 'repo_name': 'YantianZha/Distr2Vec', 'size': '3577'}