<jupyter_start><jupyter_text>Word Embeddings Word Embeddings transform text into a sequence of vectors.Word embedding is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers. Conceptually it involves a mathematical embedding from a space with one dimension per word to a continuous vector space with a much lower dimension.(Source: [Wikipedia](https://en.wikipedia.org/wiki/Word_embedding)) ![Word2Vec](img/Word2Vec_web.png)  Embedding Layers Methods to generate this mapping include neural networks, dimensionality reduction on the word co-occurrence matrix, probabilistic models, explainable knowledge base method, and explicit representation in terms of the context in which words appear.Word and phrase embeddings, when used as the underlying input representation, have been shown to boost the performance in NLP tasks such as syntactic parsing and sentiment analysis.(Source: [Wikipedia](https://en.wikipedia.org/wiki/Word_embedding)) Keras offers an Embedding layer that can be used for neural networks on text data.It requires that the input data be integer encoded, so that each word is represented by a unique integer. This data preparation step can be performed using the Tokenizer API also provided with Keras.```python e = Embedding(200, 32, input_length=50)``` ![Word2VecAnn](img/word2vec_ann.png)  Create GenSim Vectors<jupyter_code>from gensim.models import word2vec

corpus = [
   "She loves you yeah yeah yeah",
   "see you later alligator",
   "see you later crocodile",
   "i just call to say i love you",
   "and it seems to me you lived your life like a candle in the wind",
   "baby you can drive my car",
   "we all live in the yellow submarine"
]

tokenized = [s.lower().split() for s in corpus]

wv = word2vec.Word2Vec(tokenized, 
                       size=7, 
                       window=5, 
                       min_count=1
                      )
words = list(wv.wv.vocab)
len(words)

print(wv['crocodile'])
wv.wv.most_similar('crocodile')<jupyter_output><empty_output><jupyter_text>Download Pretrained GenSim Vectors Code for downloading and unzipping the 'GoogleNews-vectors-negative300' model on your computer. if you experience problems, download the file manually at [s3.amazonaws.com](https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz) and proceed to the next code snippet ```pythonimport osfrom keras.utils import get_fileimport gensimimport subprocessMODEL = 'GoogleNews-vectors-negative300.bin'path = get_file(MODEL + '.gz', 'https://s3.amazonaws.com/dl4j-distribution/%s.gz' % MODEL)if not os.path.isdir('generated'):    os.mkdir('generated')unzipped = os.path.join('generated', MODEL)if not os.path.isfile(unzipped):    with open(unzipped, 'wb') as fout:        zcat = subprocess.Popen(['zcat'],                          stdin=open(path),                          stdout=fout                         )        zcat.wait()model = gensim.models.KeyedVectors.load_word2vec_format(unzipped, binary=True)```  Loading model from local folder<jupyter_code>import gensim

from gensim.models import KeyedVectors

model = gensim.models.KeyedVectors.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
model.most_similar(positive=['fernando'])<jupyter_output><empty_output><jupyter_text>Vector Addition<jupyter_code>def A_is_to_B_as_C_is_to(a, b, c, topn=1):
    a, b, c = map(lambda x:x if type(x) == list else [x],
                  (a, b, c))
    res = model.most_similar(positive=b + c, 
                             negative=a, 
                             topn=topn
                            )
    if len(res):
        if topn == 1:
            return res[0][0]
        return [x[0] for x in res]
    return None


for country in 'Italy', 'France', 'India', 'China':
    print('%s is the capital of %s' % 
          (A_is_to_B_as_C_is_to('Germany', 'Berlin', country), country))<jupyter_output>Rome is the capital of Italy
Paris is the capital of France
Delhi is the capital of India
Beijing is the capital of China
{'fraction_non_alphanumeric': 0.08950914340712224, 'fraction_numerical': 0.009865255052935515, 'mean_word_length': 3.588300220750552, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '514570', 'license': 'MIT', 'n_tokens_mistral': 1265, 'n_tokens_neox': 1198, 'n_words': 414, 'path': '1_Neural_Networks_Review/Word Embeddings.ipynb', 'repo_name': 'TheManohar/DLearn_Pro', 'size': '4156'}