<jupyter_start><jupyter_text>Word Embedding In Natural language Processing<jupyter_code># https://www.youtube.com/watch?v=kKDYtZfriI8&list=PLmPJQXJiMoUUSqSV7jcqGiiypGmQ_ogtb&index=11
from IPython.display import Image
Image("Why.png")<jupyter_output><empty_output><jupyter_text>In this video, I will explain how to use all options in Python - Step By Step  Option 2 - Training own word2vec model<jupyter_code># define tokenized senences as training data
tokenized_sentences = [['Hello','This','is','python','training','by','Aman'],
             ['Hello','This','is','Java','training','by','Aman'],
             ['Hello','This','is','Data Science','training','by','Unfold','Data','Science'],
             ['Hello','This','is','programming','training','']]
# training word2vec model
from gensim.models import Word2Vec
import warnings
warnings.filterwarnings('ignore')
mymodel = Word2Vec(tokenized_sentences, min_count=1)
# summarizing the loaded model
print(mymodel)
# summarize vocabulary
words = list(mymodel.wv.vocab)
# summarize vocabulary
print(words)
# access word vector for one word "training"
# print(mymodel['Hello'])
print(mymodel['Data Science'])
#try finding most similar words for word "Data"
# mymodel.most_similar("laptop")  # This will diplay no word found
# mymodel.most_similar("Data")
mymodel.most_similar("Data Science")
# mymodel.similarity('python', 'training') # 0.064438656
mymodel.similarity('python', 'Data Science')<jupyter_output><empty_output><jupyter_text>First Task for you - comment me how you can solve a document classification problem using above concept<jupyter_code>#####################################Part 2###################################################<jupyter_output><empty_output><jupyter_text>Option 3 - Create Embedding model using Keras Embedding<jupyter_code>from numpy import array
from keras.preprocessing.text import one_hot
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.embeddings import Embedding
# define documents
Sent = ['Hello, how are you',
        'how are you',
        'how are you doing',
        'I am doing great',
        'I am doing good',
        'I am good']
# defining class labels
sent_labels = array([1,1,1,0,0,0])
# integer encoding of the documents
my_vocab_size = 30 # This is vector size default is 100 but here taken 30
encoded_sent = [one_hot(i, my_vocab_size) for i in Sent]
print(encoded_sent)
# padding documents to a max length =5 
length = 5
padded_sent = pad_sequences(encoded_sent, maxlen=length, padding='pre')
print(padded_sent)
# # defining the model
# mymodel = Sequential()
# mymodel.add(Embedding(my_vocab_size, 8, input_length=max_length))
# mymodel.add(Flatten())
# mymodel.add(Dense(1, activation='sigmoid'))
# defining the model
mymodel = Sequential()
mymodel.add(Embedding(my_vocab_size, 8, input_length=length))
mymodel.add(Flatten())
mymodel.add(Dense(1, activation='sigmoid'))
# compiling the model
mymodel.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# # fiting  the model
# mymodel.fit(padded_docs, labels, epochs=30)

# # evaluate the model
# modelloss, modelaccuracy = mymodel.evaluate(padded_docs, labels, verbose=0)
# print('Accuracy: %f' % (modelaccuracy*100))
padded_docs = padded_sent 
labels = sent_labels 
# fiting  the model
mymodel.fit(padded_docs, labels, epochs=30)

# evaluate the model
modelloss, modelaccuracy = mymodel.evaluate(padded_docs, labels, verbose=0)
print('Accuracy: %f' % (modelaccuracy*100))<jupyter_output>Epoch 1/30
6/6 [==============================] - 0s 26ms/step - loss: 0.6854 - accuracy: 0.8333
Epoch 2/30
6/6 [==============================] - 0s 665us/step - loss: 0.6832 - accuracy: 0.8333
Epoch 3/30
6/6 [==============================] - 0s 665us/step - loss: 0.6811 - accuracy: 0.8333
Epoch 4/30
6/6 [==============================] - 0s 665us/step - loss: 0.6789 - accuracy: 0.8333
Epoch 5/30
6/6 [==============================] - 0s 665us/step - loss: 0.6767 - accuracy: 1.0000
Epoch 6/30
6/6 [==============================] - 0s 499us/step - loss: 0.6745 - accuracy: 1.0000
Epoch 7/30
6/6 [==============================] - 0s 665us/step - loss: 0.6723 - accuracy: 1.0000
Epoch 8/30
6/6 [==============================] - 0s 498us/step - loss: 0.6702 - accuracy: 1.0000
Epoch 9/30
6/6 [==============================] - 0s 499us/step - loss: 0.6680 - accuracy: 1.0000
Epoch 10/30
6/6 [==============================] - 0s 831us/step - loss: 0.6658 - accuracy: 1.0000
Epoch 11/30
6/6 [===[...]<jupyter_text>The Prediction part<jupyter_code>mysent_to_predict = ['how are you Suman',
        'I am good']
# integer encode the documents
vocab_size = 30
encoded = [one_hot(d, vocab_size) for d in mysent_to_predict]
print(encoded)

# pad documents to a max length of 5 words
max_length = 5
mypadded = pad_sequences(encoded, maxlen=max_length, padding='pre')
print(mypadded)
mymodel.predict_classes(mypadded)<jupyter_output><empty_output><jupyter_text>Option 1 - Using Pre Trained Word2Vec Model<jupyter_code>from gensim.models import KeyedVector   
import os 
os.chdir("D:\\01_DATA_SCIENCE_FINAL\\D-00000-NLP\\NLP-CODES\\AMAN-NLP-CODES\\AMAN_NLP_VIMP-CODE\\Project-9_Word_Embedding_Python_using_Word2vec_and_keras\\")
#Download GoogleNews-vectors-negative300.bin from 
#https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz
PreTrainedModel = KeyedVectors.load_word2vec_format("GoogleNews-vectors-negative300.bin", binary=True)
# calculate: (king - man) + woman = ?
result = PreTrainedModel.most_similar("Data")
print(result)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1893033075299085, 'fraction_numerical': 0.048557353976073185, 'mean_word_length': 5.8329326923076925, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '398221', 'license': 'Unlicense', 'n_tokens_mistral': 2139, 'n_tokens_neox': 1898, 'n_words': 450, 'path': 'AMAN_NLP_VIMP-CODE/Project-6_Word_Embedding_Python_using_Word2vec_and_keras/Creating Word Embedding Using Gensim_Aman.ipynb', 'repo_name': 'vitthalkcontact/NLP', 'size': '5684'}