<jupyter_start><jupyter_text>IMDB -Movie Reviews Sentiment Classification* Word Embeddings with KerasWord embeddings are a type of word representation that allows words with similar meaning to have a similar representation.It is a distributed representation for text that is perhaps one of the key breakthroughs for the impressive performance of deep learning methods on challenging natural language processing problems.We willl use the imdb dataset to learn word embeddings as we train our dataset. This dataset contains 25,000 movie reviews from IMDB, labeled with sentiment (positive or negative). For convenience, the words are indexed by their frequency in the dataset, meaning the for that has index 1 is the most frequent word. We will only use the first 20 words from each review to speed up training, use a max vocab size of 10,000.<jupyter_code>from keras.datasets import imdb
from keras.preprocessing.sequence import pad_sequences
vocab_size = 10000 #vocab size
maxlen = 300  #number of word used from each review
#load dataset as a list of ints
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=vocab_size)
#make all sequences of the same length
x_train = pad_sequences(x_train, maxlen=maxlen)
x_test =  pad_sequences(x_test, maxlen=maxlen)
x_train.shape #number of review, number of words in each review<jupyter_output><empty_output><jupyter_text>To take a look at the review and sentiment:<jupyter_code>(training_data, training_labels), (test_data, test_labels)= imdb.load_data(num_words=vocab_size, index_from=3)
word_to_id = imdb.get_word_index()
word_to_id = {k:(v+3) for k,v in word_to_id.items()}
word_to_id["<PAD>"] = 0
word_to_id["<START>"] = 1
word_to_id["<UNK>"] = 2
id_to_word = {value:key for key,value in word_to_id.items()}
print(' '.join(id_to_word[id] for id in training_data[6] ))
print('The sentiment is:', training_labels[6])<jupyter_output><START> lavish production values and solid performances in this straightforward adaption of jane <UNK> satirical classic about the marriage game within and between the classes in <UNK> 18th century england northam and paltrow are a <UNK> mixture as friends who must pass through <UNK> and lies to discover that they love each other good humor is a <UNK> virtue which goes a long way towards explaining the <UNK> of the aged source material which has been toned down a bit in its harsh <UNK> i liked the look of the film and how shots were set up and i thought it didn't rely too much on <UNK> of head shots like most other films of the 80s and 90s do very good results
The sentiment is: 1<jupyter_text>Here is a postive review number 6 from the training set. From the word *jane* we can guess that it is one of the works of Jane Austen, perhaps Pride and Prejudice or Sense and Sensibility?  Build Keras Embedding Layer ModelWe can think of the Embedding layer as a dicionary that maps a index assigned to a word to a word vector. This layer is very flexiable and can be used in a few ways:* The embedding layer can be used at the start of a larger deep learning model. * Also we could load pre-train word embeddings into the embedding layer when we create our model.* Use the embedding layer to train our own word2vec models.The keras embedding layer doesn't require us to onehot encode our words, instead we have to give each word a unqiue intger number as an id. For the imdb dataset we've loaded this has already been done, but if this wasn't the case we could use sklearn [LabelEncoder](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html).<jupyter_code>import time
from keras.models import Sequential
from keras.layers import Flatten, Dense, Embedding
from keras.layers.convolutional import Convolution1D, MaxPooling1D
model = Sequential()
model.add(Embedding(10000, 8, input_length=maxlen)) #10000 for vocab size, 8 for dimensionality of embedding
model.add(Convolution1D(nb_filter=32, filter_length=3, border_mode='same', activation='relu'))
model.add(MaxPooling1D(pool_length=300)) #duplicate
model.add(Flatten())
model.add(Dense(250, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'])
model.summary()
x_train.shape #number of examples, number or words
x_train[1] # words are representedby numbers
y_train.shape 
y_test.shape
start = time.clock()
history = model.fit(x_train, y_train, epochs=5, batch_size=128, validation_data=(x_test, y_test))
end = time.clock()
print('Time spent:', end-start)
model.layers #get all layers from model
score = model.evaluate(x_test, y_test)
print('Trainin Acc: %.2f%%' %(score[0]*100),
     '\nTest Acc: %.2f%%' %(score[1]*100))<jupyter_output>Trainin Acc: 37.72% 
Test Acc: 86.42%
{'fraction_non_alphanumeric': 0.0785472972972973, 'fraction_numerical': 0.018158783783783782, 'mean_word_length': 5.757489300998573, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '333718', 'license': 'MIT', 'n_tokens_mistral': 1469, 'n_tokens_neox': 1395, 'n_words': 610, 'path': 'IMDB_reviews_sentiment_classify.ipynb', 'repo_name': 'jsrpy/Sentiment_Analysis_WordEmbedding', 'size': '4736'}