<jupyter_start><jupyter_text>Setup data We're going to look at the IMDB dataset, which contains movie reviews from IMDB, along with their sentiment. Keras comes with some helpers for this dataset.<jupyter_code>from keras.datasets import imdb
idx = imdb.get_word_index()<jupyter_output><empty_output><jupyter_text>This is the word list:<jupyter_code>idx_arr = sorted(idx, key=idx.get)
idx_arr[:10]<jupyter_output><empty_output><jupyter_text>...and this is the mapping from id to word<jupyter_code>idx2word = {v: k for k, v in idx.iteritems()}<jupyter_output><empty_output><jupyter_text>We download the reviews using code copied from keras.datasets:<jupyter_code>path = get_file('imdb_full.pkl',
                origin='https://s3.amazonaws.com/text-datasets/imdb_full.pkl',
                md5_hash='d091312047c43cf9e4e38fef92437263')
f = open(path, 'rb')
(x_train, labels_train), (x_test, labels_test) = pickle.load(f)
len(x_train)<jupyter_output><empty_output><jupyter_text>Here's the 1st review. As you see, the words have been replaced by ids. The ids can be looked up in idx2word.<jupyter_code>', '.join(map(str, x_train[0]))<jupyter_output><empty_output><jupyter_text>The first word of the first review is 23022. Let's see what that is.<jupyter_code>idx2word[23022]<jupyter_output><empty_output><jupyter_text>Here's the whole review, mapped from ids to words.<jupyter_code>' '.join([idx2word[o] for o in x_train[0]])<jupyter_output><empty_output><jupyter_text>The labels are 1 for positive, 0 for negative.<jupyter_code>train = [' '.join([idx2word[o] for o in review]) for review in x_train]
test = [' '.join([idx2word[o] for o in review]) for review in x_test]
train[0][:20]
labels_train[:10]<jupyter_output><empty_output><jupyter_text>Reduce vocab size by setting rare words to max index.<jupyter_code># vocab_size = 5000

# trn = [np.array([i if i<vocab_size-1 else vocab_size-1 for i in s]) for s in x_train]
# test = [np.array([i if i<vocab_size-1 else vocab_size-1 for i in s]) for s in x_test]
train[0].split('to')
%%time
count_vect = CountVectorizer(lowercase=True,
                             stop_words='english', 
                             ngram_range=(1, 1), analyzer='word', 
                             max_df=1.0, min_df=1, max_features=5000,
                             binary=False)
X_train_counts = count_vect.fit_transform(train)
print (X_train_counts.shape)
X_test_counts = count_vect.transform(test)

for c in [.01, .05, .1, .2]:
    clf = LinearSVC(C=c, max_iter=5000)
    clf.fit(X_train_counts, labels_train)

    pred = clf.predict(X_test_counts)

    print (c, accuracy_score(pred, labels_test))<jupyter_output>(25000, 5000)
0.1 0.84036
0.5 0.82284
1 0.8142
2 0.80708
5 0.80212
10 0.79896
CPU times: user 1min 14s, sys: 332 ms, total: 1min 15s
Wall time: 1min 15s<jupyter_text>Results for 1-gram + linear SVM<jupyter_code>count_vect = CountVectorizer(lowercase=True,
                             stop_words='english', 
                             ngram_range=(1, 1), analyzer='word', 
                             max_df=1.0, min_df=1, max_features=5000,
                             binary=False)
X_train_counts = count_vect.fit_transform(train)
print (X_train_counts.shape)
X_test_counts = count_vect.transform(test)

for c in [0.0005, .001, .002, .005, .01]:
    clf = LinearSVC(C=c, max_iter=5000)
    clf.fit(X_train_counts, labels_train)

    pred = clf.predict(X_test_counts)

    print ('c =',c, ' acc = ', accuracy_score(pred, labels_test))<jupyter_output>c = 0.0005  acc =  0.86852
c = 0.001  acc =  0.87272
c = 0.002  acc =  0.875
c = 0.005  acc =  0.8702
c = 0.01  acc =  0.86676<jupyter_text>Results for 1,2-gram + linear SVM<jupyter_code>count_vect = CountVectorizer(lowercase=True,
                             stop_words='english', 
                             ngram_range=(1, 2), analyzer='word', 
                             max_df=1.0, min_df=1, max_features=5000,
                             binary=False)
X_train_counts = count_vect.fit_transform(train)
print (X_train_counts.shape)
X_test_counts = count_vect.transform(test)

for c in [ .001, .01, .1, 1]:
    clf = LinearSVC(C=c, max_iter=5000)
    clf.fit(X_train_counts, labels_train)

    pred = clf.predict(X_test_counts)

    print ('c =',c, ' acc = ', accuracy_score(pred, labels_test))
count_vect = CountVectorizer(lowercase=True,
                             stop_words='english', 
                             ngram_range=(2, 2), analyzer='word', 
                             max_df=1.0, min_df=1, max_features=10000,
                             binary=False)
X_train_counts = count_vect.fit_transform(train)
print (X_train_counts.shape)
X_test_counts = count_vect.transform(test)

for c in [ .001, .01, .1, 1]:
    clf = LinearSVC(C=c, max_iter=5000)
    clf.fit(X_train_counts, labels_train)

    pred = clf.predict(X_test_counts)

    print ('c =',c, ' acc = ', accuracy_score(pred, labels_test))
count_vect = CountVectorizer(lowercase=True,
                             stop_words='english', 
                             ngram_range=(3, 3), analyzer='word', 
                             max_df=1.0, min_df=1, max_features=10000,
                             binary=False)
X_train_counts = count_vect.fit_transform(train)
print (X_train_counts.shape)
X_test_counts = count_vect.transform(test)

for c in [ .001, .01, .1, 1]:
    clf = LinearSVC(C=c, max_iter=5000)
    clf.fit(X_train_counts, labels_train)

    pred = clf.predict(X_test_counts)

    print ('c =',c, ' acc = ', accuracy_score(pred, labels_test))<jupyter_output>(25000, 10000)
c = 0.001  acc =  0.64868
c = 0.01  acc =  0.68476
c = 0.1  acc =  0.68008
c = 1  acc =  0.66184
{'fraction_non_alphanumeric': 0.12171458811077791, 'fraction_numerical': 0.05838772270241665, 'mean_word_length': 3.1447368421052633, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 42, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '199846', 'license': 'Apache-2.0', 'n_tokens_mistral': 2313, 'n_tokens_neox': 2089, 'n_words': 443, 'path': 'deeplearning1/nbs/lesson5-Michael-comparison with SVM.ipynb', 'repo_name': 'michael135/courses', 'size': '5669'}