<jupyter_start><jupyter_text>Classify different data sets  Basic includes<jupyter_code># Using pandas to load the csv file
import pandas as pd

import numpy as np
import matplotlib.pyplot as plt

from keras import models 
from keras import layers 
from keras import callbacks
from keras.utils import to_categorical

# reuters and fashin mnist data set from keras
from keras.datasets import reuters
from keras.datasets import fashion_mnist

# needed to preprocess text
from keras.preprocessing.text import Tokenizer<jupyter_output><empty_output><jupyter_text>Classify the Fashion Mnist---<jupyter_code>(fashion_train_data, fashion_train_labels), (fashion_test_data, fashion_test_labels) = fashion_mnist.load_data()

print(fashion_train_data.shape)

test_index = 10

plt.title("Label: " + str(fashion_train_labels[test_index]))
plt.imshow(fashion_train_data[test_index], cmap="gray")<jupyter_output>(60000, 28, 28)<jupyter_text>TO DO: Preprocess the data1. Normalize the input data set2. Perform one hot encoding3. Create a train, test, and validation set<jupyter_code>fashion_train_data = fashion_train_data.reshape((60000, 28 * 28))
fashion_train_data = fashion_train_data.astype('float32') / 255 

# same starndadization for the test images
fashion_test_data = fashion_test_data.reshape((10000, 28 * 28)) 
fashion_test_data = fashion_test_data.astype('float32') / 255

# one hot encoding
fashion_train_labels = to_categorical(fashion_train_labels) 
fashion_test_labels = to_categorical(fashion_test_labels)

validation_data = fashion_train_data[:10000]
validation_labels = fashion_train_labels[:10000]

x_data = fashion_train_data[50000:]
y_data = fashion_train_labels[50000:]

print(fashion_train_labels[0].shape)<jupyter_output>(10,)<jupyter_text>TO DO: Define and train a network, then plot the accuracy of the training, validation, and testing1. Use a validation set2. Propose and train a network3. Print the history of the training4. Evaluate with a test set<jupyter_code># The keras.models.Sequential class is a wrapper for the neural network model that treats 
# the network as a sequence of layers
network = models.Sequential()

# Dense layers: fully connected layers
network.add(layers.Dense(128, activation='relu', input_shape=(28 * 28,))) 
network.add(layers.Dense(64, activation='relu'))
network.add(layers.Dense(32, activation='relu'))
network.add(layers.Dense(10, activation='softmax'))

network.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])

network.summary()
history = network.fit(fashion_train_data, fashion_train_labels,batch_size = 512, epochs=50, validation_split=0.2, verbose=2)

test_loss, test_acc = network.evaluate(fashion_test_data, fashion_test_labels)

print("test loss: ", test_loss, "test accuracy: ", test_acc)<jupyter_output>Train on 48000 samples, validate on 12000 samples
Epoch 1/50
 - 2s - loss: 0.8408 - acc: 0.7022 - val_loss: 0.5891 - val_acc: 0.7841
Epoch 2/50
 - 1s - loss: 0.5258 - acc: 0.8114 - val_loss: 0.4735 - val_acc: 0.8314
Epoch 3/50
 - 1s - loss: 0.4618 - acc: 0.8349 - val_loss: 0.4264 - val_acc: 0.8496
Epoch 4/50
 - 1s - loss: 0.4205 - acc: 0.8485 - val_loss: 0.4379 - val_acc: 0.8445
Epoch 5/50
 - 1s - loss: 0.3930 - acc: 0.8571 - val_loss: 0.3876 - val_acc: 0.8619
Epoch 6/50
 - 1s - loss: 0.3658 - acc: 0.8669 - val_loss: 0.3583 - val_acc: 0.8705
Epoch 7/50
 - 1s - loss: 0.3506 - acc: 0.8698 - val_loss: 0.3512 - val_acc: 0.8737
Epoch 8/50
 - 1s - loss: 0.3342 - acc: 0.8760 - val_loss: 0.4034 - val_acc: 0.8472
Epoch 9/50
 - 0s - loss: 0.3190 - acc: 0.8822 - val_loss: 0.3571 - val_acc: 0.8717
Epoch 10/50
 - 0s - loss: 0.3097 - acc: 0.8842 - val_loss: 0.3573 - val_acc: 0.8693
Epoch 11/50
 - 1s - loss: 0.2970 - acc: 0.8891 - val_loss: 0.3763 - val_acc: 0.8625
Epoch 12/50
 - 0s - loss: 0.2903 - [...]<jupyter_text>Classifying newswires---Build a network to classify Reuters newswires into 46 different mutually-exclusive topics.  Load and review the data<jupyter_code>(reuters_train_data, reuters_train_labels), (reuters_test_data, reuters_test_labels) = reuters.load_data(num_words=10000)

print(reuters_train_data.shape)
print(reuters_train_labels.shape)
print(reuters_train_data[0])
print(reuters_train_labels[0])

print(set(reuters_train_labels))<jupyter_output>(8982,)
(8982,)
[1, 2, 2, 8, 43, 10, 447, 5, 25, 207, 270, 5, 3095, 111, 16, 369, 186, 90, 67, 7, 89, 5, 19, 102, 6, 19, 124, 15, 90, 67, 84, 22, 482, 26, 7, 48, 4, 49, 8, 864, 39, 209, 154, 6, 151, 6, 83, 11, 15, 22, 155, 11, 15, 7, 48, 9, 4579, 1005, 504, 6, 258, 6, 272, 11, 15, 22, 134, 44, 11, 15, 16, 8, 197, 1245, 90, 67, 52, 29, 209, 30, 32, 132, 6, 109, 15, 17, 12]
3
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45}<jupyter_text>Load the word index to decode the train data.<jupyter_code>word_index = reuters.get_word_index()# Turning the output into vector mode, each of length 10000
tokenizer = Tokenizer(num_words=10000)
train_data_token = tokenizer.sequences_to_matrix(reuters_train_data, mode='count')
test_data_token = tokenizer.sequences_to_matrix(reuters_test_data, mode='count')
print(train_data_token.shape)
print(test_data_token.shape)

# One-hot encoding the output
num_classes = 46
one_hot_train_labels = to_categorical(reuters_train_labels,num_classes)
one_hot_test_labels = to_categorical(reuters_test_labels, num_classes)
print(one_hot_train_labels.shape)
print(one_hot_test_labels.shape)

# Creating a validation set with the first 1000 reviews
# NOT USED
validation_data = train_data_token[:1000]
validation_labels = one_hot_train_labels[:1000]

reverse_index = dict([(value+3, key) for (key, value) in word_index.items()])

reverse_index[0] = "<PAD>"
reverse_index[1] = "<START>"
reverse_index[2] = "<UNKNOWN>"  # unknown
reverse_index[3] = "<UNUSED>"

decoded_review = ' '.join([reverse_index.get(i,'?') for i in reuters_train_data[0]])

print(decoded_review)<jupyter_output>(8982, 10000)
(2246, 10000)
(8982, 46)
(2246, 46)
<START> <UNKNOWN> <UNKNOWN> said as a result of its december acquisition of space co it expects earnings per share in 1987 of 1 15 to 1 30 dlrs per share up from 70 cts in 1986 the company said pretax net should rise to nine to 10 mln dlrs from six mln dlrs in 1986 and rental operation revenues to 19 to 22 mln dlrs from 12 5 mln dlrs it said cash flow per share this year should be 2 50 to three dlrs reuter 3<jupyter_text>TO DO: Preprocess the data1. Normalize the input data set2. Perform one hot encoding3. Create a train, test, and validation set<jupyter_code># Turning the output into vector mode, each of length 10000
tokenizer = Tokenizer(num_words=10000)
train_data_token = tokenizer.sequences_to_matrix(reuters_train_data, mode='binary')
test_data_token = tokenizer.sequences_to_matrix(reuters_test_data, mode='binary')
print(train_data_token.shape)
print(test_data_token.shape)

# One-hot encoding the output
num_classes = 46
one_hot_train_labels = to_categorical(reuters_train_labels,num_classes)
one_hot_test_labels = to_categorical(reuters_test_labels, num_classes)
print(one_hot_train_labels.shape)
print(one_hot_test_labels.shape)

# Creating a validation set with the first 1000 reviews
# NOT USED
validation_data = train_data_token[:1000]
validation_labels = one_hot_train_labels[:1000]<jupyter_output>(8982, 10000)
(2246, 10000)
(8982, 46)
(2246, 46)<jupyter_text>TO DO: Define and train a network, then plot the accuracy of the training, validation, and testing1. Use a validation set2. Propose and train a network3. Print the history of the training4. Evaluate with a test set<jupyter_code>network2 = models.Sequential()

network2.add(layers.Dense(128, activation='relu', input_dim=10000))
network2.add(layers.Dropout(0.3))
network2.add(layers.Dense(64, activation='relu'))
network2.add(layers.Dropout(0.3))
network2.add(layers.Dense(num_classes, activation='softmax'))
network2.summary()

# included the early stopping which monitors the validation loss
early_stop = callbacks.EarlyStopping(monitor='val_loss', patience=5)
network2.compile(loss='categorical_crossentropy', 
              optimizer='rmsprop',
              metrics=['accuracy'])
history = network2.fit(train_data_token, one_hot_train_labels,
          batch_size=512,
          epochs=40,
          validation_split=0.2,
          callbacks=[early_stop], 
          verbose=2)

test_loss2, test_acc2 = network2.evaluate(test_data_token, one_hot_test_labels)

print("test loss: ", test_loss2, "test accuracy: ", test_acc2)<jupyter_output>Train on 7185 samples, validate on 1797 samples
Epoch 1/40
 - 2s - loss: 2.5539 - acc: 0.4553 - val_loss: 1.6511 - val_acc: 0.6561
Epoch 2/40
 - 1s - loss: 1.5520 - acc: 0.6593 - val_loss: 1.3376 - val_acc: 0.6973
Epoch 3/40
 - 1s - loss: 1.2427 - acc: 0.7250 - val_loss: 1.1932 - val_acc: 0.7357
Epoch 4/40
 - 1s - loss: 1.0347 - acc: 0.7729 - val_loss: 1.1218 - val_acc: 0.7518
Epoch 5/40
 - 1s - loss: 0.8862 - acc: 0.8054 - val_loss: 1.0284 - val_acc: 0.7802
Epoch 6/40
 - 1s - loss: 0.7650 - acc: 0.8266 - val_loss: 0.9957 - val_acc: 0.7852
Epoch 7/40
 - 1s - loss: 0.6656 - acc: 0.8493 - val_loss: 0.9758 - val_acc: 0.7908
Epoch 8/40
 - 1s - loss: 0.5751 - acc: 0.8664 - val_loss: 0.9565 - val_acc: 0.7997
Epoch 9/40
 - 1s - loss: 0.5208 - acc: 0.8828 - val_loss: 0.9681 - val_acc: 0.7947
Epoch 10/40
 - 1s - loss: 0.4679 - acc: 0.8921 - val_loss: 0.9504 - val_acc: 0.8036
Epoch 11/40
 - 1s - loss: 0.3977 - acc: 0.9113 - val_loss: 0.9498 - val_acc: 0.8075
Epoch 12/40
 - 1s - loss: 0.3633 - ac[...]<jupyter_text>Predicting Student Admissions---Predict student admissions based on three pieces of data:- GRE Scores- GPA Scores- Class rank  Load and visualize the data<jupyter_code>student_data = pd.read_csv("student_data.csv")
print(student_data)<jupyter_output>admit  gre   gpa  rank
0        0  380  3.61     3
1        1  660  3.67     3
2        1  800  4.00     1
3        1  640  3.19     4
4        0  520  2.93     4
5        1  760  3.00     2
6        1  560  2.98     1
7        0  400  3.08     2
8        1  540  3.39     3
9        0  700  3.92     2
10       0  800  4.00     4
11       0  440  3.22     1
12       1  760  4.00     1
13       0  700  3.08     2
14       1  700  4.00     1
15       0  480  3.44     3
16       0  780  3.87     4
17       0  360  2.56     3
18       0  800  3.75     2
19       1  540  3.81     1
20       0  500  3.17     3
21       1  660  3.63     2
22       0  600  2.82     4
23       0  680  3.19     4
24       1  760  3.35     2
25       1  800  3.66     1
26       1  620  3.61     1
27       1  520  3.74     4
28       1  780  3.22     2
29       0  520  3.29     1
..     ...  ...   ...   ...
369      1  540  3.77     2
370      1  680  3.76     3
371      1  680  2.42     1
372      1  620  3.3[...]<jupyter_text>Plot of the GRE and the GPA from the data.<jupyter_code>X = np.array(student_data[["gre","gpa"]])
y = np.array(student_data["admit"])
admitted = X[np.argwhere(y==1)]
rejected = X[np.argwhere(y==0)]
plt.scatter([s[0][0] for s in rejected], [s[0][1] for s in rejected], s = 25, color = 'red', edgecolor = 'k')
plt.scatter([s[0][0] for s in admitted], [s[0][1] for s in admitted], s = 25, color = 'cyan', edgecolor = 'k')
plt.xlabel('Test (GRE)')
plt.ylabel('Grades (GPA)')

plt.show()<jupyter_output><empty_output><jupyter_text>Plot of the data by class rank.<jupyter_code>f, plots = plt.subplots(2, 2, figsize=(20,10))
plots = [plot for sublist in plots for plot in sublist]

for idx, plot in enumerate(plots):
    data_rank = student_data[student_data["rank"]==idx+1]
    plot.set_title("Rank " + str(idx+1))
    X = np.array(data_rank[["gre","gpa"]])
    y = np.array(data_rank["admit"])
    admitted = X[np.argwhere(y==1)]
    rejected = X[np.argwhere(y==0)]
    plot.scatter([s[0][0] for s in rejected], [s[0][1] for s in rejected], s = 25, color = 'red', edgecolor = 'k')
    plot.scatter([s[0][0] for s in admitted], [s[0][1] for s in admitted], s = 25, color = 'cyan', edgecolor = 'k')
    plot.set_xlabel('Test (GRE)')
    plot.set_ylabel('Grades (GPA)')<jupyter_output><empty_output><jupyter_text>TO DO: Preprocess the data1. Normalize the input data set2. Perform one hot encoding3. Create a train, test, and validation set<jupyter_code>labels = np.array(student_data["admit"])

gre = np.array(student_data["gre"])
avg_gre = np.average(gre)
std_gre = np.std(gre)

gpa = np.array(student_data["gpa"])
avg_gpa = np.average(gpa)
std_gpa = np.std(gpa)

rank = np.array(student_data[["rank"]])

# Normal data
gre = (gre-avg_gre)/std_gre
gpa = (gpa-avg_gpa)/std_gpa
rank = to_categorical(rank-1)

inputs = np.zeros((len(data),6))

inputs[:,0] = gre
inputs[:,1] = gpa
inputs[:,2:6] = rank

train_data = inputs[:300]
train_labels = to_categorical(labels[:300])

test_data = inputs [300:]
test_labels = to_categorical(labels[300:])

print (train_data.shape)
print (train_labels.shape)
print (test_data.shape)
print (test_labels.shape)<jupyter_output>(300, 6)
(300, 2)
(99, 6)
(99, 2)<jupyter_text>TO DO: Define and train a network, then plot the accuracy of the training, validation, and testing1. Use a validation set2. Propose and train a network3. Print the history of the training4. Evaluate with a test set<jupyter_code>network3 = models.Sequential()

network3.add(layers.Dense(128, activation='relu', input_dim=6))
network3.add(layers.Dropout(0.3))
network3.add(layers.Dense(16, activation='relu'))
network3.add(layers.Dropout(0.1))
network3.add(layers.Dense(2, activation='softmax'))
network3.summary()

# included the early stopping which monitors the validation loss
early_stop = callbacks.EarlyStopping(monitor='val_loss', patience=5)
network3.compile(loss='binary_crossentropy', 
              optimizer='rmsprop',
              metrics=['accuracy'])
history = network3.fit(train_data, train_labels,
          batch_size=512,
          epochs=100,
          validation_split=0.1,
          callbacks=[early_stop], 
          verbose=2)

test_loss3, test_acc3 = network3.evaluate(test_data, test_labels)

print("test loss: ", test_loss2, "test accuracy: ", test_acc2)<jupyter_output>Train on 270 samples, validate on 30 samples
Epoch 1/100
 - 1s - loss: 0.7146 - acc: 0.5037 - val_loss: 0.6694 - val_acc: 0.7333
Epoch 2/100
 - 0s - loss: 0.6674 - acc: 0.6185 - val_loss: 0.6497 - val_acc: 0.7000
Epoch 3/100
 - 0s - loss: 0.6575 - acc: 0.6185 - val_loss: 0.6363 - val_acc: 0.6667
Epoch 4/100
 - 0s - loss: 0.6358 - acc: 0.6741 - val_loss: 0.6258 - val_acc: 0.6333
Epoch 5/100
 - 0s - loss: 0.6307 - acc: 0.6704 - val_loss: 0.6166 - val_acc: 0.6333
Epoch 6/100
 - 0s - loss: 0.6094 - acc: 0.6926 - val_loss: 0.6076 - val_acc: 0.6667
Epoch 7/100
 - 0s - loss: 0.6140 - acc: 0.6889 - val_loss: 0.6008 - val_acc: 0.6667
Epoch 8/100
 - 0s - loss: 0.5944 - acc: 0.6889 - val_loss: 0.5948 - val_acc: 0.6667
Epoch 9/100
 - 0s - loss: 0.5956 - acc: 0.7000 - val_loss: 0.5910 - val_acc: 0.6667
Epoch 10/100
 - 0s - loss: 0.5872 - acc: 0.7037 - val_loss: 0.5872 - val_acc: 0.6667
Epoch 11/100
 - 0s - loss: 0.5821 - acc: 0.7370 - val_loss: 0.5839 - val_acc: 0.6667
Epoch 12/100
 - 0s - loss: 0.[...]
{'fraction_non_alphanumeric': 0.12583305839656878, 'fraction_numerical': 0.1231276806334543, 'mean_word_length': 3.754077791718946, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 15, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '610822', 'license': 'MIT', 'n_tokens_mistral': 7064, 'n_tokens_neox': 5781, 'n_words': 1598, 'path': '.ipynb_checkpoints/Keras_assignment-checkpoint.ipynb', 'repo_name': 'DeepLearningVision-2019/a4-keras-classification-CesarValladares', 'size': '15155'}