<jupyter_start><jupyter_text>![image.png](attachment:image.png)<jupyter_code># IMDB Dataset loading
# this will download our data set from the web.
# We'll name the path where we want to save it, the extension being pkl,
# which means it's a byte stream, 
# This makes it easier to convert to other python objects like list or 
# tuples later, 
# We want 10,000 words from the database, and we only want to use 10%

train,test, _ = imdb.load_data(path="imdb.pkl",n_words=10000,
                              valid_portion=0.1)
# load data will return our movie review split into a training and testing set.
#We cant then further split those sets into reviews and labels and set then
#equal to X and Y values.

trainX, trainY = train
testX, testY = test
len(trainX)<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)  While training data helps us fit our weights, validation data helps prevent over fitting by letting us tune our hyper parameters accordingly.  - Neural nets are algorithms that essentially just apply a series of computations to your matrices. So, converting them to numerical representations or vectors is necessary. ![image.png](attachment:image.png)<jupyter_code># Data preprocessing
#Sequence padding 
# The pad_sequences function will do that for our vies text.
# It'll convert each review into a matrix and pad it.
# Padding is necessary to ensure consistency in our inputs
# It will pad each sequence with a zero at the end which we specify until it.
# reaches the max possible sequence length which we'll set to 100


trainX = pad_sequences(trainX,maxlen=100,value=0.)
testX = pad_sequences(testX,maxlen=100,value=0.)<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)<jupyter_code>len(trainX)
# Converting labels to binary vectors

# We also want to convert our labels to vectors as wll and we can
# easily do that using the two categorical function.
# These are binary vectors with two classes, 1 which is positive
#or 0 whcih is negative

trainY = to_categorical(trainY,nb_classes=2)
testY = to_categorical(testY,nb_classes=2)<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)<jupyter_code># Network building
# First will be our impro layer, this is where we feed data into our network
# The only perameter we'll specify is the input shape.
net = tflearn.input_data([None,100] )
# The first element is the batch size, which we'll set to none and 
# then the lenght, which is 100, since we set our max sequence length to 100

# our next layer is our embedding layer. 
# The first perameter would be the output vector we receive
# from the previous layer, and by the way
# for every layer we write we'll be using the previous layer's outputs
# as inputs. 
# This is how data flows through a neural network, at each layer 
# it's transformed like a seven layer dip of computation. 

net = tflearn.embedding(net, input_dim=10000,output_dim=128)
# We'll set dimensions to 10,000 since that's how many words we loaded from
# our data set earlier.
# And the output dimension to 128, which is the number of dimensions in our
# resulting embedding's 
# Next, we'll feed those values to our LSTM layer (Long short term memory)
# This layer allows our network to remember data from the beginning of 
# the sequences, which will improve our prediction.

net = tflearn.lstm(net,128,dropout=0.)

# We'll set dropout to 0.08 which is a technique that helps prevent
# over fitting by randomly turning on and off different pathways in our
# network.<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)<jupyter_code># our next layer is fully connected which means that every neuron in 
# the previous layer is connected to every neuron in this layer. 

net = tflearn.fully_connected(net,2,activation='softmax')

# We have a set of learned feature vectors from previous layers,
# and adding a fully connected layer is a computationally cheap way
# of learning non-linear combinations of them.
# it's got two units, and it's using the softmax function as
# its activation function<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)<jupyter_code># this will take in a vector of values and squash it into a vector of
# output probabilities between 0 and 1, that sum to 1

# We'll use those values in our last layer, which is our regression layer.
# This will apply a regression operation to the input.
net = tflearn.regression(net,optimizer='adam',learning_rate=0.0001,
                        loss="categorical_crossentropy")
# We're going to specify an optimizer method that will minimize a given
# a loss function, as well as the learning rate, 
# which specifies how fast we want our network to train.

# The optimizer we'll use is adam, which performs gradient descent.
# and categorical cross entropy is our loss, it helps to find the difference
# between our predicted output and the expected output.<jupyter_output><empty_output><jupyter_text>![image.png](attachment:image.png)<jupyter_code># after building our neural network, we can go ahead and 
# initialize it using tflearn's deep neural net function.

# Training
model = tflearn.DNN(net,tensorboard_verbose=0)

# Then we can call our models fit function, which will launch
# the training process for our given training and validation data.

# we'll also set show metric to true so 
model.fit(trainX,trainY,validation_set=(testX,testY),show_metric=True,
         batch_size = 33)
# we can view the log of accuracy during training.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06974653963688657, 'fraction_numerical': 0.013122415962610103, 'mean_word_length': 4.856842105263158, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '58687', 'license': 'MIT', 'n_tokens_mistral': 1646, 'n_tokens_neox': 1533, 'n_words': 717, 'path': 'SirajRaval_Ejemplos/IntroDeeplearning/.ipynb_checkpoints/MiDNN-checkpoint.ipynb', 'repo_name': 'IntroCursos/MachingLearning', 'size': '5563'}