<jupyter_start><jupyter_text>TensorLayer Intro Tutorialby <NAME> This is the notebook for the [TensorLayer Introduction Tutorial Video](https://www.youtube.com/watch?v=sDKH2452IR8&feature=youtu.be&hd=1).You may read this on its own or follow along with the video.  From this tutorial you will learn* What TensorLayer is and its benefits.* How to build an Artifical Neural Network using TensorLayer.   What is TensorLayer? [TensorLayer](https://tensorlayer.readthedocs.io/en/stable/) is a Deep Learning library built on TensorFlow. The network construction is similar to Keras but takes components from TensorFlow to still integrate the graph structure.<jupyter_code># Load Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import tensorlayer as tl
from sklearn.preprocessing import LabelBinarizer, MinMaxScaler

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Data Source: [Adult Income Census Data](https://archive.ics.uci.edu/ml/datasets/Adult)<jupyter_code># Load the data
adults_train = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data')
# Create columns since columns did not exist. Used from source of data.
adults_train.columns =['Age','WorkClass','Fnlwgt','Education','EducationNum','MartialStatus','Occupation',
                      'Relationship','Race','Sex','CapitalGain','CapitalLoss','HoursPerWeek','NativeCountry','SalaryCategory']<jupyter_output><empty_output><jupyter_text>**Note**Since the tutorial is focused on TensorLayer implementation, we are going to just take the numeric data and feature-engineer the SalaryCategory Column. In reality you should feature engineer the categorical values and feature scale the numeric data. You should also apply feature selection to see if all the columns are truly necessary for the classifier.We will also not do a complete exploratory data analysis of the data for the same reason.  Removing the Categorical Columns<jupyter_code>adults_train.info()
# Get list of categorical 
def remove_cat_columns(data):
    categorical_columns = []
    for column in data.columns:
        if data[column].dtype == 'O' and column != 'SalaryCategory':
            categorical_columns.append(column)
    return categorical_columns

categorical_columns = remove_cat_columns(adults_train)
# Remove Categorical columns for training
adults_train_filtered = adults_train.drop(categorical_columns, axis=1)
# Check head of data to confirm categorical column removal
adults_train_filtered.head()<jupyter_output><empty_output><jupyter_text>Feature Engineering the Numeric Data and Labels<jupyter_code># Separate Input data from Labels
X = adults_train_filtered.drop(['SalaryCategory'],axis=1).values
y = adults_train_filtered['SalaryCategory'].values
# Binarize SalaryCategory Column
lb = LabelBinarizer()
y_lb= lb.fit_transform(y)
# Visualize the count of our labels
adults_train_filtered.SalaryCategory.value_counts().plot.bar()<jupyter_output><empty_output><jupyter_text>So our classification results will not be optimal since our data is very imbalanced. Most of the people in this data file make less than or equal to 50K.<jupyter_code># MinMaxScale the input data
mms = MinMaxScaler()
X_scaled = mms.fit_transform(X)
# Check shape for inputs
X_scaled.shape
# Check shape of target values vector
y_lb.shape
# Flatten the target values vector
y_lb = y_lb.flatten()
# See new shape of target values vector
y_lb.shape<jupyter_output><empty_output><jupyter_text>Great! Now we are set to learn how to implement our TensorLayer ANN!  TensorLayer Model Construction<jupyter_code># Initialize hyperparameters
learning_rate = 0.001
num_hidden = 6
num_outputs = 2
batch_size = 300
n_epoch = 500
print_freq = 5
# Reset the graph
tf.reset_default_graph()
# Set up placeholders
x = tf.placeholder(tf.float32, shape=[None,6], name='x')
y_ = tf.placeholder(tf.int64,shape=[None,], name='y_')
# Initiate interactive session
sess = tf.InteractiveSession()
# create a variable for the relu function
act = tf.nn.relu
# Build the network
network = tl.layers.InputLayer(x, name='input')
network = tl.layers.DenseLayer(network, n_units=num_hidden, act=act, name='h1')
network = tl.layers.DenseLayer(network, n_units=num_hidden, act=act, name='h2')
network = tl.layers.DenseLayer(network, n_units=num_hidden, act=act, name='h3')
network = tl.layers.DenseLayer(network, n_units=num_outputs, act=tf.identity, name='output')
# Set up cost and metric
y = network.outputs
cost = tl.cost.cross_entropy(y, y_, name='cross_entropy')
correct_prediction = tf.equal(tf.argmax(y,1),y_)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.int64))
y_op = tf.argmax(tf.nn.softmax(y),1)
# Set up optimizer
train_params = network.all_params
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
train_op = optimizer.minimize(cost, var_list=train_params)
# Initialize all variables in the session
sess.run(tf.global_variables_initializer())
# Fit the model
tl.utils.fit(sess, network, train_op, cost, X_scaled, y_lb, x, y_, acc=accuracy,
             n_epoch=n_epoch, print_freq=print_freq,
            eval_train=False)
# get predictions
y_pred = tl.utils.predict(sess, network, X_scaled, x=x, y_op=y_op)
# evaluate the predictions
results = tl.utils.evaluation(y_lb,y_pred, n_classes=2)<jupyter_output>[TL] confusion matrix: 
[[23528  1191]
 [ 4841  3000]]
[TL] f1-score        : [0.88637734 0.49867021]
[TL] f1-score(macro) : 0.692524
[TL] accuracy-score  : 0.814742<jupyter_text>Apply Test Data to Model<jupyter_code># load the test data
adults_test = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test', header=1)
# check test data head
adults_test.head()
# Assign Proper Columns
adults_test.columns = adults_train.columns
# Remove categorical Columns
categorical_columns = remove_cat_columns(adults_test)
adults_test_filtered = adults_test.drop(categorical_columns, axis=1)
# Separate test inputs from test labels
X_test = adults_test_filtered.drop(['SalaryCategory'], axis=1).values
y_test = adults_test_filtered['SalaryCategory'].values
# Transform test inputs with minmaxscaler
X_test_scaled = mms.transform(X_test)
# Binarize test labels
y_test_lb = lb.fit_transform(y_test).flatten()
# Visualize the count of our labels
adults_test_filtered.SalaryCategory.value_counts().plot.bar()
# Check test labels shape
y_test_lb.shape
# Apply test inputs to model
y_pred_test = tl.utils.predict(sess, network, X_test_scaled, x=x, y_op=y_op)
# Get metrics from test vs predicted test labels such as accuracy, f1-score, and the confusion matrix
results_test = tl.utils.evaluation(y_test_lb, y_pred_test, n_classes=2)
# save  the model
tl.files.save_npz_dict(train_params,name='adult_ann_basic.npz',sess=sess)
# Close the session. Don't forget to do this for the interactive sessions.
sess.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09660081348053458, 'fraction_numerical': 0.01336432306798373, 'mean_word_length': 6.19435736677116, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '625400', 'license': 'MIT', 'n_tokens_mistral': 2297, 'n_tokens_neox': 2193, 'n_words': 612, 'path': 'TensorLayer Intro Tutorial.ipynb', 'repo_name': 'JohnDeJesus22/YoutubeTutorials', 'size': '6890'}