<jupyter_start><jupyter_text>Theano ===A language in a language Dealing with weights matrices and gradients can be tricky and sometimes not trivial.Theano is a great framework for handling vectors, matrices and high dimensional tensor algebra. Most of this tutorial will refer to Theano however TensorFlow is another great framework capable of providing an incredible abstraction for complex algebra.More on TensorFlow in the next chapters.<jupyter_code>import theano
import theano.tensor as T<jupyter_output><empty_output><jupyter_text>Symbolic variables========== Theano has it's own variables and functions, defined the following<jupyter_code>x = T.scalar()
x<jupyter_output><empty_output><jupyter_text>Variables can be used in expressions<jupyter_code>y = 3*(x**2) + 1<jupyter_output><empty_output><jupyter_text>y is an expression now  Result is symbolic as well<jupyter_code>type(y)
y.shape<jupyter_output><empty_output><jupyter_text>printing As we are about to see, normal printing isn't the best when it comes to theano<jupyter_code>print(y)
theano.pprint(y)
theano.printing.debugprint(y)<jupyter_output>Elemwise{add,no_inplace} [id A] ''   
 |Elemwise{mul,no_inplace} [id B] ''   
 | |TensorConstant{3} [id C]
 | |Elemwise{pow,no_inplace} [id D] ''   
 |   |<TensorType(float64, scalar)> [id E]
 |   |TensorConstant{2} [id F]
 |TensorConstant{1} [id G]<jupyter_text>Evaluating expressions============Supply a `dict` mapping variables to values<jupyter_code>y.eval({x: 2})<jupyter_output><empty_output><jupyter_text>Or compile a function<jupyter_code>f = theano.function([x], y)
f(2)<jupyter_output><empty_output><jupyter_text>Other tensor types==========<jupyter_code>X = T.vector()
X = T.matrix()
X = T.tensor3()
X = T.tensor4()<jupyter_output><empty_output><jupyter_text>Automatic differention============- Gradients are free!<jupyter_code>x = T.scalar()
y = T.log(x)
gradient = T.grad(y, x)
print(gradient)
print(gradient.eval({x: 2}))
print((2 * gradient))<jupyter_output>Elemwise{true_div}.0
0.5
Elemwise{mul,no_inplace}.0<jupyter_text>Shared Variables- Symbolic + Storage<jupyter_code>import numpy as np
x = theano.shared(np.zeros((2, 3), dtype=theano.config.floatX))
x<jupyter_output><empty_output><jupyter_text>We can get and set the variable's value<jupyter_code>values = x.get_value()
print(values.shape)
print(values)
x.set_value(values)<jupyter_output><empty_output><jupyter_text>Shared variables can be used in expressions as well<jupyter_code>(x + 2) ** 2<jupyter_output><empty_output><jupyter_text>Their value is used as input when evaluating<jupyter_code>((x + 2) ** 2).eval()
theano.function([], (x + 2) ** 2)()<jupyter_output><empty_output><jupyter_text>Updates- Store results of function evalution- `dict` mapping shared variables to new values<jupyter_code>count = theano.shared(0)
new_count = count + 1
updates = {count: new_count}

f = theano.function([], count, updates=updates)
f()
f()
f()<jupyter_output><empty_output><jupyter_text>Warming up! Logistic Regression<jupyter_code>%matplotlib inline
import numpy as np
import theano
import theano.tensor as T
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Kaggle Challenge Data The Otto Group is one of the world’s biggest e-commerce companies, A consistent analysis of the performance of products is crucial. However, due to diverse global infrastructure, many identical products get classified differently.For this competition, we have provided a dataset with 93 features for more than 200,000 products. The objective is to build a predictive model which is able to distinguish between our main product categories. Each row corresponds to a single product. There are a total of 93 numerical features, which represent counts of different events. All features have been obfuscated and will not be defined any further.https://www.kaggle.com/c/otto-group-product-classification-challenge/data  For this section we will use the Kaggle Otto Group Challenge Data. You will find these data in `../data/kaggle_ottogroup/` folder.**Note** We already used this dataset in the [1.2 Introduction - Tensorflow](../1.2 Introduction - Tensorflow.ipynb) notebook, as well as [1.3 Introduction - Keras](../1.3 Introduction - Keras.ipynb) notebook.<jupyter_code>import os
import sys
nb_dir = os.path.abspath('..')
if nb_dir not in sys.path:
    sys.path.append(nb_dir)
from kaggle_data import load_data, preprocess_data, preprocess_labels
print("Loading data...")
X, labels = load_data('../data/kaggle_ottogroup/train.csv', train=True)
X, scaler = preprocess_data(X)
Y, encoder = preprocess_labels(labels)


X_test, ids = load_data('../data/kaggle_ottogroup/test.csv', train=False)
X_test, ids = X_test[:1000], ids[:1000]

#Plotting the data
print(X_test[:1])

X_test, _ = preprocess_data(X_test, scaler)

nb_classes = Y.shape[1]
print(nb_classes, 'classes')

dims = X.shape[1]
print(dims, 'dims')<jupyter_output>Loading data...
[[  0.   0.   0.   0.   0.   0.   0.   0.   0.   3.   0.   0.   0.   3.
    2.   1.   0.   0.   0.   0.   0.   0.   0.   5.   3.   1.   1.   0.
    0.   0.   0.   0.   1.   0.   0.   1.   0.   1.   0.   1.   0.   0.
    0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.
    0.   0.   0.   0.   0.   0.   0.   3.   0.   0.   0.   0.   1.   1.
    0.   1.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.
    0.  11.   1.  20.   0.   0.   0.   0.   0.]]
9 classes
93 dims<jupyter_text>Now lets create and train a logistic regression model.  Hands On - Logistic Regression<jupyter_code>#Based on example from DeepLearning.net
rng = np.random
N = 400
feats = 93
training_steps = 10

# Declare Theano symbolic variables
x = T.matrix("x")
y = T.vector("y")
w = theano.shared(rng.randn(feats), name="w")
b = theano.shared(0., name="b")

# Construct Theano expression graph
p_1 = 1 / (1 + T.exp(-T.dot(x, w) - b))   # Probability that target = 1
prediction = p_1 > 0.5                    # The prediction thresholded
xent = -y * T.log(p_1) - (1-y) * T.log(1-p_1) # Cross-entropy loss function
cost = xent.mean() + 0.01 * (w ** 2).sum()# The cost to minimize
gw, gb = T.grad(cost, [w, b])             # Compute the gradient of the cost
                                          # (we shall return to this in a
                                          # following sections of this tutorial
                                          # See: Intro to tf & Keras)

# Compile
train = theano.function(
          inputs=[x,y],
          outputs=[prediction, xent],
          updates=((w, w - 0.1 * gw), (b, b - 0.1 * gb)),
          allow_input_downcast=True)
predict = theano.function(inputs=[x], outputs=prediction, allow_input_downcast=True)

#Transform for class1
y_class1 = []
for i in Y:
    y_class1.append(i[0])
y_class1 = np.array(y_class1)

# Train
for i in range(training_steps):
    print('Epoch %s' % (i+1,))
    pred, err = train(X, y_class1)

print("target values for Data:")
print(y_class1)
print("prediction on training set:")
print(predict(X))<jupyter_output>Epoch 1
Epoch 2
Epoch 3
Epoch 4
Epoch 5
Epoch 6
Epoch 7
Epoch 8
Epoch 9
Epoch 10
target values for Data:
[ 0.  0.  0. ...,  0.  0.  0.]
prediction on training set:
[False False  True ..., False False False]
{'fraction_non_alphanumeric': 0.12612612612612611, 'fraction_numerical': 0.02882882882882883, 'mean_word_length': 3.652482269503546, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 67, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '238184', 'license': 'MIT', 'n_tokens_mistral': 2631, 'n_tokens_neox': 2539, 'n_words': 751, 'path': 'day03/additional materials/1.5.1 Introduction - Theano.ipynb', 'repo_name': 'MeRajat/Deep-Learning-Boot-Camp', 'size': '7215'}