<jupyter_start><jupyter_text>Classifier for Toy Problem 2Now let's build a classifier for Toy Problem 2.We've provided a utility class 'Data' (in data_reader.py) to load the training data (it works for all the toy problems).<jupyter_code>import torch
import torch.nn.functional as F
from data_reader import Data

data = Data("data/toy_problem_2_train.txt")

labels, features = data.get_sample()

print("Labels:\n"+str(labels))

print("Features:\n"+str(features))
    
target = torch.autograd.Variable(torch.LongTensor(labels))
#print("Labels Tensor:\n"+str(target))

features = torch.autograd.Variable(torch.Tensor(features))
#print("Features Tensor:\n"+str(features))<jupyter_output>Labels:
[1, 1, 1, 1, 0, 0, 0, 0, 0, 0]
Features:
[[-205, -44, -99], [-167, -84, -42], [121, 73, 82], [2, -54, 100], [94, -10, 83], [5, -23, -46], [59, 29, -57], [-27, -49, -76], [-131, -98, -100], [76, 27, 37]]<jupyter_text>We initialize the weights randomly.We can now perform 100 learning iterations below as many times as we want.Each learning iteration involves a forward pass and a backward pass.The forward pass involves the computation of the loss from the training data and the current parameters.The backward pass is performed automatically by Pytorch when you call loss.backward().Pytorch calculates all the gradients with respect to the loss.These gradients are stored in each parameter's 'grad' member variable.Notice that the code for the learning iterations is identical to that of exercise 530.<jupyter_code>weights = torch.nn.Parameter(torch.rand(3, 2))
print(weights)

for i in range(101):
    labels, features = data.get_sample(10)
    
    features = torch.autograd.Variable(torch.Tensor(features))
    #print(data)
    
    target = torch.autograd.Variable(torch.LongTensor(labels))
    #print(target)
    
    result = torch.mm(features, weights)
    #print(result)
    
    loss = F.cross_entropy(result, target)
    #print("Cross entropy loss: "+str(loss))
    
    #result = F.softmax(result)
    #result = torch.log(result)
    #print("Log softmax: "+str(result))
    #loss = F.nll_loss(result, target)
    #print("Cross entropy loss: "+str(loss))
    
    loss.backward()
    
    gradient = weights.grad
    
    learning_rate = 0.01
    
    weights.data = weights.data - learning_rate * gradient.data
    
    if i % 10 == 0:
        print("The loss is now "+str(loss.data[0]))
    
    weights.grad.data.zero_()

print("\tThe weights are now "+str(weights.data))

torch.save(weights, "models/toy_problem_2_trained_model.bin")<jupyter_output>Parameter containing:
 0.3584  0.5541
 0.1191  0.4134
 0.0212  0.0041
[torch.FloatTensor of size 3x2]

The loss is now 33.18605041503906
The loss is now 0.006583866663277149
The loss is now 0.000537065789103508
The loss is now 0.14890044927597046
The loss is now 4.168130544712767e-05
The loss is now 0.0009911548113450408
The loss is now 8.514986038208008
The loss is now 0.6193047761917114
The loss is now 2.188471571931e-13
The loss is now 3.45133900642395
The loss is now 1.425714685199253e-10
	The weights are now 
 1.2388 -0.3263
-0.4833  1.0158
-0.7238  0.7491
[torch.FloatTensor of size 3x2]<jupyter_text>ParametersAs we know, the final parameters learnt by the algorithm should look something like this$$\begin{bmatrix}1 & 0 \\ 0 & 1 \\ 0 & 1\end{bmatrix}$$or this$$\begin{bmatrix}2 & 1 \\ 1 & 2 \\ 1 & 2\end{bmatrix}$$Basically the weights values at 0,0 and 1,1 and 2,1 in the matrix should be higher than the weights at 1,0 and 0,1 and 2,0.  Classifier for Toy Problem 2We have just trained a classifier for Toy Problem 2.You can evaluate the performance of the classifier on the test data.<jupyter_code>data = Data("data/toy_problem_2_test.txt")

weights = torch.load("models/toy_problem_2_trained_model.bin")
print(weights)

labels, features = data.get_all()

features = torch.autograd.Variable(torch.Tensor(features))
#print(features)

target = torch.autograd.Variable(torch.LongTensor(labels))
#print(target)

result = torch.mm(features, weights)
#print(result)

maxv, observed = torch.max(result, 1)

total = 0
correct = 0
for i in range(len(labels)):
    total += 1
    #print(str(target.data[i]) + " " + str(observed.data[i]))
    if target.data[i] == observed.data[i]:
        correct += 1
accuracy = correct / total
print("Accuracy: "+str(accuracy))<jupyter_output>Parameter containing:
 1.2388 -0.3263
-0.4833  1.0158
-0.7238  0.7491
[torch.FloatTensor of size 3x2]

Accuracy: 0.979
{'fraction_non_alphanumeric': 0.12113055181695828, 'fraction_numerical': 0.09466128308658592, 'mean_word_length': 4.190919674039581, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '283620', 'license': 'MIT', 'n_tokens_mistral': 1788, 'n_tokens_neox': 1515, 'n_words': 460, 'path': 'exercise_550.ipynb', 'repo_name': 'ahurriyetoglu/DeepLearningBasicsTutorial', 'size': '4458'}