<jupyter_start><jupyter_text>Classifier for Toy Problem 3Now let's build a classifier for Toy Problem 3.We've provided a utility class 'Data' (in data_reader.py) to load the training data (it works for all the toy problems).<jupyter_code>import torch
import torch.nn.functional as F
from data_reader import Data

data = Data("data/toy_problem_3_train.txt")

labels, features = data.get_sample()

print("Labels:\n"+str(labels))

print("Features:\n"+str(features))
    
target = torch.autograd.Variable(torch.LongTensor(labels))
#print("Labels Tensor:\n"+str(target))

features = torch.autograd.Variable(torch.Tensor(features))
#print("Features Tensor:\n"+str(features))<jupyter_output>Labels:
[0, 1, 1, 0, 0, 0, 0, 1, 0, 1]
Features:
[[26, 31], [-33, 33], [-75, 98], [20, 48], [30, 66], [68, 17], [-30, -32], [87, -70], [29, 54], [-92, 8]]<jupyter_text>We initialize the weights randomly.<jupyter_code>weights = torch.nn.Parameter(torch.rand(2, 2))
print(weights)<jupyter_output>Parameter containing:
tensor([[ 0.1221,  0.4194],
        [ 0.5483,  0.9584]])<jupyter_text>We can now perform 100 learning iterations below as many times as we want.Notice that the code for the learning iterations is identical to that of exercise 530.<jupyter_code>for i in range(101):
    labels, features = data.get_sample(10)
    
    features = torch.autograd.Variable(torch.Tensor(features))
    #print(data)
    
    target = torch.autograd.Variable(torch.LongTensor(labels))
    #print(target)
    
    result = torch.mm(features, weights)
    #print(result)
    
    loss = F.cross_entropy(result, target)
    #print("Cross entropy loss: "+str(loss))
    
    loss.backward()
    
    gradient = weights.grad
    
    learning_rate = 0.01
    
    weights.data = weights.data - learning_rate * gradient.data
    
    if i % 10 == 0:
        print("The loss is now "+str(loss.data.item()))
    
    weights.grad.data.zero_()

print("\tThe weights are now "+str(weights.data))

torch.save(weights, "models/toy_problem_3_trained_model.bin")<jupyter_output>The loss is now 2.601632833480835
The loss is now 2.004544734954834
The loss is now 12.074604034423828
The loss is now 11.238866806030273
The loss is now 5.32888650894165
The loss is now 6.22322416305542
The loss is now 10.873896598815918
The loss is now 1.5366473197937012
The loss is now 2.999246597290039
The loss is now 10.520666122436523
The loss is now 12.201106071472168
	The weights are now 
 0.5275  0.7267
 0.0379  0.4323
[torch.FloatTensor of size 2x2]<jupyter_text>The LossObserve the loss that is printed at the end of every 10 iterations.Now matter how many hundreds of times you run the hill-climbing code, the loss does not decrease.This tells us that the machine learning algorithm is probably not learning anthing much. ParametersYou can try find the parameters manually.  We know that the matrix should be a 2x2 matrix like this.$$\begin{bmatrix}w_{0,0} & w_{0,1} \\ w_{1,0} & w_{1,1} \end{bmatrix}$$But we won't be able to come up with a good set of values that works well on this problem.  Classifier Test - Toy Problem 3We have just trained a classifier for Toy Problem 3.It doesn't seem to be learning anything (the loss on the training data does not decrease).But, to make sure, let us evaluate the performance of the classifier on the test data.<jupyter_code>data = Data("data/toy_problem_3_test.txt")

weights = torch.load("models/toy_problem_3_trained_model.bin")
print(weights)

labels, features = data.get_all()

features = torch.autograd.Variable(torch.Tensor(features))
#print(features)

target = torch.autograd.Variable(torch.LongTensor(labels))
#print(target)

result = torch.mm(features, weights)
#print(result)

maxv, observed = torch.max(result, 1)

total = 0
correct = 0
for i in range(len(labels)):
    total += 1
    #print(str(target.data[i]) + " " + str(observed.data[i]))
    if target.data[i] == observed.data[i]:
        correct += 1
accuracy = correct / total
print("Accuracy: "+str(accuracy))<jupyter_output>Parameter containing:
 0.5275  0.7267
 0.0379  0.4323
[torch.FloatTensor of size 2x2]

Accuracy: 0.517
{'fraction_non_alphanumeric': 0.11572265625, 'fraction_numerical': 0.083251953125, 'mean_word_length': 4.390789473684211, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '84813', 'license': 'MIT', 'n_tokens_mistral': 1595, 'n_tokens_neox': 1372, 'n_words': 424, 'path': 'exercise_630.ipynb', 'repo_name': 'aiaioo/DeepLearningBasicsTutorial', 'size': '4096'}