<jupyter_start><jupyter_text>The Perceptron AlgorithmThe main and ultimate aim is to find the continous function, here the Sigmoid function, rather than the discrete function that was being used initially, the step function. ![Perceptron Logic](Img/Perceptron_image.png)<jupyter_code>import numpy as np
# Setting the random seed, feel free to change it and see different solutions.
np.random.seed(42)

def stepFunction(t):
    if t >= 0:
        return 1
    return 0

def prediction(X, W, b):
    return stepFunction((np.matmul(X,W)+b)[0])

# TODO: Fill in the code below to implement the perceptron trick.
# The function should receive as inputs the data X, the labels y,
# the weights W (as an array), and the bias b,
# update the weights and bias W, b, according to the perceptron algorithm,
# and return W and b.
def perceptronStep(X, y, W, b, learn_rate = 0.01):
    # Fill in code
    for i in range(len(X)):
        y_hat = prediction(X[i],W,b)
        if y[i]-y_hat == 1:
            W[0] += X[i][0]*learn_rate
            W[1] += X[i][1]*learn_rate
            b += learn_rate
        elif y[i]-y_hat == -1:
            W[0] -= X[i][0]*learn_rate
            W[1] -= X[i][1]*learn_rate
            b -= learn_rate    
    return W, b

# This function runs the perceptron algorithm repeatedly on the dataset,
# and returns a few of the boundary lines obtained in the iterations,
# for plotting purposes.
# Feel free to play with the learning rate and the num_epochs,
# and see your results plotted below.
def trainPerceptronAlgorithm(X, y, learn_rate = 0.01, num_epochs = 25):
    x_min, x_max = min(X.T[0]), max(X.T[0])
    y_min, y_max = min(X.T[1]), max(X.T[1])
    W = np.array(np.random.rand(2,1))
    b = np.random.rand(1)[0] + x_max
    # These are the solution lines that get plotted below.
    boundary_lines = []
    for i in range(num_epochs):
        # In each epoch, we apply the perceptron step.
        W, b = perceptronStep(X, y, W, b, learn_rate)
        boundary_lines.append((-W[0]/W[1], -b/W[1]))
    return boundary_lines<jupyter_output><empty_output><jupyter_text>Building a Perceptron-based classifier<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Load input dataEach line contains space-separated numbers where the first two numbers are the features and the last number is the label<jupyter_code>text = np.loadtxt('data_perceptron.txt')
print(text)<jupyter_output>[[0.38 0.19 0.  ]
 [0.17 0.31 0.  ]
 [0.29 0.54 0.  ]
 [0.89 0.55 1.  ]
 [0.78 0.36 1.  ]]<jupyter_text>Separate the text into datapoints and labels<jupyter_code>data = text[:, :2]
print(data)
print("text.shape[0] =", text.shape[0]) # 5 - no. of rows in the dataset
# Shape an array to hold 5 rows and 1 column (based on 3rd column of dataset)
labels = text[:, 2].reshape((text.shape[0], 1)) # 5 x 1 matrix
print(labels)<jupyter_output>text.shape[0] = 5
[[0.]
 [0.]
 [0.]
 [1.]
 [1.]]<jupyter_text>Plot input data<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline

# Creates a new figure object.
plt.figure(figsize=(6,4)) # Width=6", height=4"
plt.scatter(data[:,0], data[:,1]) # A scatter plot of y vs x 
plt.xlabel('Dimension 1 (X)')
plt.ylabel('Dimension 2 (Y)')
plt.title('Input data')<jupyter_output><empty_output><jupyter_text>Define minimum and maximum values for each dimension<jupyter_code>dim1_min, dim1_max, dim2_min, dim2_max = 0, 1, 0, 1<jupyter_output><empty_output><jupyter_text>Since the data is separated into two classes, we just need one bit to represent the output. So the output layer will contain a single neuron.<jupyter_code># Number of neurons in the output layer
num_output = labels.shape[1]
print("num_output =", num_output)<jupyter_output>num_output = 1<jupyter_text>We have a dataset where the datapoints are 2-dimensional. Let's define a Perceptron with2 input neurons, where we assign one neuron for each dimension.<jupyter_code># pip install neurolab

# Neurolab - a library of basic neural networks algorithms with 
# flexible network configurations and learning algorithms for Python. 
import neurolab as nl

dim1 = [dim1_min, dim1_max]
print("dim1 =", dim1)

dim2 = [dim2_min, dim2_max]
print("dim2 =", dim2)

# In the 1st argument, the length of the list “[[0, 1],[0, 1]” is 2, 
# which indicates that there are 2 inputs. 
# Each item in this list has 2 elements. 
# 1st element indicates the minimum value that our input can take. 
# 2nd element indicates the maximum value our input can take. 
# The value “1” in 2nd argument indicates that there is a single neuron in this network.
perceptron = nl.net.newp([dim1, dim2], num_output)<jupyter_output>dim1 = [0, 1]
dim2 = [0, 1]<jupyter_text>Train the perceptron with the training data<jupyter_code># The number of epochs indicates the number of training cycles. 
# “show” - refers to the printing frequency on the terminal.
# “lr” - refers to the learning rate of the algorithm. 
# Learning rate refers to the step size that’s taken to adjust 
# the parameters so that we move towards the end goal.
error_progress = perceptron.train(data, labels, epochs=100, show=20, lr=0.03)<jupyter_output>The goal of learning is reached<jupyter_text>Plot the training progress using the error metric<jupyter_code>plt.figure()
plt.plot(error_progress)
plt.xlabel('Number of epochs')
plt.ylabel('Training error')
plt.title('Training error progress')
plt.grid()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11121374468871235, 'fraction_numerical': 0.02715684463329023, 'mean_word_length': 4.2057692307692305, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '408719', 'license': 'MIT', 'n_tokens_mistral': 1942, 'n_tokens_neox': 1824, 'n_words': 638, 'path': 'Perceptron_Introduction.ipynb', 'repo_name': 'Not-A-Builder/Basics-of-AI', 'size': '5413'}