<jupyter_start><jupyter_text>Neural Networks *Data Science Unit 4 Sprint 2 Assignment 1*  Define the Following:You can add image, diagrams, whatever you need to ensure that you understand the concepts below. Input Layer: First layer of a neural network that contains all inputted information - no bias Hidden Layer:Layer between the input and output layers, can contain any number of neurons.  Output Layer:Last layer, uses activation function to produce the output of the model Neuron:information processing unit receives and input then transforms it via activation function Weight:Value for each neuron that multiplies the output of the activation function Activation Function:function applied to the information at the neuronal level Node Map:Visual diagram or map of the architecture of a given neural network Perceptron:Simplest of neural net consisting of single neuron  Inputs -> Outputs Explain the flow of information through a neural network from inputs to outputs. Be sure to include: inputs, weights, bias, and activation functions. How does it all flow from beginning to end? Data is fed to the input layer, at each layer of the NN weights and biases are applied, then data is transformed via activation function. The weights and biases determine the impact of the layers activation functions on the successive layer. This continues through n-layers until the output layer is reached at which point a final transformation takes place based on the type of problem to coalesce the final results.  Write your own perceptron code that can correctly classify (99.0% accuracy) a NAND gate. | x1 | x2 | y ||----|----|---|| 0  | 0  | 1 || 1  | 0  | 1 || 0  | 1  | 1 || 1  | 1  | 0 |<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = { 'x1': [0,1,0,1],
         'x2': [0,0,1,1],
         'y':  [1,1,1,0],
         'bias': [1,1,1,1]
       }

df = pd.DataFrame.from_dict(data).astype('int')
class Perceptron(object):
    
    def __init__(self, rate = .01, niter = 1000):
        self.rate = rate
        self.niter = niter
    
    @staticmethod
    def sigmoid(x):
        return 1/(1+np.exp(-x))
    
    def sigmoid_derivative(self, x):
        return self.sigmoid(x) * (1 - self.sigmoid(x))

    def fit(self, X, y):

        self.weight = 2 * np.random.random((1, X.shape[1])) - 1
        self.errors = [] 

        for i in range(self.niter):
            err = 0
            for xi, target in zip(X, y):
                pred = self.predict(xi)
                delta_w = self.rate * (target - pred) * self.sigmoid_derivative(xi)
                self.weight += delta_w * xi
                err += int(np.linalg.norm(delta_w) != 0.0)
            self.errors.append(err)
        return self

    def net_input(self, X):
        return np.dot(self.weight, X)

    def predict(self, X):
        return self.sigmoid(self.net_input(X))
p = Perceptron()
p.fit(df[['x1','x2', 'bias']].values, df['y'].values)
import matplotlib.pyplot as plt

pn = Perceptron(0.5, 75)
pn.fit(X, y)
plt.plot(range(1, len(pn.errors) + 1), pn.errors, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of misclassifications')
plt.show()<jupyter_output><empty_output><jupyter_text>Implement your own Perceptron Class and use it to classify a binary dataset: - [The Pima Indians Diabetes dataset](https://raw.githubusercontent.com/ryanleeallred/datasets/master/diabetes.csv) You may need to search for other's implementations in order to get inspiration for your own. There are *lots* of perceptron implementations on the internet with varying levels of sophistication and complexity. Whatever your approach, make sure you understand **every** line of your implementation and what its purpose is.<jupyter_code>diabetes = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/diabetes.csv')
diabetes.head()<jupyter_output><empty_output><jupyter_text>Although neural networks can handle non-normalized data, scaling or normalizing your data will improve your neural network's learning speed. Try to apply the sklearn `MinMaxScaler` or `Normalizer` to your diabetes dataset.<jupyter_code>from sklearn.preprocessing import MinMaxScaler, Normalizer, OrdinalEncoder
y = diabetes['Outcome']
encoded = OrdinalEncoder().fit_transform(diabetes)
diabetes_encoded = pd.DataFrame(encoded)
diabetes_encoded.columns = diabetes.columns
diabetes_encoded.head()
X = encoded[:,[0,8]]
class Perceptron(object):
  def __init__(self, rate = 0.01, niter = 10):
    self.rate = rate
    self.niter = niter

  def fit(self, X, y):
    """Fit training data
    X : Training vectors, X.shape : [#samples, #features]
    y : Target values, y.shape : [#samples]
    """

    self.weight = np.zeros(1 + X.shape[1]) # weights
    self.errors = []  # Number of misclassifications

    for i in range(self.niter):

      err = 0
      for xi, target in zip(X, y):
      
        delta_w = self.rate * (target - self.predict(xi))
        self.weight[1:] += delta_w * xi
        self.weight[0] += delta_w
        err += int(delta_w != 0.0)
      self.errors.append(err)
      
    return self

  def net_input(self, X):
    """Calculate net input"""
    return np.dot(X, self.weight[1:]) + self.weight[0]

  def predict(self, X):
    """Return class label after unit step"""
    return np.where(self.net_input(X) >= 0.0, 1, -1)
import matplotlib.pyplot as plt

pn = Perceptron(0.5, 75)
pn.fit(X, y)
plt.plot(range(1, len(pn.errors) + 1), pn.errors, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Number of misclassifications')
plt.show()
from sklearn.metrics import accuracy_score
from sklearn.metrics import roc_auc_score


train_predictions = pn.predict(X)

print(f'Train Accuracy: {(accuracy_score(y, train_predictions)*100):.02f}%')<jupyter_output>Train Accuracy: 34.90%
{'fraction_non_alphanumeric': 0.10901540058833709, 'fraction_numerical': 0.016784910884236028, 'mean_word_length': 3.605577689243028, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 14, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '375423', 'license': 'MIT', 'n_tokens_mistral': 1883, 'n_tokens_neox': 1755, 'n_words': 616, 'path': 'curriculum/unit-4-machine-learning/sprint-2-intro-nn/module1-Intro-to-Neural-Networks/LS_DS_421_Intro_to_NN_Assignment.ipynb', 'repo_name': 'BrianThomasRoss/lambda-school', 'size': '5779'}