<jupyter_start><jupyter_text>Task 01 :**What are the different types of Activation Function used in Neural Networks? Briefly explain them.** There are four commonly used activation functions.i) **Sigmoid Function:** A mathematical function having a characteristic "S"-shaped curve or sigmoid curve. Its output ranges from 0 to 1. This can be defined by \\$$g(z) = \frac{1}{1+e^{-z}}$$ii) **Tanh Function:** This is a rescaled logistic sigmoid function. Its output ranges from -1 to 1. This can be defined by \\$$g(z) = \frac{e^{z}+e^{-z}}{e^{z}-e^{-z}}$$iii) **ReLU (Rectifier Linear Unit) Function:** Defined as the positive part of its argument where z is the input to a neuron. This can be defined by \\$$g(z) = max(0, z)$$iv) **Leaky ReLU (Rectifier Linear Unit) Function:** Leaky ReLUs are an attempt to fix the *dying ReLU* problem. Instead of the function being zero when x < 0, a leaky ReLU will instead have a small negative slope $\alpha$ . This can be defined by \\$$g(z) = max(\alpha z, z)$$  Task 02 :**Why are activation funtions important in Neural Network and when do we use them?** Importance of activation functions : i) **Introducing non-linearity :** Non-linear functions make back-propagation possible. Without non-linear function, back-propargation is useless. Back-propagation minimizes the error and enhances the accuracy or efficiency of the model. Moreover, without non-linearity logistic regression with deep neural network is useless. Briefly, it's nothing but logistic regression with extra steps and more time.ii) **Thresholding during classification :** An activation function serves as a threshold, alternatively called classification or a partition.  Task 03 :**Given that for a two class problem, w1, w2 = [x1, x2] contains two feature vectors andthey are w1 = [(1,0), (0,1), (1,1)] & w2 = [(1,2), (0,3), (2,1)]. Implement the batch weightupdate for 5 epochs. [Initially assume the weights are 0.1 and learning rate is 0.3].N.B: Implement this without the help of any library.**<jupyter_code>def sigmoid(z):
    s = 1/(1+np.exp(-z))
    return s
def propagate(w, b, X, Y):
    
    #forward propagation
    m = X.shape[1] #no of samples
    A = sigmoid(np.matmul(w.T, X)+ b) #activation
    cost = -1/m*np.sum(Y*np.log(A)+ (1-Y)*np.log(1-A)) #cost function

    #backward propagation
    dw = (1/m)*(np.matmul(X, (A-Y).T)) 
    db = (1/m)*np.sum(A-Y)

    return dw, db, cost
def optimize(w, b, X, Y, num_iterations, learning_rate):
    
    for i in range(num_iterations):
        dw, db, cost = propagate(w, b, X, Y)

        #Weight Update
        w = w - learning_rate*dw
        b = b - learning_rate*db
        print ("Cost after iteration %i: %f" %(i+1, cost))

    return
w = np.array([[.1],[.1]])
b = 0.1
X = np.array([[1.,0,1.,1.,0,2.],[0,1.,1.,2.,3.,1.]])
Y = np.array([[0,0,0,1,1,1]])
optimize(w, b, X, Y, num_iterations= 5, learning_rate = 0.3)<jupyter_output>Cost after iteration 1: 0.664946
Cost after iteration 2: 0.650251
Cost after iteration 3: 0.638410
Cost after iteration 4: 0.628234
Cost after iteration 5: 0.619109<jupyter_text>Task 04:**Install Tensorflow and load the Breast Cancer Dataset given to you.N.B: The dataset is already split into training and testing sets.**<jupyter_code>import tensorflow as tf
from tensorflow import keras
from keras import layers, regularizers
from keras.layers import Input, Dense, Activation, BatchNormalization, Dropout
from keras.models import Model, Sequential
from keras.utils import normalize, to_categorical
import keras.backend as K
X_train = pd.read_excel('/content/drive/My Drive/Colab Notebooks/Machine Learning Lab/Lab 6/X_train.xlsx')
X_train
X_test = pd.read_excel('/content/drive/My Drive/Colab Notebooks/Machine Learning Lab/Lab 6/X_test.xlsx')
X_test
Y_train = pd.read_excel('/content/drive/My Drive/Colab Notebooks/Machine Learning Lab/Lab 6/Y_train.xlsx')
Y_train
Y_test = pd.read_excel('/content/drive/My Drive/Colab Notebooks/Machine Learning Lab/Lab 6/Y_test.xlsx')
Y_test<jupyter_output><empty_output><jupyter_text>Task 05:**Build your Neural Network model by importing the Sequential Model from Keras library.Note: Since Breast Cancer is a 30 feature dataset use Relu activation function for yourinput & hidden layers and Sigmoid activation function for the output layer.**<jupyter_code>X_train = normalize(X_train)
X_test = normalize(X_test)
model = tf.keras.models.Sequential([
  keras.layers.Dense(64,input_shape=(X_train.shape[1],),name='input_layer'),
  keras.layers.Activation(tf.nn.relu,name='relu_1'),
  keras.layers.Dense(128,name='hidden_layer_1'),
  keras.layers.Activation(tf.nn.relu,name='relu_2'),
  keras.layers.Dense(64,name='hidden_layer_2'),
  keras.layers.Activation(tf.nn.relu,name='relu_3'),
  keras.layers.Dense(32,name='hidden_layer_3'),
  keras.layers.Activation(tf.nn.relu,name='relu_4'),
  keras.layers.Dense(16,name='hidden_layer_4'),
  keras.layers.Activation(tf.nn.relu,name='relu_5'),
  keras.layers.Dense(1,name='output_layer'),
  keras.layers.Activation(tf.nn.sigmoid,name='sigmoid'),
])


model.summary()<jupyter_output>Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_layer (Dense)          (None, 64)                1984      
_________________________________________________________________
relu_1 (Activation)          (None, 64)                0         
_________________________________________________________________
hidden_layer_1 (Dense)       (None, 128)               8320      
_________________________________________________________________
relu_2 (Activation)          (None, 128)               0         
_________________________________________________________________
hidden_layer_2 (Dense)       (None, 64)                8256      
_________________________________________________________________
relu_3 (Activation)          (None, 64)                0         
______________________________________________________[...]<jupyter_text>Task 06:**Calculate the loss function and optimizer for the Network.Hint: Use classifier.compile() to perform this. Since this is a binary classification problemfor the ’loss’ parameter use ’binary_crossentropy’.**<jupyter_code>adam =  keras.optimizers.Adam(learning_rate=0.001)
model.compile(optimizer=adam, loss='binary_crossentropy', metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Task 07:**Train your Neural Network model using the fit function and set the parameter ’epochs’ as100 and ’batch_size’ as 1.**<jupyter_code>history = model.fit(X_train, Y_train,batch_size=1, epochs=100)
# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()<jupyter_output>dict_keys(['loss', 'accuracy'])<jupyter_text>Task 08:**Evaluate your Neural Network to check whether the model is best fit for the given problemand corresponding data. Print the test accuracy and test loss.Hint: Use classifier.evaluate() function to implement this.**<jupyter_code>preds = model.evaluate(X_test, Y_test,batch_size=1)
print ("Loss = " + str(preds[0]))
print ("Test Accuracy = " + str(preds[1]*100.0) + "%")<jupyter_output>114/114 [==============================] - 0s 926us/step - loss: 0.2689 - accuracy: 0.9649
Loss = 0.26888787746429443
Test Accuracy = 96.49122953414917%<jupyter_text>Task 09:**Predict your model using the predict() function and print the correctly & incorrectlyclassified samples out of the total test samples.Hint: Since you used a sigmoid function to get your output, the predicted values arebetween 0 & 1. Thus in order to make them binary use 0.5 as a threshold. For instance,ifpredicted output >= 0.5, y_pred = 1 else y_pred = 0.**<jupyter_code>y_pred = model.predict(X_test, batch_size=1)
y_pred = y_pred >= 0.5 
y_pred = [(1 if True in x else 0) for x in y_pred]
y_pred = np.asarray(y_pred)
Y_test = np.asarray(Y_test)
total = Y_test.shape[0]
print("Total test Samples: ",total)
correct = 0
for i in range(y_pred.shape[0]):
  if y_pred[i] == Y_test[i] :
    correct = correct + 1
print("Correctly detected: ",correct)
print("Incorrectly detected = ",total-correct)<jupyter_output>Total test Samples:  114
Correctly detected:  110
Incorrectly detected =  4<jupyter_text>Task 10:**Show the Confusion Matrix and visualize it with the help of seaborn library.**<jupyter_code>mat = confusion_matrix(y_pred, Y_test)
heat = sns.heatmap(mat,square=True,annot=True,fmt='d',cbar=True,cmap=plt.cm.gist_heat)
class_label=['Has No Cancer','Has Cancer']
heat.set_xticklabels(class_label)
heat.set_yticklabels(class_label)
heat.set_xlabel('Predicted Label')
heat.set_ylabel('True Label')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1308123557216665, 'fraction_numerical': 0.03132900956359239, 'mean_word_length': 4.915474642392718, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '587340', 'license': 'MIT', 'n_tokens_mistral': 3089, 'n_tokens_neox': 2820, 'n_words': 856, 'path': 'implementations/TF_Keras.ipynb', 'repo_name': 'msi1427/msiai', 'size': '9097'}