<jupyter_start><jupyter_text>Convolutional Neural Networkshttps://www.learnopencv.com/image-classification-using-convolutional-neural-networks-in-keras/In machine learning, a convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks, most commonly applied to analyzing visual imagery.CNNs use a variation of multilayer perceptrons designed to require minimal preprocessing. They are also known as shift invariant or space invariant artificial neural networks (SIANN), based on their shared-weights architecture and translation invariance characteristics.Convolutional networks were inspired by biological processes in that the connectivity pattern between neurons resembles the organization of the animal visual cortex. Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.CNNs use relatively little pre-processing compared to other image classification algorithms. This means that the network learns the filters that in traditional algorithms were hand-engineered. This independence from prior knowledge and human effort in feature design is a major advantage.They have applications in image and video recognition, recommender systems and natural language processing.![title](1cnn-schema.jpg)A CNN consists of an input and an output layer, as well as multiple hidden layers. The hidden layers of a CNN typically consist of convolutional layers, pooling layers, fully connected layers and normalization layers.Description of the process as a convolution in neural networks is by convention. Mathematically it is a cross-correlation rather than a convolution. This only has significance for the indices in the matrix, and thus which weights are placed at which index. ConvolutionalThe convolutional layer can be thought of as the eyes of the CNN. The neurons in this layer look for specific features. If they find the features they are looking for, they produce a high activation.Convolution can be thought of as a weighted sum between two signals ( in terms of signal processing jargon ) or functions ( in terms of mathematics ). In image processing, to calculate convolution at a particular location (x, y), we extract k x k sized chunk from the image centered at location (x,y). We then multiply the values in this chunk element-by-element with the convolution filter (also sized k x k) and then add them all to obtain a single output. Although fully connected feedforward neural networks can be used to learn features as well as classify data, it is not practical to apply this architecture to images. A very high number of neurons would be necessary, even in a shallow (opposite of deep) architecture, due to the very large input sizes associated with images, where each pixel is a relevant variable. For instance, a fully connected layer for a (small) image of size 100 x 100 has 10000 weights for each neuron in the second layer. The convolution operation brings a solution to this problem as it reduces the number of free parameters, allowing the network to be deeper with fewer parameters. For instance, regardless of image size, tiling regions of size 5 x 5, each with the same shared weights, requires only 25 learnable parameters. In this way, it resolves the vanishing or exploding gradients problem in training traditional multi-layer neural networks with many layers by using backpropagation.![title](3test.gif) PoolingAnother important concept of CNNs is pooling, which is a form of non-linear down-sampling. There are several non-linear functions to implement pooling among which max pooling is the most common. It partitions the input image into a set of non-overlapping rectangles and, for each such sub-region, outputs the maximum. The intuition is that the exact location of a feature is less important than its rough location relative to other features. The pooling layer serves to progressively reduce the spatial size of the representation, to reduce the number of parameters and amount of computation in the network, and hence to also control overfitting. It is common to periodically insert a pooling layer between successive convolutional layers in a CNN architecture. The pooling operation provides another form of translation invariance.![title](2Max_pooling.png) Fully connectedFully connected layers connect every neuron in one layer to every neuron in another layer. It is in principle the same as the traditional multi-layer perceptron neural network (MLP). Receptive fieldIn neural networks, each neuron receives input from some number locations in the previous layer. In a fully-connected layer, each neuron receives input from every element of the previous layer. In a convolutional layer, neurons receive input from only a restricted subarea of the previous layer. Typically the subarea is of a square shape (e.g., size 5 by 5). The input area of a neuron is called its receptive field. So, in a fully connected layer, the receptive field is the entire previous layer. In a convolutional layer, the receptive area is smaller than the entire previous layer. WeightsEach neuron in a neural network computes an output value by applying some function to the input values coming from the receptive field in the previous layer. The function that is applied to the input values is specified by a vector of weights and a bias (typically real numbers). Learning in a neural network progresses by making incremental adjustments to the biases and weights. The vector of weights and the bias are called a filter and represents some feature of the input (e.g., a particular shape). A distinguishing feature of CNNs is that many neurons share the same filter. This reduces memory footprint because a single bias and a single vector of weights is used across all receptive fields sharing that filter, rather than each receptive field having its own bias and vector of weights. 1. Each CNN unit has a k x k filter. This filter is passed over entire image when input of that image is given to CNN unit.2. Each CNN unit will start learning some aspect of image3. Different units will start learning differnt parts of the images for eg. for car type image some units will learn about wheels and some units will learn about doors, headlights, etc.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from sklearn.cross_validation import train_test_split 
from sklearn import preprocessing
import time
from keras.datasets import cifar10
from keras.models import Sequential
from keras.layers import Dense, Activation, Conv2D, MaxPooling2D, Dropout,Flatten
from sklearn import datasets
#(xtrain,ytrain),(xtest,ytest)=cifar10.load_data() 
import cifar10
cifar10.data_path = "data/CIFAR-10/"
cifar10.maybe_download_and_extract()

xtrain=cifar10.load_training_data()
xtrain[0].shape, xtrain[1].shape
xtrain,xval,ytrain,yval=train_test_split(xtrain[0],xtrain[1],test_size=0.1,random_state=1)
fig=plt.figure(figsize=(8,8))
for i in range(16):
    ax=fig.add_subplot(4,4,i+1)
    ax.imshow(xtrain[i])
plt.show()
oec = preprocessing.OneHotEncoder(sparse=False)
ytr=oec.fit_transform(ytrain.reshape((len(ytrain),1)))
yval=oec.transform(yval.reshape((len(yval),1)))
def createModel(inshape,nClasses):
    model = Sequential()
    model.add(Conv2D(32, (3, 3), padding='same', activation='relu', input_shape=inshape))
    model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
    model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))

    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))
 
    model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
 
    model.add(Flatten())
    
    model.add(Dense(64, activation='relu'))
    model.add(Dense(128, activation='relu'))
    model.add(Dense(nClasses, activation='softmax'))
     
    return model
model=createModel((32,32,3),10)
model.summary()
batch_size = 200
epochs =15
start=time.time()
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
 
history=model.fit(xtrain, ytr, batch_size=batch_size, epochs=epochs,validation_data=(xval,yval))
 
model.evaluate(xtrain, ytr)
end=time.time()-start
end
xtest=cifar10.load_test_data()
ypred=model.predict(xtest[0])
ypred=np.argmax(ypred,1)
from sklearn.metrics import confusion_matrix
confusion_matrix(xtest[1], ypred)
from sklearn.metrics import classification_report
print(classification_report(xtest[1], ypred))
plt.figure(figsize=[8,6])
plt.plot(history.history['loss'],'r',linewidth=3.0)
plt.plot(history.history['val_loss'],'b',linewidth=3.0)
plt.legend(['Training loss', 'Validation Loss'],fontsize=18)
plt.xlabel('Epochs ',fontsize=16)
plt.ylabel('Loss',fontsize=16)
plt.title('Loss Curves',fontsize=16)
 
# Accuracy Curves
plt.figure(figsize=[8,6])
plt.plot(history.history['acc'],'r',linewidth=3.0)
plt.plot(history.history['val_acc'],'b',linewidth=3.0)
plt.legend(['Training Accuracy', 'Validation Accuracy'],fontsize=18)
plt.xlabel('Epochs ',fontsize=16)
plt.ylabel('Accuracy',fontsize=16)

plt.title('Accuracy Curves',fontsize=16)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0711197578901859, 'fraction_numerical': 0.014591439688715954, 'mean_word_length': 5.642498205312275, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '426960', 'license': 'Apache-2.0', 'n_tokens_mistral': 2548, 'n_tokens_neox': 2347, 'n_words': 1135, 'path': 'notes/21 keras CNN/NN_keras.ipynb', 'repo_name': 'subhendu17620/ML-Coding-Ninjas', 'size': '9252'}