<commit_before>==============================================
Convolutional Neural Networks using TensorFlow
==============================================

This tutorial deals with training a classifier using convolutional
neural networks.

------------
Introduction
------------


In this tutorial we try to teach you how to implement a simple neural
network image classifier using **Convolutional Neural Networks(CNNs)**.
The main goal of this post is to show hot to train a CNN classifier
using `TensorFlow <https://www.tensorflow.org/>`__ deep learning
framework developed by Google. The deep learning concepts such as the
details of CNNs will not be discussed here. In order to get a better
idea of convolutional layers and realize how the work please refer to
`this
post <http://machinelearninguru.com/computer_vision/basics/convolution/convolution_layer.html>`__.
In the next section we start to describe procedure of learning the
classifier.

--------------
Input Pipeline
--------------

The dataset that we work on that in this tutorial is the
`MNIST <http://yann.lecun.com/exdb/mnist/>`__ dataset probably the most
famous dataset in computer vision because of its simplicity! The main
dataset consist of 60000 training and 10000 test images. However there
might be different setups for these images. The one we use is the same
in the test set but we split the training set to 55000 images as train
and 5000 images as validation set in the case that using
cross-validation for determining some hyper-parameters is desired. The
images are 28x28x1 which each of them represent a hand-written digit
from 0 to 9. Since this tutorial is supposed to be ready-to-use, we
provided the code to download and extract the MNIST data as a data
object. Thanks to TensorFlow its code is already written and is ready to
use and its source code is available at `this
repository <tensorflow.contrib.learn.python.learn.datasets.mnist>`__ .
The code for downloading and extracting MNIST dataset is as is as below:

.. code:: python

       from tensorflow.examples.tutorials.mnist import input_data
       import tensorflow as tf
       mnist = input_data.read_data_sets("MNIST_data/", reshape=False, one_hot=False)
       data = input.provide_data(mnist)


The above code download and extract MNIST data in the MNIST\_data/
folder in the current directory that we are running the python script.
The reshape flag is set to **False** because we want the image format as
it is which is 28x28x1. The reason is because we are aimed to train a
CNN classifier which takes images as input. If the one\_hot flag is set
to **True** it returns class labels as a one\_hot label. However we set
the one\_hot flag to **False** for customized preprocessing and data
organization. The **input.provide\_data** function is provided to get
any data with specific format separated by training and testing sets and
return the structured data object for further processing. From now on we
consider **data** as the data object.

In any of the train, validation and test attributes, sub-attributes of
images and labels exist. The have just not been depicted for the
simplicity of the above chart presentation. As an example if
**data.train.imege** is called its shape is
[number\_of\_training\_sample,28,28,1]. It is recommended to play around
a little bit with data object to grasp a better idea of how it works and
what is its output. The codes are available in the GitHub repository for
this post.

--------------------
Network Architecture
--------------------

After explanation of the data input pipeline, Now it's the time to go
through the neural network architecture used for this tutorial. The
implemented architecture is very similar to
`LeNet <http://yann.lecun.com/exdb/lenet/>`__ although our architecture
is implemented in a fully-convolutional fashion, i.e., there is no
fully-connected layer and all fully-connected layers are transform to
corresponding convolutional layer. In order to grasp a better idea of
how to go from a fully-connected layer to a convolutional one and vice
versa please refer to `this
link <http://cs231n.github.io/convolutional-networks/>`__. The general
architecture schematic is as below:

.. figure:: _img/architecture.png
   :scale: 50 %
   :alt: Architecture
   :align: center

   **Figure 1:** The general architecture of the network.

   
The image is depicted by
`Tensorboard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__
as a visualization tool for TensorFlow. Later on in this tutorial the
way of using Tensorboard and make the most of it will be explained. As
it can be seen by the figure, the convolutional layers are followed by
pooling layers and the last fully-connected layer is followed by a
dropout layer to decrease the overfitting. *The dropout will only be
applied in the training phase*. The code for designing the architecture
is as below:

.. code:: python

    import tensorflow as tf
    slim = tf.contrib.slim

    def net_architecture(images, num_classes=10, is_training=False,
                         dropout_keep_prob=0.5,
                         spatial_squeeze=True,
                         scope='Net'):

        # Create empty dictionary
        end_points = {}

        with tf.variable_scope(scope, 'Net', [images, num_classes]) as sc:
            end_points_collection = sc.name + '_end_points'<commit_msg>Update convolutional_neural_network.rst<commit_after>==============================================
Convolutional Neural Networks using TensorFlow
==============================================

This tutorial deals with training a classifier using convolutional
neural networks.

------------
Introduction
------------


In this tutorial we try to teach you how to implement a simple neural
network image classifier using **Convolutional Neural Networks(CNNs)**.
The main goal of this post is to show hot to train a CNN classifier
using `TensorFlow <https://www.tensorflow.org/>`__ deep learning
framework developed by Google. The deep learning concepts such as the
details of CNNs will not be discussed here. In order to get a better
idea of convolutional layers and realize how the work please refer to
`this
post <http://machinelearninguru.com/computer_vision/basics/convolution/convolution_layer.html>`__.
In the next section we start to describe procedure of learning the
classifier.

--------------
Input Pipeline
--------------

The dataset that we work on that in this tutorial is the
`MNIST <http://yann.lecun.com/exdb/mnist/>`__ dataset probably the most
famous dataset in computer vision because of its simplicity! The main
dataset consist of 60000 training and 10000 test images. However there
might be different setups for these images. The one we use is the same
in the test set but we split the training set to 55000 images as train
and 5000 images as validation set in the case that using
cross-validation for determining some hyper-parameters is desired. The
images are 28x28x1 which each of them represent a hand-written digit
from 0 to 9. Since this tutorial is supposed to be ready-to-use, we
provided the code to download and extract the MNIST data as a data
object. Thanks to TensorFlow its code is already written and is ready to
use and its source code is available at `this
repository <tensorflow.contrib.learn.python.learn.datasets.mnist>`__ .
The code for downloading and extracting MNIST dataset is as is as below:

.. code:: python

       from tensorflow.examples.tutorials.mnist import input_data
       import tensorflow as tf
       mnist = input_data.read_data_sets("MNIST_data/", reshape=False, one_hot=False)
       data = input.provide_data(mnist)


The above code download and extract MNIST data in the MNIST\_data/
folder in the current directory that we are running the python script.
The reshape flag is set to **False** because we want the image format as
it is which is 28x28x1. The reason is because we are aimed to train a
CNN classifier which takes images as input. If the one\_hot flag is set
to **True** it returns class labels as a one\_hot label. However we set
the one\_hot flag to **False** for customized preprocessing and data
organization. The **input.provide\_data** function is provided to get
any data with specific format separated by training and testing sets and
return the structured data object for further processing. From now on we
consider **data** as the data object.

In any of the train, validation and test attributes, sub-attributes of
images and labels exist. The have just not been depicted for the
simplicity of the above chart presentation. As an example if
**data.train.imege** is called its shape is
[number\_of\_training\_sample,28,28,1]. It is recommended to play around
a little bit with data object to grasp a better idea of how it works and
what is its output. The codes are available in the GitHub repository for
this post.

--------------------
Network Architecture
--------------------

After explanation of the data input pipeline, Now it's the time to go
through the neural network architecture used for this tutorial. The
implemented architecture is very similar to
`LeNet <http://yann.lecun.com/exdb/lenet/>`__ although our architecture
is implemented in a fully-convolutional fashion, i.e., there is no
fully-connected layer and all fully-connected layers are transform to
corresponding convolutional layer. In order to grasp a better idea of
how to go from a fully-connected layer to a convolutional one and vice
versa please refer to `this
link <http://cs231n.github.io/convolutional-networks/>`__. The general
architecture schematic is as below:

.. figure:: _img/architecture.png
   :scale: 50 %
   :alt: Architecture
   :align: "center"

   **Figure 1:** The general architecture of the network.

   
The image is depicted by
`Tensorboard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__
as a visualization tool for TensorFlow. Later on in this tutorial the
way of using Tensorboard and make the most of it will be explained. As
it can be seen by the figure, the convolutional layers are followed by
pooling layers and the last fully-connected layer is followed by a
dropout layer to decrease the overfitting. *The dropout will only be
applied in the training phase*. The code for designing the architecture
is as below:

.. code:: python

    import tensorflow as tf
    slim = tf.contrib.slim

    def net_architecture(images, num_classes=10, is_training=False,
                         dropout_keep_prob=0.5,
                         spatial_squeeze=True,
                         scope='Net'):

        # Create empty dictionary
        end_points = {}

        with tf.variable_scope(scope, 'Net', [images, num_classes]) as sc:
            end_points_collection = sc.name + '_end_points'
{'fraction_non_alphanumeric': 0.08481646273637375, 'fraction_numerical': 0.008527994067482388, 'mean_word_length': 3.7424175824175823, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 4, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6075484', 'n_tokens_mistral': 2844, 'n_tokens_neox': 2639, 'n_words': 1430}