<commit_before>Introduction to astroNN Neural Nets
=======================================================

Available astroNN Neural Net Classes
--------------------------------------

All astroNN Neural Nets are inherited from some child classes which inherited NeuralNetMaster, NeuralNetMaster also
relies relies on two major component, `Normalizer` and `GeneratorMaster`

::

    Normalizer (astroNN.nn.utilities.normalizer.Normalizer)

    GeneratorMaster (astroNN.nn.utilities.generator.GeneratorMaster)
    ├── CNNDataGenerator
    ├── Bayesian_DataGenerator
    └── CVAE_DataGenerator

    NeuralNetMaster (astroNN.models.NeuralNetMaster.NeuralNetMaster)
    ├── CNNBase
    │   ├── ApogeeCNN
    │   ├── StarNet2017
    │   └── Cifar10CNN
    ├── BayesianCNNBase
    │   ├── MNIST_BCNN  # For authors testing only
    │   └── ApogeeBCNN
    ├── ConvVAEBase
    │   └── APGOEECVAE  # For authors testing only
    └── CGANBase
        └── GalaxyGAN2017  # For authors testing only

Creating Your Own Model with astroNN Neural Net Classes
----------------------------------------------------------

You can create your own neural network model inherits from astroNN Neural Network class to take advantage of the existing
code in this package. Here we will go throught how to create a simple model to do classification with MNIST dataset with
one convolutional layer and one fullly connected layer neural network.

lets create a python script named `custom_models.py` under an a.rbitrary folder, lets say `~\` which is your home folder,
add ``~\custom_models.py`` to astroNN configuration file.

.. code-block:: python

    # import everything we need

    # astroNN keras_import_manager will import tf.keras iautomatically if keras is not detected
    from astroNN.config import keras_import_manager
    # this is the astroNN neural net abstract class we will going to inherit from
    from astroNN.models.CNNBase import CNNBase

    keras = keras_import_manager()
    regularizers = keras.regularizers
    MaxPooling2D, Conv2D, Dense, Flatten, Activation, Input = keras.layers.MaxPooling2D, keras.layers.Conv2D, \
                                                              keras.layers.Dense, keras.layers.Flatten, \
                                                              keras.layers.Activation, keras.layers.Input

    class my_custom_model(BayesianCNNBase):

    def __init__(self, lr=0.005):
        # standard super for inheriting abstrack class
        super().__init__()

        # some default hyperparameters
        self._implementation_version = '1.0'
        self.initializer = 'he_normal'
        self.activation = 'relu'
        self.num_filters = [8]
        self.filter_len = (3, 3)
        self.pool_length = (4, 4)
        self.num_hidden = [128]
        self.max_epochs = 1
        self.lr = lr
        self.reduce_lr_epsilon = 0.00005

        self.task = 'classification'
        # you should set the targetname some that you know what those output neurones are representing
        # in this case the outpu the neurones are simply representing digits
        self.targetname = ['Zero', 'One', 'Two', 'Three', 'Four', 'Five', 'Six', 'Seven', 'Eight', 'Nine']

        # set default input norm mode to 255 to normalize images correctly
        self.input_norm_mode = 255
        # set default labels norm mode to 0 (equivalent to do nothing) to normalize labels correctly
        self.labels_norm_mode = 0

    def model(self):
        input_tensor = Input(shape=self.input_shape, name='input')
        cnn_layer_1 = Conv2D(kernel_initializer=self.initializer, padding="same", filters=self.num_filters[0],
                             kernel_size=self.filter_len)(input_tensor)
        activation_1 = Activation(activation=self.activation)(cnn_layer_1)
        maxpool_1 = MaxPooling2D(pool_size=self.pool_length)(activation_1)
        flattener = Flatten()(maxpool_1)
        layer_2 = Dense(units=self.num_hidden[0], kernel_initializer=self.initializer)(flattener)
        activation_2 = Activation(activation=self.activation)(layer_2)
        layer_3 = Dense(units=self.labels_shape, kernel_initializer=self.initializer)(activation_2)
        output = Activation(activation=self._last_layer_activation, name='output')(layer_3)

        model = Model(inputs=input_tensor, outputs=output)

        return model

Save the file and we can open python under the same location as the python script

.. code-block:: python

    # import everything we need
    from custom_models import my_custom_model
    from keras.datasets import mnist
    from keras.utils import np_utils<commit_msg>fix the code in tut [ci skip]<commit_after>Introduction to astroNN Neural Nets
=======================================================

Available astroNN Neural Net Classes
--------------------------------------

All astroNN Neural Nets are inherited from some child classes which inherited NeuralNetMaster, NeuralNetMaster also
relies relies on two major component, `Normalizer` and `GeneratorMaster`

::

    Normalizer (astroNN.nn.utilities.normalizer.Normalizer)

    GeneratorMaster (astroNN.nn.utilities.generator.GeneratorMaster)
    ├── CNNDataGenerator
    ├── Bayesian_DataGenerator
    └── CVAE_DataGenerator

    NeuralNetMaster (astroNN.models.NeuralNetMaster.NeuralNetMaster)
    ├── CNNBase
    │   ├── ApogeeCNN
    │   ├── StarNet2017
    │   └── Cifar10CNN
    ├── BayesianCNNBase
    │   ├── MNIST_BCNN  # For authors testing only
    │   └── ApogeeBCNN
    ├── ConvVAEBase
    │   └── APGOEECVAE  # For authors testing only
    └── CGANBase
        └── GalaxyGAN2017  # For authors testing only

Creating Your Own Model with astroNN Neural Net Classes
----------------------------------------------------------

You can create your own neural network model inherits from astroNN Neural Network class to take advantage of the existing
code in this package. Here we will go throught how to create a simple model to do classification with MNIST dataset with
one convolutional layer and one fullly connected layer neural network.

lets create a python script named `custom_models.py` under an a.rbitrary folder, lets say `~/` which is your home folder,
add ``~/custom_models.py`` to astroNN configuration file.

.. code-block:: python

    # import everything we need

    # astroNN keras_import_manager will import tf.keras iautomatically if keras is not detected
    from astroNN.config import keras_import_manager
    # this is the astroNN neural net abstract class we will going to inherit from
    from astroNN.models.CNNBase import CNNBase

    keras = keras_import_manager()
    regularizers = keras.regularizers
    MaxPooling2D, Conv2D, Dense, Flatten, Activation, Input = keras.layers.MaxPooling2D, keras.layers.Conv2D, \
                                                              keras.layers.Dense, keras.layers.Flatten, \
                                                              keras.layers.Activation, keras.layers.Input

    # now we are creating a custom model based on astroNN neural net abstract class
    class my_custom_model(CNNBase):
        def __init__(self, lr=0.005):
            # standard super for inheriting abstrack class
            super().__init__()

            # some default hyperparameters
            self._implementation_version = '1.0'
            self.initializer = 'he_normal'
            self.activation = 'relu'
            self.num_filters = [8]
            self.filter_len = (3, 3)
            self.pool_length = (4, 4)
            self.num_hidden = [128]
            self.max_epochs = 1
            self.lr = lr
            self.reduce_lr_epsilon = 0.00005

            self.task = 'classification'
            # you should set the targetname some that you know what those output neurones are representing
            # in this case the outpu the neurones are simply representing digits
            self.targetname = ['Zero', 'One', 'Two', 'Three', 'Four', 'Five', 'Six', 'Seven', 'Eight', 'Nine']

            # set default input norm mode to 255 to normalize images correctly
            self.input_norm_mode = 255
            # set default labels norm mode to 0 (equivalent to do nothing) to normalize labels correctly
            self.labels_norm_mode = 0

        def model(self):
            input_tensor = Input(shape=self.input_shape, name='input')
            cnn_layer_1 = Conv2D(kernel_initializer=self.initializer, padding="same", filters=self.num_filters[0],
                                 kernel_size=self.filter_len)(input_tensor)
            activation_1 = Activation(activation=self.activation)(cnn_layer_1)
            maxpool_1 = MaxPooling2D(pool_size=self.pool_length)(activation_1)
            flattener = Flatten()(maxpool_1)
            layer_2 = Dense(units=self.num_hidden[0], kernel_initializer=self.initializer)(flattener)
            activation_2 = Activation(activation=self.activation)(layer_2)
            layer_3 = Dense(units=self.labels_shape, kernel_initializer=self.initializer)(activation_2)
            output = Activation(activation=self._last_layer_activation, name='output')(layer_3)

            model = Model(inputs=input_tensor, outputs=output)

            return model

Save the file and we can open python under the same location as the python script

.. code-block:: python

    # import everything we need
    from custom_models import my_custom_model
    from keras.datasets import mnist
    from keras.utils import np_utils
{'fraction_non_alphanumeric': 0.10529094941387686, 'fraction_numerical': 0.012461717182384624, 'mean_word_length': 2.697774306911363, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6426819', 'n_tokens_mistral': 2708, 'n_tokens_neox': 2560, 'n_words': 815}