<jupyter_start><jupyter_text>Models subpackage tutorial The NeuralModel class is a generic class used to manage neural networks implemented with Keras. It offers methods to save, load, train and use for classification the neural networks.Melusine provides two built-in Keras model : cnn_model and rnn_model based on the models used in-house at Maif. However the user is free to implement neural networks tailored for its needs.  The dataset The NeuralModel class can take as input either :- a text input : a cleaned text, usually the cleaned body or the concatenation of the cleaned body and the cleaned header.- a text input and a metadata input : the metadata input has to be dummified.  Text input<jupyter_code>import pandas as pd

df_emails_preprocessed = pd.read_csv('./data/emails_preprocessed.csv', encoding='utf-8', sep=';')
df_emails_preprocessed['clean_header'] = df_emails_preprocessed['clean_header'].astype(str)
df_emails_preprocessed['clean_body'] = df_emails_preprocessed['clean_body'].astype(str)
df_emails_preprocessed.columns<jupyter_output><empty_output><jupyter_text>The new clean_text column is the concatenation of the clean_header column and the clean_body column :<jupyter_code>df_emails_preprocessed['clean_text'] = df_emails_preprocessed['clean_header'] + " " + df_emails_preprocessed['clean_body']


df_emails_preprocessed.clean_text[0]<jupyter_output><empty_output><jupyter_text>Metadata input By default the metadata used are :- the extension : gmail, outlook, wanadoo..- the day of the week at which the email has been sent- the hour at which the email has been sent- the minute at which the email has been sent<jupyter_code>df_meta = pd.read_csv('./data/metadata.csv', encoding='utf-8', sep=';')
df_meta.columns
df_meta.head()<jupyter_output><empty_output><jupyter_text>Defining X and y X is a Pandas dataframe with a clean_text column that will be used for the text input and columns containing the dummified metadata.<jupyter_code>X = pd.concat([df_emails_preprocessed['clean_text'],df_meta],axis=1)<jupyter_output><empty_output><jupyter_text>y is a numpy array containing the encoded labels :<jupyter_code>from sklearn.preprocessing import LabelEncoder
y = df_emails_preprocessed['label']
le = LabelEncoder()
y = le.fit_transform(y)
y<jupyter_output><empty_output><jupyter_text>The NeuralModel class<jupyter_code>from melusine.models.train import NeuralModel<jupyter_output><empty_output><jupyter_text>The NeuralModel class is a generic class used to manage neural networks implemented with Keras. It offers methods to save, load, train and use for classification the neural networks.Its arguments are :- **architecture_function :** a function returning a Model instance from Keras.- **pretrained_embedding :** the pretrained embedding matrix as an numpy array.- **text_input_column :** the name of the column that will provide the text input, by default clean_text.- **meta_input_list :** the list of the names of the columns containing the metadata. If empty list or None the model is used without metadata. Default value, ['extension', 'dayofweek', 'hour', 'min'].- **vocab_size :** the size of vocabulary for neurol network model. Default value, 25000.- **seq_size :** the maximum size of input for neural model. Default value, 100.- **loss :** the loss function for training. Default value, 'categorical_crossentropy'.- **batch_size :** the size of batches for the training of the neural network model. Default value, 4096.- **n_epochs :** the number of epochs for the training of the neural network model. Default value, 15.  architecture_function<jupyter_code>from melusine.models.neural_architectures import cnn_model, rnn_model<jupyter_output><empty_output><jupyter_text>**architecture_function** is a function returning a Model instance from Keras.Melusine provides two built-in neural networks : **cnn_model** and **rnn_model** based on the models used in-house at Maif.  pretrained_embedding The embedding have to be trained on the user's dataset.<jupyter_code>from melusine.nlp_tools.embedding import Embedding
pretrained_embedding = Embedding().load('./data/embedding.pickle')<jupyter_output><empty_output><jupyter_text>NeuralModel used with text and metadata input This neural network model will use the **clean_text** column for the text input and the dummified **extension**, **dayofweek**, **hour** and **min** as metadata input :<jupyter_code>nn_model = NeuralModel(architecture_function=cnn_model,
                       pretrained_embedding=pretrained_embedding,
                       text_input_column="clean_text",
                       meta_input_list=['extension', 'dayofweek', 'hour', 'min'],
                       n_epochs=10)<jupyter_output><empty_output><jupyter_text>Training the neural network During the training, logs are saved in "train" situated in the data directory. Use tensorboard to follow training using - "tensorboard --logdir data" from your terminal  - directly from a notebook with "%load_ext tensorboard" and "%tensorboard --logdir data" magics command (see https://www.tensorflow.org/tensorboard/tensorboard_in_notebooks)<jupyter_code>nn_model.fit(X,y,tensorboard_log_dir="./data")<jupyter_output>Epoch 1/10
1/1 [==============================] - 0s 2ms/step - loss: 2.4012 - accuracy: 0.0250
Epoch 2/10
1/1 [==============================] - 0s 2ms/step - loss: 2.3427 - accuracy: 0.2250
Epoch 3/10
1/1 [==============================] - 0s 4ms/step - loss: 2.2729 - accuracy: 0.3000
Epoch 4/10
1/1 [==============================] - 0s 3ms/step - loss: 2.1311 - accuracy: 0.3750
Epoch 5/10
1/1 [==============================] - 0s 1ms/step - loss: 2.0898 - accuracy: 0.3250
Epoch 6/10
1/1 [==============================] - 0s 2ms/step - loss: 2.0259 - accuracy: 0.2750
Epoch 7/10
1/1 [==============================] - 0s 2ms/step - loss: 2.0069 - accuracy: 0.2750
Epoch 8/10
1/1 [==============================] - 0s 1ms/step - loss: 1.9776 - accuracy: 0.3000
Epoch 9/10
1/1 [==============================] - 0s 1ms/step - loss: 1.9858 - accuracy: 0.3500
Epoch 10/10
1/1 [==============================] - 0s 3ms/step - loss: 1.9276 - accuracy: 0.4000<jupyter_text>![title](../docs/_static/tensorboard.png)  Saving the neural network The **save_nn_model** method saves :- the Keras model as a json file - the weights as a h5 file<jupyter_code>nn_model.save_nn_model("./data/nn_model")<jupyter_output><empty_output><jupyter_text>Once the **save_nn_model** used the NeuralModel object can be saved as a pickle file :<jupyter_code>import joblib
_ = joblib.dump(nn_model,"./data/nn_model.pickle",compress=True)<jupyter_output><empty_output><jupyter_text>Loading the neural network The NeuralModel saved as a pickle file has to be loaded first :<jupyter_code>nn_model = joblib.load("./data/nn_model.pickle")<jupyter_output><empty_output><jupyter_text>Then the Keras model and its weights can be loaded :<jupyter_code>nn_model.load_nn_model("./data/nn_model")<jupyter_output><empty_output><jupyter_text>Making predictions<jupyter_code>y_res = nn_model.predict(X)
y_res = le.inverse_transform(y_res)
y_res<jupyter_output><empty_output><jupyter_text>NeuralModel used with only text input<jupyter_code>X = df_emails_preprocessed[['clean_text']]
nn_model = NeuralModel(architecture_function=cnn_model,
                       pretrained_embedding=pretrained_embedding,
                       text_input_column="clean_text",
                       meta_input_list=None,
                       n_epochs=10)
nn_model.fit(X,y)
y_res = nn_model.predict(X)
y_res = le.inverse_transform(y_res)
y_res<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13419286935929484, 'fraction_numerical': 0.02552295750559137, 'mean_word_length': 5.42603550295858, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 64, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '286518', 'license': 'Apache-2.0', 'n_tokens_mistral': 2503, 'n_tokens_neox': 2308, 'n_words': 697, 'path': 'tutorial/tutorial07_models.ipynb', 'repo_name': 'AntoineSimoulin/melusine', 'size': '7601'}