<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning) 

import string
import pandas as pd 
from sqlalchemy import create_engine
import re
import pickle



import nltk
from nltk.tokenize import word_tokenize

from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.multioutput import MultiOutputClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import classification_report, accuracy_score
# load data from database
engine = create_engine('sqlite:///DisasterResponse.db')

query = "SELECT * From DisasterResponse"
df = pd.read_sql_query(query, engine,)

X = df['message'].values
Y = df.drop(columns=['id', 'message', 'original', 'genre'])<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>stop_words = nltk.corpus.stopwords.words("english")
lemmatizer = nltk.stem.wordnet.WordNetLemmatizer()

def tokenize(text):
    '''
    Description:
        - Normalization to lowercase
        - Remove punctuation characters
        - Tokenization, lemmatization, and stop word removal
    
    Arguments:
        - text as string
    
    Returns:
        - tokens: a list of tockenized text
    '''
    
    #normalize text and remove punctuation
    text = re.sub(r"[^a-zA-Z0-9]", " ", text.lower())
    
    # tokenize text
    tokens = word_tokenize(text)
    
    # lemmatize andremove stop words
    tokens = [lemmatizer.lemmatize(word) for word in tokens if word not in stop_words]
    
    return tokens
#test of tokenization function

text = "Sentence first. Sentence second. Sentence third"
tokenize(text)<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>pipeline = Pipeline([
    ('vect', CountVectorizer(tokenizer=tokenize)),
    ('tfidf', TfidfTransformer()),
    ('forest', MultiOutputClassifier( RandomForestClassifier() ))
    ])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state =30, test_size =0.33)
pipeline.fit(X_train, Y_train)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>def model_performance(model, x_test, y_test):
    y_predicted = model.predict(x_test)
    y_predicted_df = pd.DataFrame(y_predicted, columns = y_test.columns)

    overall_accuracy = (y_predicted == y_test).mean().mean()
    print("Overall accuracy: {0} \n".format(overall_accuracy))

    for column in y_test.columns:
        print('Feature: {}\n'.format(column))
        print('Accuracy: ', accuracy_score(y_test[column], y_predicted_df[column]))
        print(classification_report(y_test[column],y_predicted_df[column]))
        print('------------------------------------------------------\n')
model_performance(pipeline, X_test, Y_test)<jupyter_output>Overall accuracy: 0.9439042584661294 

Feature: related

Accuracy:  0.8064590809121426
             precision    recall  f1-score   support

          0       0.62      0.47      0.54      1983
          1       0.85      0.91      0.88      6598
          2       0.50      0.14      0.22        58

avg / total       0.79      0.81      0.80      8639

------------------------------------------------------

Feature: request

Accuracy:  0.8858664197245052
             precision    recall  f1-score   support

          0       0.90      0.98      0.93      7138
          1       0.80      0.46      0.58      1501

avg / total       0.88      0.89      0.87      8639

------------------------------------------------------

Feature: offer

Accuracy:  0.996064359300845
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      8606
          1       0.00      0.00      0.00        33

avg / total       0.99      1.00      0.99      8639

----------[...]<jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>pipeline2 = Pipeline([
    ('vect', CountVectorizer(tokenizer=tokenize)),
    ('tfidf', TfidfTransformer()),
    ('forest', MultiOutputClassifier( RandomForestClassifier() ))
    ])

parameters = {
        'vect__ngram_range': ((1, 1), (1, 2)) 
}

# another possible parameters - skipped to optimize run time
# 'vect__max_features': (5000, 10000),
#' clf__estimator__min_samples_split': [2, 4],
# ...

grid_search = GridSearchCV(pipeline2, param_grid=parameters, verbose=5, n_jobs = 4)
grid_search.fit(X_train, Y_train)<jupyter_output>Fitting 3 folds for each of 2 candidates, totalling 6 fits
[CV] vect__ngram_range=(1, 1) ........................................
[CV]  vect__ngram_range=(1, 1), score=0.2468354430379747, total=  36.9s
[CV] vect__ngram_range=(1, 1) ........................................<jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>model_performance(grid_search, X_test, Y_test)<jupyter_output>Overall accuracy: 0.9422644081748144 

Feature: related

Accuracy:  0.791874059497627
             precision    recall  f1-score   support

          0       0.57      0.50      0.53      1983
          1       0.85      0.88      0.87      6598
          2       0.29      0.28      0.28        58

avg / total       0.78      0.79      0.79      8639

------------------------------------------------------

Feature: request

Accuracy:  0.882162287301771
             precision    recall  f1-score   support

          0       0.89      0.98      0.93      7138
          1       0.79      0.44      0.56      1501

avg / total       0.87      0.88      0.87      8639

------------------------------------------------------

Feature: offer

Accuracy:  0.9961801134390554
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      8606
          1       0.00      0.00      0.00        33

avg / total       0.99      1.00      0.99      8639

-----------[...]<jupyter_text>8. Export your model as a pickle file<jupyter_code>with open("model.pickle", 'wb') as pickle_file:
    pickle.dump(grid_search, pickle_file)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13305696710350362, 'fraction_numerical': 0.06311848087723991, 'mean_word_length': 2.415068493150685, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '370961', 'license': 'MIT', 'n_tokens_mistral': 2507, 'n_tokens_neox': 2256, 'n_words': 617, 'path': 'classification_of_messages/models/ML Pipeline Preparation.ipynb', 'repo_name': 'JMarcan/ml_and_natural_language_processing', 'size': '7478'}