<jupyter_start><jupyter_text>ML Pipeline Preparation 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and y<jupyter_code># import libraries
import pandas as pd
import numpy as np
import matplotlib as plt
from sqlalchemy import create_engine
import re
import nltk
nltk.download(['punkt','stopwords','wordnet', 'averaged_perceptron_tagger'])
from nltk.tokenize import word_tokenize
from nltk.stem import WordNetLemmatizer
from nltk.corpus import stopwords

from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV
from sklearn.multioutput import MultiOutputClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
import pickle
# load data from database
engine = create_engine('sqlite:///../data/Disaster_response_data.db')
df = pd.read_sql('SELECT * FROM Disaster_data', engine)
df.head()
X = df['message']
y = df.iloc[:,4:]
X.head()
y.head()<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process text data<jupyter_code>def tokenize(text):
    # Normalize
    text = re.sub(r'[^a-zA-Z0-9]', ' ', text.lower())
    # Tokenization
    words = word_tokenize(text)
    # Remove stop words
    words = [w for w in words if w not in stopwords.words('english')]
    # Lemmatize
    lemmatizer = [WordNetLemmatizer().lemmatize(w) for w in words]
    return lemmatizer

tokenize(X[0])
# for message in X:
#     tokens = tokenize(message)
#     print(message)
#     print(tokens,'\n')<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline takes in the `message` column as input and output classification results on the other 36 categories in the dataset.<jupyter_code>pipeline = Pipeline([
    ('vect', CountVectorizer(tokenizer=tokenize)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(RandomForestClassifier(n_estimators=100))),
    ])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y)
X_train.head()
pipeline.fit(X_train, y_train)
# Predict on test data
y_pred = pipeline.predict(X_test)
y_pred
y_test.values
y_test.columns
def display_results(y_test, y_pred):
    """ Function to display test scores
    
    Arguments:
        y_test {np array} -- test set
        y_pred {np array} -- predicted result
    """
#     y_pred = model.predict(X_test)
    for i, col in enumerate(y_test):
        print(col)
        print(classification_report(y_test[col], y_pred[:, i]))<jupyter_output><empty_output><jupyter_text>5. Test the modelReport the f1 score, precision and recall for each output category of the dataset. This is done by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>for pred, label, col in zip(y_pred.transpose(), y_test.values.transpose(), y_test.columns):
    print(col)
    print(classification_report(label, pred))<jupyter_output>related
              precision    recall  f1-score   support

           0       0.69      0.42      0.52      1535
           1       0.84      0.94      0.89      4983
           2       0.23      0.42      0.29        36

    accuracy                           0.81      6554
   macro avg       0.59      0.59      0.57      6554
weighted avg       0.80      0.81      0.80      6554

request
              precision    recall  f1-score   support

           0       0.91      0.98      0.94      5451
           1       0.83      0.50      0.62      1103

    accuracy                           0.90      6554
   macro avg       0.87      0.74      0.78      6554
weighted avg       0.89      0.90      0.89      6554

offer
              precision    recall  f1-score   support

           0       1.00      1.00      1.00      6528
           1       0.00      0.00      0.00        26

    accuracy                           1.00      6554
   macro avg       0.50      0.50      0.50      655[...]<jupyter_text>6. Improve the modelUse grid search to find better parameters.<jupyter_code>pipeline.get_params()
# parameters = {
#               'clf__estimator__n_estimators': [100, 200], 
#               'clf__estimator__min_samples_split': [2, 5, 10],
#               } 
parameters = {
#              'clf__min_samples_leaf': 1,
                'clf__estimator__min_samples_split': (5, 10),  
                'clf__estimator__n_estimators': [20, 40]
             }
cv = GridSearchCV(pipeline, parameters, cv=5)

pipeline.get_params().keys()
cv.fit(X_train, y_train)
cv.best_params_
cv.best_estimator_<jupyter_output><empty_output><jupyter_text>7. Test the modelShow the accuracy, precision, and recall of the tuned model.<jupyter_code>y_pred_cv = cv.predict(X_test)
for i, col in enumerate(y_test):
    print(col)
    print(classification_report(y_test[col], y_pred_cv[:, i]))<jupyter_output>related
              precision    recall  f1-score   support

           0       0.70      0.43      0.54      1516
           1       0.84      0.94      0.89      4987
           2       0.45      0.33      0.38        51

    accuracy                           0.82      6554
   macro avg       0.66      0.57      0.60      6554
weighted avg       0.81      0.82      0.80      6554

request
              precision    recall  f1-score   support

           0       0.91      0.97      0.94      5403
           1       0.82      0.52      0.64      1151

    accuracy                           0.90      6554
   macro avg       0.86      0.75      0.79      6554
weighted avg       0.89      0.90      0.89      6554

offer
              precision    recall  f1-score   support

           0       1.00      1.00      1.00      6522
           1       0.00      0.00      0.00        32

    accuracy                           1.00      6554
   macro avg       0.50      0.50      0.50      655[...]<jupyter_text>8. Export your model as a pickle file<jupyter_code>with open('../models/model.pkl', 'wb') as file:
    pickle.dump(cv, file)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08606431852986217, 'fraction_numerical': 0.06140888208269525, 'mean_word_length': 2.1811982464685826, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 25, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '240818', 'license': 'FTL', 'n_tokens_mistral': 2264, 'n_tokens_neox': 2084, 'n_words': 549, 'path': 'data_preprocess/ML Pipeline Preparation.ipynb', 'repo_name': 'gj0706/disaster_response_pipeline', 'size': '6530'}