<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import numpy as np
import pandas as pd
pd.set_option('display.max_columns', 500)

import sys
import os
import re
from sqlalchemy import create_engine
import pickle

from scipy.stats import gmean

# import relevant functions/modules from the sklearn
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import fbeta_score, make_scorer
from sklearn.ensemble import GradientBoostingClassifier, RandomForestClassifier, AdaBoostClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer
from sklearn.multioutput import MultiOutputClassifier
from sklearn.base import BaseEstimator,TransformerMixin

# create the regex for matching urls
url_regex = 'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+'

# import nltk and download packages
import nltk
nltk.download(['punkt', 'wordnet', 'averaged_perceptron_tagger'])
# load data from database
engine = create_engine('sqlite:///../data/DisasterResponse.db')
df = pd.read_sql_table('messages', engine)

# store the category names for later use
category_names = list(df.columns[4:])

df.describe()
# Remove child_alone column because it is all zeros
df = df.drop(['child_alone'],axis=1)

df.groupby("related").count()
# Extract X and y variables from the data for the modelling
X = df['message']
y = df.iloc[:,4:]<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code># function that takes text and returns clean tokens
def tokenize(text):
    # remove urls from the text
    detected_urls = re.findall(url_regex, text)
    for url in detected_urls:
        text = text.replace(url, "urlplaceholder")

    # create the tokens
    tokens = nltk.word_tokenize(text)
    lemmatizer = nltk.WordNetLemmatizer()

    # clean the tokens
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok).lower().strip()
        clean_tokens.append(clean_tok)

    return clean_tokens

# Class to extract the first verb of the text
class FirstVerbExtractor(BaseEstimator, TransformerMixin):
    # find the first verb
    def first_verb(self, text):
        sentence_list = nltk.sent_tokenize(text)
        for sentence in sentence_list:
            pos_tags = nltk.pos_tag(tokenize(sentence))
            first_word, first_tag = pos_tags[0]
            if first_tag in ['VB', 'VBP'] or first_word == 'RT':
                return True
        return False

    # Return itself 
    def fit(self, X, y=None):
        return self

    # Transform across the X series
    def transform(self, X):
        X_tagged = pd.Series(X).apply(self.first_verb)
        return pd.DataFrame(X_tagged)<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code># Create a pipeline using set features
pipeline_base = Pipeline([
        ('features', FeatureUnion([

            ('text_pipeline', Pipeline([
                ('count_vectorizer', CountVectorizer(tokenizer=tokenize)),
                ('tfidf_transformer', TfidfTransformer())
            ]))
            
        ])),

        ('classifier', MultiOutputClassifier(AdaBoostClassifier()))
    ])

# Create pipeline with the firstverb transformer
pipeline_firstverb = Pipeline([
        ('features', FeatureUnion([

            ('text_pipeline', Pipeline([
                ('count_vectorizer', CountVectorizer(tokenizer=tokenize)),
                ('tfidf_transformer', TfidfTransformer())
            ])),

            ('first_verb_transformer', FirstVerbExtractor())
        ])),

        ('classifier', MultiOutputClassifier(AdaBoostClassifier()))
    ])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code># Train the base pipeline
X_train, X_test, y_train, y_test = train_test_split(X, y)
pipeline_fitted = pipeline_base.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code># Test the model
y_prediction_train = pipeline_fitted.predict(X_train)
y_prediction_test = pipeline_fitted.predict(X_test)

# Print classification report on test data
print(classification_report(y_test.values, y_prediction_test, target_names=y.columns.values))
# Print classification report on training data
print('\n',classification_report(y_train.values, y_prediction_train, target_names=y.columns.values))<jupyter_output>precision    recall  f1-score   support

               related       0.83      0.94      0.88     15030
               request       0.78      0.56      0.65      3336
                 offer       0.35      0.10      0.15        94
           aid_related       0.77      0.61      0.68      8145
          medical_help       0.66      0.30      0.41      1565
      medical_products       0.71      0.34      0.46       988
     search_and_rescue       0.68      0.24      0.35       537
              security       0.40      0.06      0.11       355
              military       0.67      0.38      0.49       657
                 water       0.79      0.64      0.71      1251
                  food       0.81      0.68      0.74      2188
               shelter       0.81      0.56      0.66      1741
              clothing       0.78      0.46      0.58       309
                 money       0.67      0.35      0.46       478
        missing_people       0.71    [...]<jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code># Improve the model with grid search
parameters_grid = {'classifier__estimator__learning_rate': [0.01, 0.02, 0.05],
              'classifier__estimator__n_estimators': [10, 20, 40]}

cv = GridSearchCV(pipeline_base, param_grid=parameters_grid, scoring='f1_micro', n_jobs=-1)

# Fit the model again
cv.fit(X_train, y_train)
# Get the prediction values from the grid search cross validator
y_prediction_test = cv.predict(X_test)
y_prediction_train = cv.predict(X_train)<jupyter_output><empty_output><jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code># Print classification report on test data
print(classification_report(y_test.values, y_prediction_test, target_names=y.columns.values))
# Print classification report on training data
print('\n',classification_report(y_train.values, y_prediction_train, target_names=y.columns.values))<jupyter_output>precision    recall  f1-score   support

               related       0.76      1.00      0.87     15030
               request       0.61      0.38      0.47      3336
                 offer       1.00      0.02      0.04        94
           aid_related       0.80      0.19      0.31      8145
          medical_help       0.64      0.11      0.19      1565
      medical_products       0.71      0.01      0.01       988
     search_and_rescue       0.65      0.20      0.31       537
              security       0.00      0.00      0.00       355
              military       0.55      0.16      0.24       657
                 water       0.58      0.85      0.69      1251
                  food       0.77      0.68      0.72      2188
               shelter       0.84      0.30      0.45      1741
              clothing       0.50      0.00      0.01       309
                 money       1.00      0.01      0.01       478
        missing_people       0.68    [...]<jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF<jupyter_code>#Use pipeline_fitted which includes FirstVerbEstimator
X_train, X_test, y_train, y_test = train_test_split(X, y)
pipeline_fitted = pipeline_firstverb.fit(X_train, y_train)

y_prediction_train = pipeline_fitted.predict(X_train)
y_prediction_test = pipeline_fitted.predict(X_test)

# Print classification report on test data
print(classification_report(y_test.values, y_prediction_test, target_names=y.columns.values))
# Print classification report on training data
print('\n',classification_report(y_train.values, y_prediction_train, target_names=y.columns.values))<jupyter_output>precision    recall  f1-score   support

               related       0.82      0.95      0.88     15099
               request       0.79      0.54      0.64      3363
                 offer       0.64      0.10      0.17        89
           aid_related       0.77      0.60      0.67      8152
          medical_help       0.67      0.29      0.41      1568
      medical_products       0.69      0.34      0.46       988
     search_and_rescue       0.70      0.24      0.35       538
              security       0.56      0.13      0.21       351
              military       0.67      0.36      0.46       653
                 water       0.77      0.65      0.71      1227
                  food       0.81      0.70      0.75      2146
               shelter       0.80      0.56      0.66      1724
              clothing       0.75      0.47      0.58       281
                 money       0.64      0.35      0.45       455
        missing_people       0.66    [...]<jupyter_text>9. Export your model as a pickle file<jupyter_code>m = pickle.dumps('classifier.pkl')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07992442135096835, 'fraction_numerical': 0.05470004723665565, 'mean_word_length': 2.3341732283464567, 'pattern_counts': {'":': 0, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '47609', 'license': 'MIT', 'n_tokens_mistral': 3389, 'n_tokens_neox': 3120, 'n_words': 866, 'path': 'models/ML Pipeline Preparation.ipynb', 'repo_name': 'palouda/DisasterResponse', 'size': '10585'}