<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import sys
import numpy as np
import pandas as pd
import re
from sqlalchemy import create_engine

# import nltlk (Natural Language Toolkit)
import nltk
from nltk import pos_tag, ne_chunk
from nltk.tokenize import word_tokenize, sent_tokenize
from nltk.stem import WordNetLemmatizer
nltk.download(['punkt', 'wordnet', 'averaged_perceptron_tagger', 'stopwords'])

# import scikit-learn
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.metrics import f1_score, confusion_matrix, accuracy_score, classification_report
from sklearn.multioutput import MultiOutputClassifier

from sklearn.ensemble import RandomForestClassifier
from sklearn.naive_bayes import MultinomialNB

# import picke to save a trained model
import pickle
# load data from database
engine = create_engine('sqlite:///DisasterResponse.db')
df = pd.read_sql("SELECT * FROM DisasterResponse", engine)
X = df['message'].values
Y = df.iloc[:,4:]
category_names = Y.columns.tolist()
X.dtype
pd.set_option('display.max_columns', 500)
df.head()
df["num_words_original"]= df.message.str.split(" ").apply(len)
df["num_words_"]= df.message.str.split(" ")
# df.new.head()
df.head()<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>def tokenize(text):
    '''
    Function splits text into separate words and gets a word lowercased and removes whitespaces at the ends of a word. The funtions also cleans irrelevant stopwords.
    
    Input:
    1. text: text message
    
    Output:
    1. Clean_tokens : list of tokenized clean words
    '''
    # Get rid of other sepcial characters   
    text = re.sub(r"[^a-zA-Z0-9]", " ", text)
    # Tokenize
    tokens = word_tokenize(text)
    # Lemmatize
    lemmatizer = WordNetLemmatizer()
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok, pos='v').lower().strip()
        clean_tokens.append(clean_tok)
    
    # Remove stop words    
    stopwords = nltk.corpus.stopwords.words('english')
    clean_tokens = [token for token in clean_tokens if token not in stopwords]
    
    return clean_tokens<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>class TokenCounter(BaseEstimator, TransformerMixin):
    def count_tokens(self, text):
        tokens = nltk.word_tokenize(text)
        number_of_tokens = len(tokens)
        return  number_of_tokens
    
    def fit(self, X, y=None):
        return self

    def transform(self, X):
        X_tagged = pd.Series(X).apply(self.count_tokens)
        return pd.DataFrame(X_tagged)
pipeline = Pipeline([
    ('features', FeatureUnion([
        ('text_pipeline', Pipeline([
            ('vect', CountVectorizer(tokenizer=tokenize)),
            ('tfidf', TfidfTransformer())
        ])),
        ('counttk', TokenCounter())
    ])),
    ('clf', MultiOutputClassifier(RandomForestClassifier()))
])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, random_state =42)

print(type(y_train))
y_train.head()
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>def display_results(y_test, y_pred, category_names):
    for i in range(0, len(category_names)):
        print(category_names[i])
        print(classification_report(y_test[category_names[i]], y_pred[:, i]))
        print("f1_score of {} is {:.2f}".format(category_names[i], f1_score(y_test[category_names[i]], y_pred[:, i])))
        print("**"*30)
display_results(y_test, y_pred, category_names)<jupyter_output><empty_output><jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>pipeline.get_params()
parameters = {
    'features__text_pipeline__vect__max_df': (0.5,1.0),
    'clf__estimator__n_estimators': [50,100]
}

cv = GridSearchCV(pipeline, param_grid=parameters, cv=2, n_jobs=-1, verbose=10)
cv.fit(X_train, y_train)
cv.best_params_<jupyter_output>Fitting 2 folds for each of 4 candidates, totalling 8 fits
[CV] clf__estimator__n_estimators=50, features__text_pipeline__vect__max_df=0.5 
[CV]  clf__estimator__n_estimators=50, features__text_pipeline__vect__max_df=0.5, score=0.2525175961017867, total= 2.2min
[CV] clf__estimator__n_estimators=50, features__text_pipeline__vect__max_df=0.5<jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>pipeline.set_params(**cv.best_params_)
pipeline.fit(X_train, y_train)
y_pred_1 = pipeline.predict(X_test)
display_results(y_test, y_pred_1, category_names)<jupyter_output>related
             precision    recall  f1-score   support

          0       0.70      0.38      0.49      1844
          1       0.83      0.95      0.89      6072

avg / total       0.80      0.82      0.80      7916

f1_score of related is 0.89
************************************************************
request
             precision    recall  f1-score   support

          0       0.90      0.98      0.94      6560
          1       0.84      0.49      0.62      1356

avg / total       0.89      0.90      0.89      7916

f1_score of request is 0.62
************************************************************
offer
             precision    recall  f1-score   support

          0       1.00      1.00      1.00      7879
          1       1.00      0.03      0.05        37

avg / total       1.00      1.00      0.99      7916

f1_score of offer is 0.05
************************************************************
aid_related
             precision    recall  f1-score   support

  [...]<jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF<jupyter_code># Already TokenCounter class was used, so skip this.<jupyter_output><empty_output><jupyter_text>9. Export your model as a pickle file<jupyter_code>pickle.dump(pipeline, open("classifier.pkl", 'wb'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11352431712191872, 'fraction_numerical': 0.028914057295136575, 'mean_word_length': 4.03758389261745, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '177845', 'license': 'MIT', 'n_tokens_mistral': 2456, 'n_tokens_neox': 2349, 'n_words': 633, 'path': 'notebooks/ML Pipeline Preparation.ipynb', 'repo_name': 'dalpengholic/Udacity_Disaster_Response_Project', 'size': '7505'}