<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code>import pickle
import string
import unittest
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np
from sqlalchemy import create_engine
import nltk
nltk.download('punkt')
nltk.download('stopwords')
nltk.download('wordnet')
nltk.download('averaged_perceptron_tagger')

from nltk.tokenize import word_tokenize, sent_tokenize
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
from nltk.stem.wordnet import WordNetLemmatizer
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.ensemble import RandomForestClassifier
from sklearn.multioutput import MultiOutputClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import classification_report, f1_score, make_scorer
from sklearn.base import BaseEstimator, TransformerMixin
# load data from database
engine = create_engine('sqlite:///DisasterResponse.db')
df = pd.read_sql("SELECT * FROM Disaster", engine)
# Since the original messages are in multiple languages droping it for now
df.drop(['original','genre','id'],inplace=True,axis=1)
#df.set_index('id',inplace=True)
print(df.shape)
df.head(2)
X = df.message.values
Y = df[df.columns[1:]]
Y.head(3)<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>def tokenize(text):
    tokens=word_tokenize(text)
    lemmatizer=WordNetLemmatizer()
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok).lower().strip()
        clean_tokens.append(clean_tok)

    return clean_tokens<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>pipeline = Pipeline([
    ('vect',CountVectorizer(tokenizer=tokenize)),
    ('tfidf',TfidfTransformer()),
    ('clf',MultiOutputClassifier(RandomForestClassifier()))
])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, Y)
pipeline.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.Reference: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html```classification_report(y_true, y_pred, target_names=target_names)```We need to iterate through every class for train and test , with column name<jupyter_code>def get_result(y_pred,y_test):

    results_dict = {}

    for pred, label, col in zip(y_pred.transpose(), y_test.values.transpose(), y_test.columns):
        #print(col)
        #print(classification_report(label, pred))
        results_dict[col] = classification_report(label, pred,output_dict=True)
    weighted_avg = {}
    for key in results_dict.keys():
        weighted_avg[key] = results_dict[key]['weighted avg']

    df_wavg = pd.DataFrame(weighted_avg).transpose()
    return df_wavg
y_pred=pipeline.predict(X_test)
df_wavg=get_result(y_pred,y_test)
df_wavg.head()<jupyter_output><empty_output><jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>pipeline.get_params()
parameters = {
    'vect__ngram_range': ((1, 1), (1, 2)),
    'vect__max_df': [0.5],
    'vect__max_features': [5000],
    'clf__estimator__max_depth': (25, 50, 100, None),
    'clf__estimator__min_samples_split': (2, 10, 25, 50, 100), 
    'clf__estimator__n_estimators': [200]
}
cv = GridSearchCV(pipeline, parameters, cv=5, n_jobs=3)<jupyter_output><empty_output><jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>cv.fit(X_train, y_train)
cv.best_params_
cv.best_score_

y_preds = cv.predict(X_test)
results_cv = get_result(y_preds,y_test)
results_cv.head()<jupyter_output><empty_output><jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF  A.  Let's improve are tokenizer by removing stop words, lemmatizing and stemming<jupyter_code>import re
def tokenize(text):
    text = re.sub(r"[^a-zA-Z0-9]", " ", text.lower())
    words = word_tokenize(text)
    words = [word for word in words if word not in stopwords.words('english')]
    lemmed = [WordNetLemmatizer().lemmatize(word) for word in words]
    lemmed = [WordNetLemmatizer().lemmatize(word, pos='v') for word in lemmed]
    stemmed = [PorterStemmer().stem(word) for word in lemmed]
    return stemmed<jupyter_output><empty_output><jupyter_text>B. Lets add additional features using custom transformer<jupyter_code>class WordCount(BaseEstimator, TransformerMixin):
    def word_count(self, text):
        table = text.maketrans(dict.fromkeys(string.punctuation))
        words = word_tokenize(text.lower().strip().translate(table))
        return len(words)
    
    def fit(self, x, y=None):
        return self
    
    def transform(self, x):
        count = pd.Series(x).apply(self.word_count)
        return pd.DataFrame(count)
pipeline = Pipeline([

    ('feature',FeatureUnion([
        ('text',Pipeline([
        ('vect',CountVectorizer(tokenizer=tokenize, max_df=0.5, 
                                 max_features=5000, ngram_range=(1, 2),
                               )),
        ('tfidf',TfidfTransformer())
    ])),
    ('word_count',WordCount())

    ])),
   ("clf", MultiOutputClassifier(RandomForestClassifier(min_samples_split=2, random_state=42, verbose=3)))
])
pipeline.fit(X_train, y_train)
y_pred=pipeline.predict(X_test)
df_wavg=get_result(y_pred,y_test)
df_wavg.head()<jupyter_output>[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    1.0s finished
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    1.4s finished
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s
[Parallel(n_jobs=1)]: Done 100 out of 100 | elapsed:    0.4s finished
[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concur[...]<jupyter_text>9. Export your model as a pickle file<jupyter_code>with open('model.pkl', 'wb') as file:
    pickle.dump(pipeline, file)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10771360261733988, 'fraction_numerical': 0.017616710708443437, 'mean_word_length': 5.146945088940448, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '91230', 'license': 'MIT', 'n_tokens_mistral': 2703, 'n_tokens_neox': 2582, 'n_words': 661, 'path': 'ML Pipeline Preparation.ipynb', 'repo_name': 'maitreytalware/Disaster-Response-Pipeline', 'size': '7947'}