<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
import nltk
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import fbeta_score, make_scorer
from sklearn.ensemble import GradientBoostingClassifier, RandomForestClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.feature_extraction.text import TfidfTransformer, CountVectorizer
from sklearn.multioutput import MultiOutputClassifier
import pickle
nltk.download('punkt')
nltk.download('wordnet')
# load data from database
engine = create_engine('sqlite:///disaster_response.db')
df = pd.read_sql('SELECT * FROM messages_df', engine)
X = df.filter(items=['id', 'message', 'original', 'genre'])
y = df.drop(['id', 'message', 'original', 'genre', 'child_alone'], axis=1)#'child_alone' has no responses
#Mapping the '2' values in 'related' to '1' - because I consider them as a response (that is, '1')
y['related']=y['related'].map(lambda x: 1 if x == 2 else x)
df.head()<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>def tokenize(text):
    tokens = nltk.word_tokenize(text)
    lemmatizer = nltk.WordNetLemmatizer()
    return [lemmatizer.lemmatize(w).lower().strip() for w in tokens]<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>pipeline = Pipeline([('cvect', CountVectorizer(tokenizer = tokenize)),
                     ('tfidf', TfidfTransformer()),
                     ('clf', RandomForestClassifier())
                     ])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y)
pipeline.fit(X_train['message'], y_train)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>y_pred_test = pipeline.predict(X_test['message'])
y_pred_train = pipeline.predict(X_train['message'])
print('-----------------------------On testing data set----------------------------------------')
print(classification_report(y_test.values, y_pred_test, target_names=y.columns.values))
print('-----------------------------On training data set----------------------------------------')
print('\n',classification_report(y_train.values, y_pred_train, target_names=y.columns.values))<jupyter_output>-----------------------------On testing data set----------------------------------------
                        precision    recall  f1-score   support

               related       0.85      0.90      0.87      5073
               request       0.85      0.40      0.54      1135
                 offer       0.00      0.00      0.00        34
           aid_related       0.78      0.43      0.55      2778
          medical_help       0.62      0.02      0.03       530
      medical_products       0.74      0.05      0.09       340
     search_and_rescue       0.67      0.01      0.02       161
              security       0.00      0.00      0.00       109
              military       0.00      0.00      0.00       243
                 water       0.87      0.22      0.36       403
                  food       0.85      0.24      0.38       750
               shelter       0.83      0.09      0.16       591
              clothing       0.75      0.08      0.15       106
              [...]<jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>parameters = {'clf__max_depth': [10, 20, None],
              'clf__min_samples_leaf': [1, 2, 4],
              'clf__min_samples_split': [2, 5, 10],
              'clf__n_estimators': [10, 20, 40]}

cv = GridSearchCV(pipeline, param_grid=parameters, scoring='f1_micro', verbose=1, n_jobs=-1)
cv.fit(X_train['message'], y_train)<jupyter_output>Fitting 3 folds for each of 81 candidates, totalling 243 fits<jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>y_pred_test = cv.predict(X_test['message'])
y_pred_train = cv.predict(X_train['message'])
print('--------------------------------Testing Data-----------------------------------------------')
print(classification_report(y_test.values, y_pred_test, target_names=y.columns.values))
print('--------------------------------Traning Data-----------------------------------------------')
print('\n',classification_report(y_train.values, y_pred_train, target_names=y.columns.values))<jupyter_output>--------------------------------Testing Data-----------------------------------------------
                        precision    recall  f1-score   support

               related       0.83      0.95      0.89      5073
               request       0.85      0.42      0.56      1135
                 offer       0.00      0.00      0.00        34
           aid_related       0.81      0.52      0.63      2778
          medical_help       0.67      0.01      0.02       530
      medical_products       0.71      0.03      0.06       340
     search_and_rescue       0.00      0.00      0.00       161
              security       0.00      0.00      0.00       109
              military       1.00      0.00      0.01       243
                 water       0.89      0.21      0.34       403
                  food       0.88      0.29      0.44       750
               shelter       0.87      0.11      0.20       591
              clothing       0.50      0.01      0.02       106
           [...]<jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF<jupyter_code>pipeline2 = Pipeline([('cvect', CountVectorizer(tokenizer = tokenize)),
                     ('tfidf', TfidfTransformer()),
                     ('clf', GradientBoostingClassifier())
                     ])

X_train2, X_test2, y_train2, y_test2 = train_test_split(X, y)
pipeline.fit(X_train2['message'], y_train2)
y_pred_test2 = pipeline.predict(X_test2['message'])
y_pred_train2 = pipeline.predict(X_train2['message'])
print('-----------------------------On testing data set----------------------------------------')
print(classification_report(y_test2.values, y_pred_test2, target_names=y.columns.values))
print('-----------------------------On training data set----------------------------------------')
print('\n',classification_report(y_train2.values, y_pred_train2, target_names=y.columns.values))<jupyter_output>-----------------------------On testing data set----------------------------------------
                        precision    recall  f1-score   support

               related       0.83      0.90      0.87      4995
               request       0.80      0.37      0.51      1096
                 offer       0.00      0.00      0.00        30
           aid_related       0.79      0.42      0.55      2709
          medical_help       0.48      0.02      0.04       497
      medical_products       0.92      0.04      0.08       302
     search_and_rescue       0.67      0.01      0.02       200
              security       0.00      0.00      0.00       117
              military       0.00      0.00      0.00       213
                 water       0.83      0.09      0.16       419
                  food       0.85      0.20      0.32       734
               shelter       0.78      0.07      0.12       579
              clothing       0.83      0.05      0.09       100
              [...]<jupyter_text>9. Export your model as a pickle file<jupyter_code>cv.best_params_
m = pickle.dumps('clf')
pickle.dump(cv, open('disaster_model.sav', 'wb'))
loaded_model = pickle.load(open('disaster_model.sav', 'rb'))

print(loaded_model)
y_pred_test = loaded_model.predict(X_test['message'])
y_pred_train = loaded_model.predict(X_train['message'])
print('--------------------------------Testing Data-----------------------------------------------')
print(classification_report(y_test.values, y_pred_test, target_names=y.columns.values))
print('--------------------------------Traning Data-----------------------------------------------')
print('\n',classification_report(y_train.values, y_pred_train, target_names=y.columns.values))<jupyter_output>--------------------------------Testing Data-----------------------------------------------
                        precision    recall  f1-score   support

               related       0.83      0.95      0.89      5073
               request       0.85      0.42      0.56      1135
                 offer       0.00      0.00      0.00        34
           aid_related       0.81      0.52      0.63      2778
          medical_help       0.67      0.01      0.02       530
      medical_products       0.71      0.03      0.06       340
     search_and_rescue       0.00      0.00      0.00       161
              security       0.00      0.00      0.00       109
              military       1.00      0.00      0.01       243
                 water       0.89      0.21      0.34       403
                  food       0.88      0.29      0.44       750
               shelter       0.87      0.11      0.20       591
              clothing       0.50      0.01      0.02       106
           [...]
{'fraction_non_alphanumeric': 0.16312191416635016, 'fraction_numerical': 0.06598936574249906, 'mean_word_length': 1.5356283100625903, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '564354', 'license': 'FTL', 'n_tokens_mistral': 3188, 'n_tokens_neox': 2887, 'n_words': 763, 'path': 'notebook/ML Pipeline Preparation.ipynb', 'repo_name': 'omtripathi786/disaster_response_pipelines', 'size': '10532'}