<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import re
import pandas as pd
import numpy as np
from sqlalchemy import create_engine
import nltk
from nltk.corpus import stopwords
from nltk.stem.wordnet import WordNetLemmatizer
from nltk.tokenize import word_tokenize

nltk.download('punkt')
nltk.download('stopwords')
nltk.download('wordnet')

from sklearn.pipeline import Pipeline
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.multioutput import MultiOutputClassifier

# load data from database
engine = create_engine('sqlite:///disaster_response.db')
df = pd.read_sql_table('messages', engine)
# extract the message column
# this will be used to create the data matrix we will be predicting on 
print(df['message'][df['message'].apply(lambda x: '#' in x)])
X = df['message'].values
# extract the columns we want to predict
category_names = list(df.columns)[4:]
Y = df[category_names].values
print(category_names)
#Notice that many rows have multiple applicable categories!
print(df[category_names].sum(axis=1).head())<jupyter_output>15       A Comitee in Delmas 19, Rue ( street ) Janvier...
164      Hello, we live in La Plaine, close to Route Na...
186      I need help rescuing someone under the rubble ...
231      Hello carrefour truitier road jasmin villa B #...
351      HELP ME PLEASE GOD WILL REPAY YOU THANK YOU. A...
                               ...                        
13140    Hurricane sandy aftermath @Oakwood Beach, Stat...
16504                                               #NAME?
16804    Much of the negatively affected region has rec...
24909    ###For information, please contact: <NAME>...
25527    Despite the huge international support of the ...
Name: message, Length: 1050, dtype: object
['related', 'request', 'offer', 'aid_related', 'medical_help', 'medical_products', 'search_and_rescue', 'security', 'military', 'child_alone', 'water', 'food', 'shelter', 'clothing', 'money', 'missing_people', 'refugees', 'death', 'other_aid', 'infrastructure_related', 'transport', 'buildings', 'electricity[...]<jupyter_text>2. Write a tokenization function to process your text data There are three things we want to do:- normalize case- remove punctuation- split the text into tokens<jupyter_code>def tokenize(text):
    stop_words = stopwords.words("english")
    lemmatizer = WordNetLemmatizer()
    
    # substitute non-alpha-numeric characters with a space and change to lower case.
#    text = re.compile("[^a-zA-Z0-9]").sub(" ", text.lower())
    
    #TODO: do we need to look for hashtags and url's?  The corpus might contain tweets
    # If we remove the "#" at the beginning then the hashtag just becomes another word
    # but should we priorize the hashtags in the model?
    
    # tokenize text
    tokens = word_tokenize(text)
    
    # lemmatize and remove stop words
    clean_tokens = []
    for tok in tokens:
        if tok not in stop_words:
            clean_tok = lemmatizer.lemmatize(tok).lower().strip()
            clean_tokens.append(clean_tok)
    return clean_tokens<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>pipeline = Pipeline([
    ('vect', CountVectorizer(tokenizer=tokenize)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(RandomForestClassifier(n_jobs=4, random_state=42)))
])
pipeline.get_params()<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>import time
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state=42)
start = time.time()
pipeline.fit(X_train, Y_train)
print(f'training took {time.time()-start} seconds')
Y_pred = pipeline.predict(X_test)
print(Y_pred.shape)<jupyter_output>training took 123.27684354782104 seconds
(6545, 36)<jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>for i, col in enumerate(Y_pred.T):
    print(classification_report(Y_test[:,i], col))<jupyter_output>precision    recall  f1-score   support

          0       0.61      0.42      0.50      1521
          1       0.84      0.92      0.87      4981
          2       0.35      0.16      0.22        43

avg / total       0.78      0.80      0.78      6545

             precision    recall  f1-score   support

          0       0.89      0.97      0.93      5446
          1       0.76      0.41      0.53      1099

avg / total       0.87      0.88      0.86      6545

             precision    recall  f1-score   support

          0       0.99      1.00      1.00      6508
          1       0.00      0.00      0.00        37

avg / total       0.99      0.99      0.99      6545

             precision    recall  f1-score   support

          0       0.74      0.85      0.79      3825
          1       0.73      0.57      0.64      2720

avg / total       0.73      0.74      0.73      6545

             precision    recall  f1-score   support

          0       0.92      0.99 [...]<jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>X_train, X_test, Y_train, Y_test = train_test_split(X, Y, random_state=42)
max_depth = [int(x) for x in np.linspace(start=10, stop=110, num=11)]
max_depth.append(None)
parameters = {
    #number of trees in the random forest
    'clf__estimator__n_estimators': [int(x) for x in np.linspace(start=2, stop=200, num=5)],
    #number of features to consider at each split
    'clf__estimator__max_features': ['auto', 'sqrt'],
    #max num levels in tree
    'clf__estimator__max_depth': max_depth,
    #minimum number of samples required before splitting a node
    'clf__estimator__min_samples_split': [2, 5, 10],
    #minimum number of samples required at each leaf
    'clf__estimator__min_samples_leaf': [1, 2, 4],
    #do bootstraping
    'clf__estimator__bootstrap': [True, False]
    }
cv = RandomizedSearchCV(pipeline, param_distributions=parameters, n_iter=50, cv=3, random_state=42, n_jobs=-1)
cv.fit(X_train, Y_train)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0920978627671541, 'fraction_numerical': 0.04204161979752531, 'mean_word_length': 3.1669595782073814, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '121080', 'license': 'MIT', 'n_tokens_mistral': 2332, 'n_tokens_neox': 2157, 'n_words': 679, 'path': 'notebooks/ML Pipeline Preparation.ipynb', 'repo_name': 'grtyvr/DisasterRespnse-Udacity', 'size': '7116'}