<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import pandas as pd
from sqlalchemy import create_engine
import nltk
nltk.download(['punkt', 'wordnet', 'stopwords'])
import re
import pandas as pd
from nltk.tokenize import word_tokenize
from nltk.stem import WordNetLemmatizer
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.metrics import confusion_matrix
from sklearn.multioutput import MultiOutputClassifier
from nltk.corpus import stopwords
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV
from sklearn.neighbors import KNeighborsClassifier
import pickle
engine = create_engine('sqlite:///DisasterResponse.db')
df = pd.read_sql_table('DisasterData', engine)
df = df.iloc[:1000]
df.shape
# load data from database
def load_data():
    engine = create_engine('sqlite:///DisasterResponse.db')
    df = pd.read_sql_table('DisasterData', engine)
    df = df.iloc[:1000]#since the dataset is too big to run, only the first 1000 rows were used.
    X = df['message']
    Y = df.iloc[:,5:]
    return X, Y
X, Y = load_data()
Y.head()<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>def tokenize(text):
    text = re.sub(r"[^a-zA-Z0-9]", " ", text)
    tokens = word_tokenize(text)
    tokens = [w for w in tokens if w not in stopwords.words('english')]
    lemmatizer = WordNetLemmatizer()
    clean_tokens = []
    for tok in tokens:
        clean_tok = lemmatizer.lemmatize(tok).lower().strip()
        clean_tokens.append(clean_tok)
    return clean_tokens<jupyter_output><empty_output><jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>pipeline = Pipeline([
    ('vect', CountVectorizer(tokenizer = tokenize)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(RandomForestClassifier()))
])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>X_train,X_test, Y_train, Y_test = train_test_split(X,Y)
pipeline.fit(X_train, Y_train)
Y_pred = pipeline.predict(X_test)
Y_pred
Y_pred = pd.DataFrame(Y_pred, columns = Y.columns)
Y_pred<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>from sklearn.metrics import classification_report
for column in Y.columns:
    print('\n ---- {} ----\n{}\n'.format(column,classification_report(Y_test[column],Y_pred[column])))
pipeline.get_params<jupyter_output><empty_output><jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>from sklearn.model_selection import GridSearchCV
parameters = {
    'vect__ngram_range':((1,1), (1,2)),
    'clf__estimator__n_estimators': [5,10]
}
cv = GridSearchCV(pipeline, parameters)
tuned_model = cv.fit(X_train, Y_train)
Y_pred_tuned = cv.predict(X_test)
Y_pred_tuned = pd.DataFrame(Y_pred_tuned, columns = Y.columns)
Y_pred_tuned
tuned_model.best_params_<jupyter_output><empty_output><jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>for column in Y.columns:
    print('\n ---- {} ----\n{}\n'.format(column,classification_report(Y_test[column],Y_pred_tuned[column])))<jupyter_output>---- related ----
             precision    recall  f1-score   support

          0       0.64      0.16      0.25        45
          1       0.83      0.98      0.90       202
          2       0.00      0.00      0.00         3

avg / total       0.78      0.82      0.77       250



 ---- request ----
             precision    recall  f1-score   support

          0       0.76      0.82      0.79       109
          1       0.85      0.80      0.82       141

avg / total       0.81      0.81      0.81       250



 ---- offer ----
             precision    recall  f1-score   support

          0       1.00      1.00      1.00       249
          1       0.00      0.00      0.00         1

avg / total       0.99      1.00      0.99       250



 ---- aid_related ----
             precision    recall  f1-score   support

          0       0.76      0.69      0.73       107
          1       0.78      0.84      0.81       143

avg / total       0.78      0.78      0.77       250



[...]<jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF<jupyter_code># using another algorithm: logistic regression
from sklearn.neighbors import KNeighborsClassifier
pipeline_KN = Pipeline([
    ('vect', CountVectorizer(tokenizer = tokenize)),
    ('tfidf', TfidfTransformer()),
    ('clf', MultiOutputClassifier(KNeighborsClassifier()))])
print(pipeline_KN.get_params())
parameters_KN = {
    'vect__ngram_range':((1,1), (1,2)),
    'clf__estimator__n_neighbors': [5,8]   
    }
cv_KN = GridSearchCV(pipeline_KN, parameters_KN)
cv_KN.fit(X_train,Y_train)
cv_KN.best_params_
Y_pred_KN = cv_KN.predict(X_test)
Y_pred_KN = pd.DataFrame(Y_pred_KN, columns = Y.columns)
Y_pred_KN
for column in Y.columns:
    print('\n ---- {} ----\n{}\n'.format(column,classification_report(Y_test[column],Y_pred_KN[column])))<jupyter_output>---- related ----
             precision    recall  f1-score   support

          0       0.70      0.36      0.47        45
          1       0.86      0.97      0.91       202
          2       0.00      0.00      0.00         3

avg / total       0.82      0.84      0.82       250



 ---- request ----
             precision    recall  f1-score   support

          0       0.73      0.63      0.68       109
          1       0.74      0.82      0.78       141

avg / total       0.74      0.74      0.73       250



 ---- offer ----
             precision    recall  f1-score   support

          0       1.00      1.00      1.00       249
          1       0.00      0.00      0.00         1

avg / total       0.99      1.00      0.99       250



 ---- aid_related ----
             precision    recall  f1-score   support

          0       0.78      0.59      0.67       107
          1       0.74      0.87      0.80       143

avg / total       0.76      0.75      0.75       250



[...]<jupyter_text>9. Export your model as a pickle file<jupyter_code>pickle.dump(cv, open('classifier.pkl', 'wb'))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09903985268972774, 'fraction_numerical': 0.04840194660002631, 'mean_word_length': 2.581723975506359, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 19, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '18403', 'license': 'MIT', 'n_tokens_mistral': 2614, 'n_tokens_neox': 2393, 'n_words': 652, 'path': 'notebook/ML Pipeline Preparation.ipynb', 'repo_name': 'ronger4242/DisasterPipeline', 'size': '7603'}