<jupyter_start><jupyter_text>ML Pipeline PreparationFollow the instructions below to help you create your ML pipeline. 1. Import libraries and load data from database.- Import Python libraries- Load dataset from database with [`read_sql_table`](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_sql_table.html)- Define feature and target variables X and Y<jupyter_code># import libraries
import pandas as pd
import numpy as np
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer
from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.multioutput import MultiOutputClassifier
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
from sqlalchemy import create_engine
import pickle


# download NLTK data
import re
import nltk
nltk.download(['punkt', 'wordnet','stopwords'])

from nltk.stem import WordNetLemmatizer
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
# load data from database
engine = create_engine('sqlite:///messages.db')
df = pd.read_sql_table('messages',engine)
X = df['message']
Y = df.iloc[:,4:]
categories = list(df.columns[4:])
X.head()
Y.head()<jupyter_output><empty_output><jupyter_text>2. Write a tokenization function to process your text data<jupyter_code>def tokenize(text):
    '''
    Applies Natural Language Processing to raw text, namely: normalizes case, removes punctuation and english stop words, tokenizes and lemmatizes words.
    
    Args:
    text: str - raw message (text) to be cleaned
    
    Returns:
    tokens: cleaned, tokenized and lemmatized text
    '''
    
    #Normalize case and remove punctuation
    text = re.sub(r'[^a-zA-Z0-9]',' ' , text.lower())
    
    #Split text into words
    tokens = word_tokenize(text)
    
    # Initiate Lemmatizer
    lemmatizer = WordNetLemmatizer()
    
    #Lemmatize and remove stop words
    tokens = [lemmatizer.lemmatize(w) for w in tokens if w not in stopwords.words('english')]
    
    return tokens

#test the tokenize function
for message in X[:5]:
    tokens=tokenize(message)
    print(message)
    print(tokens, '\n')<jupyter_output>Weather update - a cold front from Cuba that could pass over Haiti
['weather', 'update', 'cold', 'front', 'cuba', 'could', 'pas', 'haiti'] 

Is the Hurricane over or is it not over
['hurricane'] 

Looking for someone but no name
['looking', 'someone', 'name'] 

UN reports Leogane 80-90 destroyed. Only Hospital St. Croix functioning. Needs supplies desperately.
['un', 'report', 'leogane', '80', '90', 'destroyed', 'hospital', 'st', 'croix', 'functioning', 'need', 'supply', 'desperately'] 

says: west side of Haiti, rest of the country today and tonight
['say', 'west', 'side', 'haiti', 'rest', 'country', 'today', 'tonight']<jupyter_text>3. Build a machine learning pipelineThis machine pipeline should take in the `message` column as input and output classification results on the other 36 categories in the dataset. You may find the [MultiOutputClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.multioutput.MultiOutputClassifier.html) helpful for predicting multiple target variables.<jupyter_code>#ML Pipeline using Random Forest Classifier
pipeline = Pipeline([
        ('vect', CountVectorizer(tokenizer=tokenize)),
        ('tfidf', TfidfTransformer()),
        ('clf', MultiOutputClassifier(RandomForestClassifier()))
    ])<jupyter_output><empty_output><jupyter_text>4. Train pipeline- Split data into train and test sets- Train pipeline<jupyter_code>#Split data into train and test
X_train, X_test, y_train, y_test = train_test_split(X, Y)

#Train pipeline
pipeline.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>5. Test your modelReport the f1 score, precision and recall for each output category of the dataset. You can do this by iterating through the columns and calling sklearn's `classification_report` on each.<jupyter_code>#Predict on test data
y_pred = pipeline.predict(X_test)

for i in range(Y.shape[1]):
    print('Category:', Y.columns[i], '\n', classification_report(y_test.iloc[:,1].values, y_pred[:,i]))    
accuracy = (y_pred == y_test).mean()

avg_accuracy = accuracy.mean()

print("Accuracy:", accuracy)
print("Average Accuracy:", avg_accuracy)
#Function to calculate basic statistics for total accuracy of the model
def calculate_stats(accuracy):
    '''
    Takes a list of accuracies and calculates the basic statistics, like minimum, maximum, mean and median
    
    Args:
    accuracy: str - list of accuracies for each category
    
    Returns: non
    '''
    minimum = accuracy.min()
    maximum = accuracy.max()
    mean = accuracy.mean()
    median = accuracy.median()
    
    return print('Min:', minimum ,'\n','Max:', maximum ,'\n','Mean:', mean ,'\n','Median:', median)
#Apply stats function to Random Forest Pipeline
calculate_stats(accuracy)<jupyter_output>Min: 0.755538579068 
 Max: 1.0 
 Mean: 0.944346829641 
 Median: 0.958441558442<jupyter_text>6. Improve your modelUse grid search to find better parameters.<jupyter_code>#Get pipeline parameters
pipeline.get_params()
parameters = {#'clf__estimator__bootstrap': [True,False],
              #'clf__estimator__criterion': ['gini', 'entropy']
              #'clf__estimator__n_estimators':[1,10,20,30,60],
              'clf__estimator__n_estimators': [10,30]
             }
cv = GridSearchCV(pipeline, param_grid=parameters)
cv.fit(X_train,y_train)
cv.best_estimator_
cv.best_params_<jupyter_output><empty_output><jupyter_text>7. Test your modelShow the accuracy, precision, and recall of the tuned model.  Since this project focuses on code quality, process, and  pipelines, there is no minimum performance metric needed to pass. However, make sure to fine tune your models for accuracy, precision and recall to make your project stand out - especially for your portfolio!<jupyter_code>#Predict on test data with tuned model
y_pred_cv = cv.predict(X_test)

for i in range(Y.shape[1]):
    print('Category:', Y.columns[i], '\n', classification_report(y_test.iloc[:,1].values, y_pred_cv[:,i]))
accuracy_tunned = (y_pred_cv == y_test).mean()
avg_accuracy_tunned = accuracy_tunned.mean()

print("Accuracy Tunned:", accuracy_tunned)

print("Average Accuracy Tunned:", avg_accuracy_tunned)<jupyter_output>Accuracy Tunned: related                   0.814515
request                   0.892743
offer                     0.995416
aid_related               0.771123
medical_help              0.922383
medical_products          0.956914
search_and_rescue         0.973415
security                  0.981054
military                  0.966387
child_alone               1.000000
water                     0.953094
food                      0.940107
shelter                   0.935523
clothing                  0.986555
money                     0.980138
missing_people            0.989610
refugees                  0.970053
death                     0.960886
other_aid                 0.863866
infrastructure_related    0.938732
transport                 0.957219
buildings                 0.949885
electricity               0.980443
tools                     0.993430
hospitals                 0.989458
shops                     0.996028
aid_centers               0.988694
other_infrastructure      0.959206
wea[...]<jupyter_text>8. Try improving your model further. Here are a few ideas:* try other machine learning algorithms* add other features besides the TF-IDF<jupyter_code>#K Nearest Neighbors
from sklearn.neighbors import KNeighborsClassifier

#Pipeline with K Nearest Neighbors estimator
pipeline_knn = Pipeline([
        ('vect', CountVectorizer(tokenizer=tokenize)),
        ('tfidf', TfidfTransformer()),
        ('clf', MultiOutputClassifier(KNeighborsClassifier()))
    ])

#Train KNN pipeline
pipeline_knn.fit(X_train, y_train)

#Predict on test data with KNN classifier
y_pred_knn = pipeline_knn.predict(X_test)

for i in range(Y.shape[1]):
    print('Category:', Y.columns[i], '\n', classification_report(y_test.iloc[:,1].values, y_pred_knn[:,i]))
accuracy_knn = (y_pred_knn == y_test).mean()

avg_accuracy_knn = accuracy_knn.mean()

print("Accuracy KNN:", accuracy)

print("Average Accuracy KNN:", avg_accuracy_knn)
#AdaBoostClassifier

#Pipeline with  AdaBoost Classifier
pipeline_boost = Pipeline([
        ('vect', CountVectorizer(tokenizer=tokenize)),
        ('tfidf', TfidfTransformer()),
        ('clf', MultiOutputClassifier(AdaBoostClassifier()))
    ])

#Train SVC pipeline
pipeline_boost.fit(X_train, y_train)

#Predict on test data with SVC classifier
y_pred_boost = pipeline_boost.predict(X_test)

for i in range(Y.shape[1]):
    print('Category:', Y.columns[i], '\n', classification_report(y_test.iloc[:,1].values, y_pred_boost[:,i]))    
accuracy_boost = (y_pred_boost == y_test).mean()

avg_accuracy_boost = accuracy_boost.mean()

print("Accuracy Boost:", accuracy_boost)

print("Average Accuracy AdaBoost:", avg_accuracy_boost)
#Accuracy for RandomForest Model
calculate_stats(accuracy)
#Accuracy for RandomForest Model Tunned
calculate_stats(accuracy_tunned)
#Accuracty for AdaBoost Model
calculate_stats(accuracy_boost)
#Accuracy for KNN Model
calculate_stats(accuracy_knn)
#We will consider Random Forest Tunned (with 30 estimators) as our final model<jupyter_output><empty_output><jupyter_text>9. Export your model as a pickle file<jupyter_code>pickle.dump(cv, open('model.pkl', "wb"))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10148411371237458, 'fraction_numerical': 0.029786789297658864, 'mean_word_length': 4.128081457663451, 'pattern_counts': {'":': 0, '<': 34, '<?xml version=': 0, '>': 34, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '470642', 'license': 'MIT', 'n_tokens_mistral': 3192, 'n_tokens_neox': 2959, 'n_words': 839, 'path': 'ML Pipeline Preparation.ipynb', 'repo_name': 'MargaridaFernandes1/Udacity-Project2-DisasterResponse', 'size': '9568'}