<jupyter_start><jupyter_text>Space X Falcon 9 First Stage Landing Prediction  Machine Learning Prediction   PrologueSpace X advertises Falcon 9 rocket launches on its website with a cost of 62 million dollars; other providers cost upward of 165 million dollars each, much of the savings is because Space X can reuse the first stage. Therefore if we can determine if the first stage will land, we can determine the cost of a launch. This information can be used if an alternate company wants to bid against space X for a rocket launch. In this lab, you will create a machine learning pipeline to predict if the first stage will land given the data from the preceding labs.  Objectives1. Perform EDA and Determine Training Labels2. Find best Hyperparameter for SVM, Classification Trees and Logistic Regression  Import Libraries and Define Auxiliary Functions<jupyter_code>import pandas as pd  #data-manipulation
import numpy as np #multi-dimensional
import matplotlib.pyplot as plt #plotting-function
import seaborn as sns #advanced-visualization
from sklearn import preprocessing #standarsize-data
from sklearn.model_selection import train_test_split #data-split
from sklearn.model_selection import GridSearchCV #test-parameters-classification-algorithms
from sklearn.linear_model import LogisticRegression #log-reg-model
from sklearn.svm import SVC #svm-model
from sklearn.tree import DecisionTreeClassifier #deci-tree
from sklearn.neighbors import KNeighborsClassifier #knn<jupyter_output><empty_output><jupyter_text>Function to plot the confusion matrix<jupyter_code>def plot_confusion_matrix(y,y_predict):
    from sklearn.metrics import confusion_matrix
    cm = confusion_matrix(y,y_predict)
    ax = plt.subplot()
    sns.heatmap(cm, annot=True, ax=ax);
    ax.set_xlabel('Predicted labels')
    ax.set_ylabel('True labels')
    ax.set_title('Confusion Matrix');
    ax.xaxis.set_ticklabels(['did not land', 'land']); 
    ax.yaxis.set_ticklabels(['did not land', 'landed'])<jupyter_output><empty_output><jupyter_text>Load Data<jupyter_code>data = pd.read_csv("Data/dataset_part2.csv")
data.head()
X = pd.read_csv("Data/dataset_part3.csv")
X.head(100)<jupyter_output><empty_output><jupyter_text>Create a NumPy array from the column Class in data, by applying the method to_numpy() then assign it to the variable Y<jupyter_code>y = data['Class'].to_numpy()<jupyter_output><empty_output><jupyter_text>Standardize the data in X<jupyter_code>transform = preprocessing.StandardScaler()
X = transform.fit(X).transform(X)<jupyter_output><empty_output><jupyter_text>Splitting the data into training and test set<jupyter_code>X_train, X_test, Y_train, Y_test = train_test_split(X, y, test_size=0.2, random_state=2)
Y_test.shape<jupyter_output><empty_output><jupyter_text>Logistic Regression- Find the best parameters<jupyter_code>parameters = {'C':[0.01,0.1,1], 'penalty':['l2'], 'solver':['lbfgs']}
lr = LogisticRegression()
gs_cv = GridSearchCV(lr, parameters, scoring='accuracy', cv=10)
logreg_cv = gs_cv.fit(X_train, Y_train)
print("Tuned hyperparameters :(best parameters)", logreg_cv.best_params_)
print("Accuracy for logistic regression:", logreg_cv.best_score_)<jupyter_output>Tuned hyperparameters :(best parameters) {'C': 0.1, 'penalty': 'l2', 'solver': 'lbfgs'}
Accuracy for logistic regression: 0.8214285714285714<jupyter_text>Calculate the accuracy on the test data.<jupyter_code>print('Accuracy for logistic regression:', logreg_cv.score(X_test, Y_test))<jupyter_output>Accuracy for logistic regression: 0.8333333333333334<jupyter_text>Confusion Matrix<jupyter_code>yhat = logreg_cv.predict(X_test)
plot_confusion_matrix(Y_test, yhat)<jupyter_output><empty_output><jupyter_text>Examining the confusion matrix, we see that logistic regression can distinguish between the different classes. We see that the major problem is false positives.  Support Vector Machine<jupyter_code>parameters = {'kernel':('linear','rbf','poly','sigmoid'), 'C': np.logspace(-3, 3, 5), 'gamma': np.logspace(-3, 3, 5)}
svm = SVC()
gs_cv = GridSearchCV(svm, parameters, scoring='accuracy', cv=10)
svm_cv = gs_cv.fit(X_train, Y_train)
print("Tuned hyperparameters (best parameters):", svm_cv.best_params_)
print("Accuracy for support vector machine:", svm_cv.best_score_)<jupyter_output>Tuned hyperparameters (best parameters): {'C': 1.0, 'gamma': 0.03162277660168379, 'kernel': 'sigmoid'}
Accuracy for support vector machine: 0.85<jupyter_text>Calculate the accuracy on the test<jupyter_code>print("Accuracy for support vector machine on the test data using the method score:", svm_cv.score(X_test, Y_test))<jupyter_output>Accuracy for support vector machine on the test data using the method score: 0.8333333333333334<jupyter_text>Confusion Matrix<jupyter_code>yhat = svm_cv.predict(X_test)
plot_confusion_matrix(Y_test, yhat)<jupyter_output><empty_output><jupyter_text>Decision Tree<jupyter_code>parameters = {'criterion': ['gini', 'entropy'],
     'splitter': ['best', 'random'],
     'max_depth': [2*n for n in range(1,10)],
     'max_features': ['auto', 'sqrt'],
     'min_samples_leaf': [1, 2, 4],
     'min_samples_split': [2, 5, 10]}

tree = DecisionTreeClassifier()
gs_cv = GridSearchCV(tree,parameters, scoring='accuracy', cv=10)
tree_cv = gs_cv.fit(X_train, Y_train)
print("Tuned hpyerparameters :(best parameters) ", tree_cv.best_params_)
print("Accuracy for decision tree classifier:", tree_cv.best_score_)<jupyter_output>Tuned hpyerparameters :(best parameters)  {'criterion': 'gini', 'max_depth': 6, 'max_features': 'auto', 'min_samples_leaf': 1, 'min_samples_split': 5, 'splitter': 'best'}
Accuracy for decision tree classifier: 0.8642857142857142<jupyter_text>Accuracy<jupyter_code>print("Accuracy for decision tree classifier on the test data using the method score:", tree_cv.score(X_test, Y_test))<jupyter_output>Accuracy for decision tree classifier on the test data using the method score: 0.3888888888888889<jupyter_text>Confusion Matrix<jupyter_code>yhat = svm_cv.predict(X_test)
plot_confusion_matrix(Y_test, yhat)<jupyter_output><empty_output><jupyter_text>kNN Model<jupyter_code>parameters = {'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
              'algorithm': ['auto', 'ball_tree', 'kd_tree', 'brute'],
              'p': [1,2]}

KNN = KNeighborsClassifier()
gs_cv = GridSearchCV(KNN, parameters, scoring='accuracy', cv=10)
knn_cv = gs_cv.fit(X_train, Y_train)
print("Tuned hpyerparameters :(best parameters) ",knn_cv.best_params_)
print("Accuracy for k nearest neighbors:",knn_cv.best_score_)<jupyter_output>Tuned hpyerparameters :(best parameters)  {'algorithm': 'auto', 'n_neighbors': 6, 'p': 1}
Accuracy for k nearest neighbors: 0.8607142857142855<jupyter_text>Accuracy<jupyter_code>print("Accuracy for k nearest neighbors on the test data using the method score: ",knn_cv.score(X_test, Y_test))<jupyter_output>Accuracy for k nearest neighbors on the test data using the method score:  0.8333333333333334<jupyter_text>Confusion Matrix<jupyter_code>yhat = knn_cv.predict(X_test)
plot_confusion_matrix(Y_test,yhat)<jupyter_output><empty_output><jupyter_text>The Best Model<jupyter_code>algorithms = {'KNN':knn_cv.best_score_,'Decision Tree':tree_cv.best_score_,'Logistic Regression':logreg_cv.best_score_,'SVM':svm_cv.best_score_}
best_algorithm = max(algorithms, key= lambda x: algorithms[x])

print('The method which performs best is \"',best_algorithm,'\" with a score of',algorithms[best_algorithm])
algo_df = pd.DataFrame.from_dict(algorithms, orient='index', columns=['Accuracy'])
algo_df.head()
algo_df.sort_values(['Accuracy'], inplace=True)
algo_df.head()
algo_df.reset_index(inplace=True)
algo_df.head()
algo_df.rename(columns = {'index': 'Algorithm'}, inplace = True)
algo_df.head()
import plotly.express as px
import plotly.graph_objects as go
fig = px.bar(algo_df, x='Algorithm', y='Accuracy', hover_data=['Algorithm', 'Accuracy'], color='Accuracy', color_continuous_scale='rdylbu')
fig.update_layout(title='Algorithm vs. Accuracy', xaxis_title='Algorithm', yaxis_title='Accuracy' )
fig.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.115523017744137, 'fraction_numerical': 0.0263059932994168, 'mean_word_length': 7.666666666666667, 'pattern_counts': {'":': 0, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 4, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '587828', 'license': 'MIT', 'n_tokens_mistral': 2913, 'n_tokens_neox': 2698, 'n_words': 679, 'path': 'spacex-machine-learning.ipynb', 'repo_name': 'sinanthahir/Spacex_launch_prediction', 'size': '8059'}