<jupyter_start><jupyter_text>Selecting Columns for Prediction   *We eliminate all the columns that are either of the following*    [x]Assigned after other values are measured from readings        [x]Contain ID or name attributes        [x]Are error attributes<jupyter_code>ALL_COLUMNS = df.columns
ERROR_COLUMNS = [col for col in ALL_COLUMNS if "err" in col]
EXCLUDE = ["rowid", "kepid", "kepoi_name","kepler_name","koi_score", "koi_disposition", "koi_pdisposition", "koi_tce_delivname", "koi_tce_plnt_num"] + ERROR_COLUMNS
TO_USE = list(set(ALL_COLUMNS) - set(EXCLUDE))
print(f"Columns being analysed: {len(TO_USE)}")
df[TO_USE].head()<jupyter_output>Columns being analysed: 19<jupyter_text>Extracting Dataframe SubsetWe extract only those rows which are either positive or negative examples i.e. the disposition value is not CANDIDATE.This is because CANDIDATE is the default value for all obects of interest picked up by Kepler. The labels CONFIRMED and FALSE POSITIVE serve as our positive and negative labelsDifference between disposition and pdisposition?We can use either of the two columns as our target variable, since both of them contain the target labels we desire. While pdisposition gives us the most probable labels based on the physical explanation given by the readings, disposition values are based on the NASA Exoplanet Archive Data.<jupyter_code>np.unique(df["koi_disposition"].values)
subset_df = df[df["koi_disposition"] != "CANDIDATE"]
print(subset_df.shape)

subset_df.head()
msno.matrix(subset_df)
# white indicates missing data<jupyter_output><empty_output><jupyter_text>Handling missing Data<jupyter_code># remove columns with more than 40% missing data
def remove_missing(f):
    count = len(df[df[f].isnull()])
    if count/len(df) > 0.4:
        return 1
    else:
        return 0

missing_values = [x for x in subset_df.columns if remove_missing(x)]
subset_df = subset_df.drop(missing_values, axis=1)

subset_df = subset_df.dropna()
msno.matrix(subset_df)<jupyter_output><empty_output><jupyter_text>Obtaining X and y          We can encode the categorical labels to 1 and 0 for easier comparison    CONFIRMED - 1    FALSE POSITIVE - 0<jupyter_code>X = subset_df[TO_USE].values
y = subset_df["koi_disposition"].apply(lambda x: x=='CONFIRMED').astype(int).values
subset_df["koi_disposition"].value_counts()<jupyter_output><empty_output><jupyter_text>Data Visualization<jupyter_code>def getVisualisationPCA(X, y):
    x = StandardScaler().fit_transform(X)
    pca = PCA(n_components=2)
    principal_components = pca.fit_transform(x)
    
    pca_df = pd.DataFrame(
        data = principal_components,
        columns = ['principal component 1', 'principal component 2']
    )
    
    pca_df["TARGET"] = y
    labels = np.unique(y)
    labels = ["CONFIRMED" if i ==1 else "FALSE POSITIVE" for i in labels]
    pca_df["TARGET"] = ["CONFIRMED" if i ==1 else "FALSE POSITIVE" for i in pca_df["TARGET"].values]
    plt.grid()
    colors = cm.rainbow(np.linspace(0, 1, len(labels)))
    for label, color in zip(labels, colors):
        indicesToKeep = pca_df['TARGET'] == label
        plt.scatter(
            pca_df.loc[indicesToKeep, 'principal component 1'],
            pca_df.loc[indicesToKeep, 'principal component 2'],
            c = color,
            label=label
        )
        
    plt.legend()
    plt.grid()
    plt.savefig("pca.png", dpi=600)
    
def getVarianceContribution(X, y):
    cols = X.shape[1]
    x = StandardScaler().fit_transform(X)
    pca = PCA().fit(x)
    variance = pca.explained_variance_ratio_
    s = np.sum(variance)
    p = variance/s
    plt.grid()
    plt.bar(list(range(1, cols+1)), p)
    plt.xlabel('number of components')
    plt.ylabel('cumulative explained variance')
    plt.savefig("variance.png", dpi=600)
    plt.show()
    
    
def getFeatureCorrelation(X, y):
    visualizer = FeatureCorrelation(labels=TO_USE)
    visualizer.fit(X, y)
    #visualizer.show("correlation.png", dpi=600)
    
    
def getPearsonRanking(X):
    #visualizer = rank2d(X, features=TO_USE)
    visualizer = Rank2D(algorithm='pearson', features=TO_USE)
    visualizer.fit(X, y)           
    visualizer.transform(X)        
    #visualizer.show(outpath="pearson_ranking.png", dpi=600)
    

def getRadialViz(X, y):
    visualizer = RadViz(classes=[0, 1], features = TO_USE)
    visualizer.fit(X, y)           
    visualizer.transform(X)        
    #visualizer.show("radial.png", dpi=600)
getPearsonRanking(X)
getVisualisationPCA(X, y)
getVarianceContribution(X, y)
getFeatureCorrelation(X, y)<jupyter_output><empty_output><jupyter_text>* We can see that our columns are not interdependent and we can use all our columns        * We observe that the two classes are not linearly separable, hence a simple Logistic Regression classifier will    not do well.        * Most important, we see there is a class imbalance present, hence we cannot go on accuracy scores and will need to compare scores for prediction across both classes - F1 scores and balanced accuracy scoresWe need to hence using different techniques of non-linear classification, which can handle the non-linearity issue as well as class imbalance. We can employ Bagging or Boosting Techniques such as    * SVM    * Random Forests    * AdaBoost  Splitting into Train and Test sets<jupyter_code>X = StandardScaler().fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=0, stratify=y)
X_train.shape, X_test.shape, y_train.shape, y_test.shape
# List to contain model and accuracy values
eva_list = []
# to contain model names
models =[]<jupyter_output><empty_output><jupyter_text>ML Algorithms  Logistic Regression<jupyter_code>lr = LogisticRegression()
lr.fit(X_train, y_train)
y_pred = lr.predict(X_test)

lr_acc = round(lr.score(X_train, y_train) * 100, 2)
lr_acc_test = round(accuracy_score(y_test, y_pred) * 100, 2)
print(f'Train Accuracy of Logistic Regression: % {lr_acc}')
print(f'Test Accuracy of Logistic Regression: % {lr_acc_test}')

# Get precision, recall, and f1
precision, recall, f1, support = score(y_test, y_pred, average = 'macro')
print(f'Precision : {precision}')
print(f'Recall : {recall}')
print(f'F1-score : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'Logistic Regression'),
    ('Train Accuracy', round(lr_acc, 2)),
    ('Test Accuracy', round(lr_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))<jupyter_output><empty_output><jupyter_text>KNN<jupyter_code>knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(X_train, y_train)
y_pred = knn.predict(X_test)

knn_acc = round(knn.score(X_train, y_train) * 100, 2)
knn_acc_test = round(accuracy_score(y_test, y_pred) * 100, 2)
print(f'Train Accuracy of KNN: % {knn_acc}')
print(f'Test Accuracy of KNN: % {knn_acc_test}')

# Get precision, recall, and f1
precision, recall, f1, support = score(y_test, y_pred, average = 'macro')
print(f'Precision : {precision}')
print(f'Recall : {recall}')
print(f'F1-score : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'KNN'),
    ('Train Accuracy', round(knn_acc, 2)),
    ('Test Accuracy', round(knn_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))<jupyter_output><empty_output><jupyter_text>Naive Bayes<jupyter_code>nbg = GaussianNB()
nbg.fit(X_train, y_train)
y_pred = nbg.predict(X_test)

nbg_acc=round(nbg.score(X_train, y_train) * 100, 2)
nbg_acc_test=round(accuracy_score(y_test, y_pred) * 100, 2)

print(f'Train Accuracy Score of GaussianNB: % {nbg_acc}')
print(f'Test Accuracy Score of GaussianNB: % {nbg_acc_test}')

# Get precision, recall, f1 scores
precision, recall, f1, support = score(y_test, y_pred, average='macro')
print(f'Precision : {precision}')
print(f'Recall    : {recall}')
print(f'F1-score   : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'GaussianNB'),
    ('Train Accuracy', round(nbg_acc, 2)),
    ('Test Accuracy', round(nbg_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))<jupyter_output><empty_output><jupyter_text>Decision Trees<jupyter_code>dt = DecisionTreeClassifier()
dt.fit(X_train, y_train)
y_pred = dt.predict(X_test)

dt_acc = round(dt.score(X_train, y_train) * 100, 2)
dt_acc_test = round(accuracy_score(y_test, y_pred) * 100, 2)
print(f'Train Accuracy Score of Decision Trees: % {dt_acc}')
print(f'Test Accuracy Score of Decision Trees: % {dt_acc_test}')

# Get precision, recall, f1 scores
precision, recall, f1, support = score(y_test, y_pred, average='macro')
print(f'Precision : {precision}')
print(f'Recall    : {recall}')
print(f'F1-score   : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'Decision Trees'),
    ('Train Accuracy', round(dt_acc, 2)),
    ('Test Accuracy', round(dt_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))
# View feature importances
importances_dt = pd.DataFrame({'feature': subset_df[TO_USE].columns,
                             'importance': dt.feature_importances_})
importances_dt = importances_dt.sort_values('importance',ascending=False).set_index('feature')
importances_dt.head(10)<jupyter_output><empty_output><jupyter_text>Random Forest<jupyter_code>num_estimator = 100
seed = 7
rf = RandomForestClassifier(n_estimators=num_estimator)
rf.fit(X_train, y_train)
y_pred = rf.predict(X_test)

rf_acc = round(rf.score(X_train, y_train) * 100, 2)
rf_acc_test = round(accuracy_score(y_test, y_pred) * 100, 2)
print(f'Train Accuracy Score of Random Forest: % {rf_acc}')
print(f'Test Accuracy Score of Random Forest: % {rf_acc_test}')

# Get precision, recall, f1 scores
precision, recall, f1, support = score(y_test, y_pred, average='macro')
print(f'Precision : {precision}')
print(f'Recall    : {recall}')
print(f'F1-score   : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'Random Forest'),
    ('Train Accuracy', round(rf_acc, 2)),
    ('Test Accuracy', round(rf_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))
# View feature importances
importances_rf = pd.DataFrame({'feature': subset_df[TO_USE].columns,
                             'importance': rf.feature_importances_})
importances_rf = importances_rf.sort_values('importance',ascending=False).set_index('feature')
importances_rf.head(10)<jupyter_output><empty_output><jupyter_text>Bagging Classifier<jupyter_code>bag = BaggingClassifier(base_estimator=rf, n_estimators=num_estimator,
    bootstrap=True, n_jobs=-1, random_state=seed)

bag.fit(X_train, y_train)
y_pred = bag.predict(X_test)

bag_acc = round(bag.score(X_train, y_train) * 100, 2)
bag_acc_test = round(accuracy_score(y_test, y_pred) * 100, 2)

print(f'Train Accuracy Score of Bagged Classifier: % {bag_acc}')
print(f'Test Accuracy Score of Bagged Classifier: % {bag_acc_test}')

# Get precision, recall, f1 scores
precision, recall, f1, support = score(y_test, y_pred, average='macro')
print(f'Precision : {precision}')
print(f'Recall    : {recall}')
print(f'F1-score   : {f1}')
# Add performance parameters to list
eva_list.append(dict([
    ('Model', 'Bagging Classifier'),
    ('Train Accuracy', round(bag_acc, 2)),
    ('Test Accuracy', round(bag_acc_test, 2)),
    ('Precision', round(precision, 2)),
    ('Recall', round(recall, 2)),
    ('F1', round(f1, 2))
     ]))<jupyter_output><empty_output><jupyter_text>Results<jupyter_code>results = pd.DataFrame(data=eva_list)
results = results[['Model', 'Train Accuracy', 'Test Accuracy', 'Precision', 'Recall', 'F1']]
results = results.sort_values(by='Test Accuracy', ascending=False)
results = results.set_index('Model')
results<jupyter_output><empty_output><jupyter_text>Confusion Matrix<jupyter_code>conf_mtrx = confusion_matrix(y_test, y_pred)
plt.figure(dpi=125)
sns.heatmap(conf_mtrx, cmap=plt.cm.Reds, annot=True, square=True,fmt='d',
           xticklabels=['Non-Exo', 'Exo'],
           yticklabels=['Non-Exo', 'Exo'])

plt.xlabel('Predicted objects')
plt.ylabel('Actual objects');
# saving model
pickle.dump(rf, open("rf_model.pkl","wb"))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1339538648320518, 'fraction_numerical': 0.01432618373128288, 'mean_word_length': 5.065783014236622, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '263534', 'license': 'MIT', 'n_tokens_mistral': 4621, 'n_tokens_neox': 4341, 'n_words': 1028, 'path': 'part_1_Exoplanet_Prediction/Exoplanet_Prediction.ipynb', 'repo_name': 'prkhrv/Understanding-Exoplanets-with-Data-Science', 'size': '12355'}