<jupyter_start><jupyter_text>Kepler Exoplanet Search Results 10000 exoplanet candidates examined by the Kepler Space Observatory [NASA](https://exoplanetarchive.ipac.caltech.edu/cgi-bin/TblView/nph-tblView?app=ExoTbls&config=cumulative)  Define ML ProblemI used measurements from the Kepler Space Telescope to build a model that can accurately predict whether exoplanets are present around stars in our region of the milky way.This model can help us better understand our local region of the galaxy, as well as provide a window into the possibilities of life in the universe.I will use two models: a Gradient Boosted Tree, as well as a Multinomial Logistic Regression, to classify an observation as 'CONFIRMED', 'CANDIDATE', or 'FALSE POSITIVE'.  Data Dictionary  |column_names|definition||------------|----------||koi_disposition|Exoplanet Archive Disposition||koi_period|Orbital Period [days]||koi_time0bk|Transit Epoch [BKJD]||koi_impact|Impact Parameter||koi_duration|Transit Duration [hrs]||koi_depth|Transit Depth [ppm]||koi_prad|Planetary Radius [Earth radii]||koi_teq|Equilibrium Temperature [K]||koi_insol|Insolation Flux [Earth flux]||koi_model_snr|Transit Signal-to-Noise||koi_tce_plnt_num|TCE Planet Number||koi_tce_delivname|TCE Delivery||koi_steff|Stellar Effective Temperature [K]||koi_slogg|Stellar Surface Gravity [log10(cm/s**2)]||koi_srad|Stellar Radius [Solar radii]||ra|RA [decimal degrees]||dec|Dec [decimal degrees]||koi_kepmag|Kepler-band [mag]|  Imports<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import shap
from category_encoders import OrdinalEncoder, OneHotEncoder
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, plot_confusion_matrix
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.inspection import permutation_importance
from xgboost import XGBClassifier<jupyter_output><empty_output><jupyter_text>Wrangle Feature Selection* Read csv and set the ID as index* Drop columns with greater than 300 null values* Drop high cardinality categorical variables* Drop repeat observations * Drop columns that leak information about the target as well as reduce dimensionality<jupyter_code>def wrangle(filepath):
    # read df
    df = pd.read_csv(filepath).set_index('kepid')
    
    # drop cols with < 3000 null values
    df.dropna(axis=1, thresh=3000, inplace=True)
    
    # drop high cardinality cols 
    cols = [col for col in df.select_dtypes('object').columns
            if df[col].nunique() > 100]
    df.drop(columns=cols, inplace=True)
    
    # drop repeat rows (observations)
    df.drop_duplicates(inplace=True)
    
    # drop cols to prevent leakage and reduce dimensionality
    df.drop(columns=['koi_pdisposition', 'koi_score', 'koi_fpflag_nt', 
                     'koi_fpflag_ss', 'koi_fpflag_co', 'koi_fpflag_ec',
                     'koi_period_err1', 'koi_period_err2', 'koi_time0bk_err1',
                     'koi_time0bk_err2', 'koi_time0bk_err2', 'koi_impact_err1',
                     'koi_impact_err2', 'koi_duration_err1', 'koi_duration_err2',
                     'koi_depth_err1', 'koi_depth_err2', 'koi_prad_err1', 
                     'koi_prad_err2', 'koi_insol_err1', 'koi_insol_err2', 
                     'koi_steff_err1', 'koi_steff_err2', 'koi_slogg_err1', 'koi_slogg_err2',
                     'koi_srad_err1', 'koi_srad_err2'], inplace=True)
    
    return df

FILE_PATH = 'data/kep_observations.csv'

df = wrangle(FILE_PATH)
print(df.shape)
df.head()<jupyter_output>(9564, 18)<jupyter_text>EDA<jupyter_code># plot distribution of target labels
plt.barh(df['koi_disposition'].unique(), df['koi_disposition'].value_counts().sort_values())
plt.title('Distribution of Target Labels')
plt.xlabel('Count')
plt.ylabel('Target Labels');
df.describe()
vars = ['koi_period', 'koi_time0bk', 'koi_impact', 'koi_duration', 'koi_depth', 'koi_prad', 'koi_teq', 'koi_steff', 'koi_srad']
sns.pairplot(df, vars=vars);<jupyter_output><empty_output><jupyter_text>Split<jupyter_code>target = 'koi_disposition'
# target vector
y = df[target]
# feature matrix
X = df.drop(columns=target)

# 80/20 train validation split
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

# reset indices for clasification report
X_train.reset_index(inplace=True)
X_val.reset_index(inplace=True)
# drop old index
X_train = X_train.drop(columns='index')
X_val = X_val.drop(columns='index')<jupyter_output><empty_output><jupyter_text>Baseline<jupyter_code># majority class
print('Baseline Accuracy:', y_train.value_counts(normalize=True).max())<jupyter_output>Baseline Accuracy: 0.5098679911122729<jupyter_text>Build Models  Gradient Boosted Tree* Ordinally Encoded* Imputation Strategy 'mean'* Gradient Boosted Tree    * 90 estimators     * 'max_depth' of 2<jupyter_code>xgb = make_pipeline(
    OrdinalEncoder(),
    SimpleImputer(),
    XGBClassifier(n_estimators=90,
                  max_depth=2,
                  random_state=42, 
                  n_jobs=-1)
)

xgb.fit(X_train, y_train);<jupyter_output>The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].<jupyter_text>Multinomial Logistic Regression* One Hot Encoded * Imputation Strategy 'mean'* Multinomial Logistic Regressionor     * Solver 'sag'<jupyter_code>mlr = make_pipeline(
    OneHotEncoder(),
    SimpleImputer(),
    LogisticRegression(multi_class='multinomial',
                       solver='sag',
                       max_iter=10000,
                       random_state=42)
)

mlr.fit(X_train, y_train);<jupyter_output>is_categorical is deprecated and will be removed in a future version.  Use is_categorical_dtype instead<jupyter_text>Communicate Results   Gradient Boosted Tree Accuracy<jupyter_code>print('XGB Training Accuracy:', xgb.score(X_train, y_train))
print('XGB Validation Accuracy:', xgb.score(X_val, y_val))<jupyter_output>XGB Training Accuracy: 0.8062998300875702
XGB Validation Accuracy: 0.7600627286983795<jupyter_text>Multinomial Logestic Regression Accuracy<jupyter_code>print('MLR Training Accuracy:', mlr.score(X_train, y_train))
print('MLR Validation Accuracy:', mlr.score(X_val, y_val))<jupyter_output>MLR Training Accuracy: 0.5490785518232911
MLR Validation Accuracy: 0.5269210663878725<jupyter_text>Confusion Matrix<jupyter_code>plot_confusion_matrix(xgb, X_val, y_val, normalize='true')
plt.title('Kepler XGB Tree [Normalized]');<jupyter_output><empty_output><jupyter_text>Classification Report<jupyter_code>print('-------------------------------------------------------')
print('Gradient Boosted Tree'.upper())
print('-------------------------------------------------------')
print(classification_report(y_train, xgb.predict(X_train)))
print()
print('-------------------------------------------------------')
print('Multinomial Logestic Regression'.upper())
print('-------------------------------------------------------')
print(classification_report(y_train, mlr.predict(X_train)))<jupyter_output>-------------------------------------------------------
GRADIENT BOOSTED TREE
-------------------------------------------------------
                precision    recall  f1-score   support

     CANDIDATE       0.70      0.60      0.64      1882
     CONFIRMED       0.77      0.88      0.82      1868
FALSE POSITIVE       0.87      0.87      0.87      3901

      accuracy                           0.81      7651
     macro avg       0.78      0.78      0.78      7651
  weighted avg       0.80      0.81      0.80      7651


-------------------------------------------------------
MULTINOMIAL LOGESTIC REGRESSION
-------------------------------------------------------
                precision    recall  f1-score   support

     CANDIDATE       1.00      0.00      0.00      1882
     CONFIRMED       0.41      0.38      0.39      1868
FALSE POSITIVE       0.59      0.90      0.71      3901

      accuracy                           0.55      7651
     macro avg       0.67      0.42      0.3[...]
{'fraction_non_alphanumeric': 0.1460875807609476, 'fraction_numerical': 0.035774108638430245, 'mean_word_length': 3.1795, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 39, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '443155', 'license': 'MIT', 'n_tokens_mistral': 2944, 'n_tokens_neox': 2661, 'n_words': 616, 'path': 'notebooks/kep_exo_search_results.ipynb', 'repo_name': 'illicitDev/kepler-observations', 'size': '8358'}