<jupyter_start><jupyter_text><jupyter_code>%%capture
!pip install category-encoders
import pandas as pd

# Import what you feel is neccessary from scikit-learn
import category_encoders as ce
from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import plot_confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
from sklearn.linear_model import LogisticRegression
from sklearn.decomposition import PCA
import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>Adult Dataset Classification   Target = "<=50k" [Data Dictionary/Details](https://archive.ics.uci.edu/ml/datasets/Adult)- age: continuous.- workclass: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.- fnlwgt: continuous.- education: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.- education-num: continuous.- marital-status: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.- occupation: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.- relationship: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.- sex: Female, Male.- capital-gain: continuous.- capital-loss: continuous.- hours-per-week: continuous.- native-country: United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.- "50K, <=50K (TARGET)-----RULES-----   Your data has already been split into training and testing sets, do not use `train_test_split` Only use your training set for EDA, Cleaning, Modeling, etc.- **You will have to perform the same transformations on your test set that you apply to your training set. The column names are exactly the same as your training set.** **DO NOT** look into your test set until you are ready to test the accuracy of your model. Your test set is supposed to represent unseen data and lets treat it that way until we are ready to test our accuracy. Your transformations on your training set should work on your test set since it has the exact same column names and the values are exactly the same.<jupyter_code># Starter Code --DO NOT ADJUST--
col_names = ["age", "workclass", "fnlwgt", "education", "education_num", "marital_status", "ooccupation", "relationship", "race", "sex", "capital_gain", "capital_loss", "hours_per_week", "native_country", "target"]
training = pd.read_csv("https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data", names=col_names).drop("race", axis=1)
test = pd.read_csv("https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test", names=col_names, skiprows=1).drop("race", axis=1)

training["target"] = training["target"].apply(lambda x: x.strip())
test["target"] = test["target"].apply(lambda x: x.strip()[:-1])

# Create Train
X_train = training.drop("target", axis=1)
y_train = training["target"]

# Create Test
X_test = test.drop("target", axis=1)
y_test = test["target"]<jupyter_output><empty_output><jupyter_text>**Determine your baseline model accuracy**<jupyter_code># Mean baseline is <=50k with a 75.9% accuracy
training["target"].value_counts(normalize=True)<jupyter_output><empty_output><jupyter_text>**Visualize the data in your training set (EDA)**- Please only create 3 visualization explaining different insights in the data. So you have enough time to model. - You can only use one of each plot type, this means you cant make 3 bar plot, 3 Scatter Plots, etc.<jupyter_code>training["age"].plot(kind="hist", title="Histogram of Age");
training["marital_status"].value_counts().plot(kind="bar", title="Distribution of Marital Status");
training.plot(kind="scatter", y="age", x="capital_gain", title="Relationship between capital_gain and age");<jupyter_output><empty_output><jupyter_text>**Clean and Preprocess your Data** (Hint: One-Hot-Encode Categoricals)
Hint: https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.get_dummies.html  **Create a kNN model**<jupyter_code>pipe = make_pipeline(
    ce.OneHotEncoder(),
    StandardScaler(),
    KNeighborsClassifier(n_jobs=-1)
)

p_grid = {
    "onehotencoder__drop_invariant": [True, False],
    "standardscaler__with_mean": [True, False],
    "standardscaler__with_std": [True, False],
    "kneighborsclassifier__n_neighbors": [11,13,16],
    "kneighborsclassifier__weights": ["uniform", "distance"],
    "kneighborsclassifier__algorithm": ["auto", "ball_tree", "kd_tree"],
    "kneighborsclassifier__metric": ["minkowski", "euclidean"]
}
grid = GridSearchCV(pipe, param_grid=p_grid, cv=10, scoring="roc_auc", n_jobs=4, verbose=2)
grid.fit(X_train, y_train)
# Best Params
grid.best_params_
# Score

accuracy_score(y_test, grid.best_estimator_.predict(X_test))<jupyter_output><empty_output><jupyter_text>**Show us your confusion matrix and explain the results**<jupyter_code>plot_confusion_matrix(grid, X_test, y_test)
print(classification_report(y_test, grid.best_estimator_.predict(X_test)))<jupyter_output>precision    recall  f1-score   support

       <=50K       0.86      0.93      0.89     12435
        >50K       0.69      0.52      0.60      3846

    accuracy                           0.83     16281
   macro avg       0.78      0.73      0.74     16281
weighted avg       0.82      0.83      0.82     16281<jupyter_text>your stuff here  **Compare to Baseline model and explain the results** your stuff here  **BONUS ☝️**- Create a Logistic Regression Model- Compare results to kNN and Baseline and explain the results<jupyter_code>pipe2 = make_pipeline(
    ce.OneHotEncoder(),
    StandardScaler(),
    LogisticRegression()
)

p_grid2 = {
    "onehotencoder__drop_invariant": [True, False],
    "standardscaler__with_mean": [True, False],
    "standardscaler__with_std": [True, False],

}
grid2 = GridSearchCV(pipe2, param_grid=p_grid2, cv=10, scoring="roc_auc", n_jobs=-1, verbose=2)
grid2.fit(X_train, y_train)
accuracy_score(y_test, grid2.best_estimator_.predict(X_test))
plot_confusion_matrix(grid2, X_test, y_test)
print(classification_report(y_test, grid2.best_estimator_.predict(X_test)))<jupyter_output>precision    recall  f1-score   support

       <=50K       0.88      0.93      0.91     12435
        >50K       0.73      0.60      0.66      3846

    accuracy                           0.85     16281
   macro avg       0.81      0.76      0.78     16281
weighted avg       0.85      0.85      0.85     16281
{'fraction_non_alphanumeric': 0.11911086257514329, 'fraction_numerical': 0.026981686005871662, 'mean_word_length': 4.792712550607288, 'pattern_counts': {'":': 10, '<': 32, '<?xml version=': 0, '>': 29, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '301510', 'license': 'MIT', 'n_tokens_mistral': 2438, 'n_tokens_neox': 2277, 'n_words': 613, 'path': 'stack_4/day_10/05_knn_classification_weekend_project.ipynb', 'repo_name': 'mpHarm88/learn_datascience', 'size': '7153'}