<jupyter_start><jupyter_text>Simple Binary Classification with defaultsIn this notebook we will use the Adult Census dataset. Download the data from [here](https://www.kaggle.com/wenruliu/adult-income-dataset/downloads/adult.csv/2).<jupyter_code>import numpy as np
import pandas as pd
import torch

from pytorch_widedeep.preprocessing import WidePreprocessor, DensePreprocessor
from pytorch_widedeep.models import Wide, DeepDense, WideDeep
from pytorch_widedeep.metrics import Accuracy, Precision
df = pd.read_csv('data/adult/adult.csv.zip')
df.head()
# For convenience, we'll replace '-' with '_'
df.columns = [c.replace("-", "_") for c in df.columns]
# binary target
df['income_label'] = (df["income"].apply(lambda x: ">50K" in x)).astype(int)
df.drop('income', axis=1, inplace=True)
df.head()<jupyter_output><empty_output><jupyter_text>Preparing the dataHave a look to notebooks one and two if you want to get a good understanding of the next few lines of code (although there is no need to use the package)<jupyter_code>wide_cols = ['education', 'relationship','workclass','occupation','native_country','gender']
crossed_cols = [('education', 'occupation'), ('native_country', 'occupation')]
cat_embed_cols = [('education',16), ('relationship',8), ('workclass',16), ('occupation',16),('native_country',16)]
continuous_cols = ["age","hours_per_week"]
target_col = 'income_label'
# TARGET
target = df[target_col].values

# WIDE
preprocess_wide = WidePreprocessor(wide_cols=wide_cols, crossed_cols=crossed_cols)
X_wide = preprocess_wide.fit_transform(df)

# DEEP
preprocess_deep = DensePreprocessor(embed_cols=cat_embed_cols, continuous_cols=continuous_cols)
X_deep = preprocess_deep.fit_transform(df)
print(X_wide)
print(X_wide.shape)
print(X_deep)
print(X_deep.shape)<jupyter_output>[[ 0.          0.          0.         ...  0.         -0.99512893
  -0.03408696]
 [ 1.          1.          0.         ...  0.         -0.04694151
   0.77292975]
 [ 2.          1.          1.         ...  0.         -0.77631645
  -0.03408696]
 ...
 [ 1.          3.          0.         ...  0.          1.41180837
  -0.03408696]
 [ 1.          0.          0.         ...  0.         -1.21394141
  -1.64812038]
 [ 1.          4.          6.         ...  0.          0.97418341
  -0.03408696]]
(48842, 7)<jupyter_text>Defining the model<jupyter_code>wide = Wide(wide_dim=X_wide.shape[1], pred_dim=1)
deepdense = DeepDense(hidden_layers=[64,32], 
                      deep_column_idx=preprocess_deep.deep_column_idx,
                      embed_input=preprocess_deep.embeddings_input,
                      continuous_cols=continuous_cols)
model = WideDeep(wide=wide, deepdense=deepdense)
model<jupyter_output><empty_output><jupyter_text>As you can see, the model is not particularly complex. In mathematical terms (Eq 3 in the [original paper](https://arxiv.org/pdf/1606.07792.pdf)): $$pred = \sigma(W^{T}_{wide}[x, \phi(x)] + W^{T}_{deep}a_{deep}^{(l_f)} +  b) $$ The architecture above will output the 1st and the second term in the parenthesis. `WideDeep` will then add them and apply an activation function (`sigmoid` in this case). For more details, please refer to the paper.  Compiling and Running/FittingOnce the model is built, we just need to compile it and run it<jupyter_code>model.compile(method='binary', metrics=[Accuracy, Precision])
model.fit(X_wide=X_wide, X_deep=X_deep, target=target, n_epochs=5, batch_size=256, val_split=0.2)<jupyter_output>0%|          | 0/153 [00:00<?, ?it/s]
{'fraction_non_alphanumeric': 0.13142037302725967, 'fraction_numerical': 0.052510760401721666, 'mean_word_length': 3.6294820717131473, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '75520', 'license': 'MIT', 'n_tokens_mistral': 1305, 'n_tokens_neox': 1172, 'n_words': 275, 'path': 'examples/03_Binary_Classification_with_Defaults.ipynb', 'repo_name': 'yuanzhiKe/pytorch-widedeep', 'size': '3485'}