<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "In this exercise you'll apply more advanced encodings to encode the categorical variables ito improve your classifier model. The encodings you will implement are:\n",
    "\n",
    "- Count Encoding\n",
    "- Target Encoding\n",
    "- CatBoost Encoding\n",
    "\n",
    "You'll refit the classifier after each encoding to check its performance on hold-out data. First, run the next cell to repeat the work you did in the last exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn import preprocessing, metrics\n",
    "import lightgbm as lgb\n",
    "\n",
    "# Set up code checking\n",
    "# This can take a few seconds, thanks for your patience\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.feature_engineering.ex2 import *\n",
    "\n",
    "clicks = pd.read_parquet('../input/feature-engineering-data/baseline_data.pqt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here I'll define a couple functions to help test the new encodings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_data_splits(dataframe, valid_fraction=0.1):\n",
    "    \"\"\" Splits a dataframe into train, validation, and test sets. First, orders by \n",
    "        the column 'click_time'. Set the size of the validation and test sets with\n",
    "        the valid_fraction keyword argument.\n",
    "    \"\"\"\n",
    "\n",
    "    dataframe = dataframe.sort_values('click_time')\n",
    "    valid_rows = int(len(dataframe) * valid_fraction)\n",
    "    train = dataframe[:-valid_rows * 2]\n",
    "    # valid size == test size, last two sections of the data\n",
    "    valid = dataframe[-valid_rows * 2:-valid_rows]\n",
    "    test = dataframe[-valid_rows:]\n",
    "    \n",
    "    return train, valid, test\n",
    "\n",
    "def train_model(train, valid, test=None, feature_cols=None):\n",
    "    if feature_cols is None:\n",
    "        feature_cols = train.columns.drop(['click_time', 'attributed_time',\n",
    "                                           'is_attributed'])\n",
    "    dtrain = lgb.Dataset(train[feature_cols], label=train['is_attributed'])\n",
    "    dvalid = lgb.Dataset(valid[feature_cols], label=valid['is_attributed'])\n",
    "    \n",
    "    param = {'num_leaves': 64, 'objective': 'binary', \n",
    "             'metric': 'auc', 'seed': 7}\n",
    "    num_round = 1000\n",
    "    print(\"Training model!\")\n",
    "    bst = lgb.train(param, dtrain, num_round, valid_sets=[dvalid], \n",
    "                    early_stopping_rounds=20, verbose_eval=False)\n",
    "    \n",
    "    valid_pred = bst.predict(valid[feature_cols])\n",
    "    valid_score = metrics.roc_auc_score(valid['is_attributed'], valid_pred)\n",
    "    print(f\"Validation AUC score: {valid_score}\")\n",
    "    \n",
    "    if test is not None: \n",
    "        test_pred = bst.predict(test[feature_cols])\n",
    "        test_score = metrics.roc_auc_score(test['is_attributed'], test_pred)\n",
    "        return bst, valid_score, test_score\n",
    "    else:\n",
    "        return bst, valid_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run this cell to get a baseline score. If your encodings do better than this, you can keep them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Baseline model\")\n",
    "train, valid, test = get_data_splits(clicks)\n",
    "_ = train_model(train, valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Categorical encodings and leakage\n",
    "\n",
    "These encodings are all based on statistics calculated from the dataset like counts and means. Considering this, what data should you be using to calculate the encodings?\n",
    "\n",
    "Run the following line after you've decided your answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check your answer (Run this code cell to receive credit!)\n",
    "q_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) Count encodings\n",
    "\n",
    "Here, encode the categorical features `['ip', 'app', 'device', 'os', 'channel']` using the count of each value in the data set. Using `CountEncoder` from the `category_encoders` library, fit the encoding using the categorical feature columns defined in `cat_features`. Then apply the encodings to the train and validation sets, adding them as new columns with names suffixed `\"_count\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import category_encoders as ce\n",
    "\n",
    "cat_features = ['ip', 'app', 'device', 'os', 'channel']\n",
    "train, valid, test = get_data_splits(clicks)\n",
    "\n",
    "# Create the count encoder\n",
    "count_enc = ____\n",
    "\n",
    "# Learn encoding from the training set\n",
    "____\n",
    "\n",
    "# Apply encoding to the train and validation sets as new columns\n",
    "# Make sure to add `_count` as a suffix to the new columns\n",
    "train_encoded = ____\n",
    "valid_encoded = ____\n",
    "\n",
    "# Check your answer\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment if you need some guidance\n",
    "q_2.hint()\n",
    "q_2.solution()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%%RM_IF(PROD)%%\n",
    "cat_features = ['ip', 'app', 'device', 'os', 'channel']\n",
    "train, valid, test = get_data_splits(clicks)\n",<commit_msg>feat-eng/ex2: comment out hint/solution for step 2<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "In this exercise you'll apply more advanced encodings to encode the categorical variables ito improve your classifier model. The encodings you will implement are:\n",
    "\n",
    "- Count Encoding\n",
    "- Target Encoding\n",
    "- CatBoost Encoding\n",
    "\n",
    "You'll refit the classifier after each encoding to check its performance on hold-out data. First, run the next cell to repeat the work you did in the last exercise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn import preprocessing, metrics\n",
    "import lightgbm as lgb\n",
    "\n",
    "# Set up code checking\n",
    "# This can take a few seconds, thanks for your patience\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.feature_engineering.ex2 import *\n",
    "\n",
    "clicks = pd.read_parquet('../input/feature-engineering-data/baseline_data.pqt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here I'll define a couple functions to help test the new encodings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_data_splits(dataframe, valid_fraction=0.1):\n",
    "    \"\"\" Splits a dataframe into train, validation, and test sets. First, orders by \n",
    "        the column 'click_time'. Set the size of the validation and test sets with\n",
    "        the valid_fraction keyword argument.\n",
    "    \"\"\"\n",
    "\n",
    "    dataframe = dataframe.sort_values('click_time')\n",
    "    valid_rows = int(len(dataframe) * valid_fraction)\n",
    "    train = dataframe[:-valid_rows * 2]\n",
    "    # valid size == test size, last two sections of the data\n",
    "    valid = dataframe[-valid_rows * 2:-valid_rows]\n",
    "    test = dataframe[-valid_rows:]\n",
    "    \n",
    "    return train, valid, test\n",
    "\n",
    "def train_model(train, valid, test=None, feature_cols=None):\n",
    "    if feature_cols is None:\n",
    "        feature_cols = train.columns.drop(['click_time', 'attributed_time',\n",
    "                                           'is_attributed'])\n",
    "    dtrain = lgb.Dataset(train[feature_cols], label=train['is_attributed'])\n",
    "    dvalid = lgb.Dataset(valid[feature_cols], label=valid['is_attributed'])\n",
    "    \n",
    "    param = {'num_leaves': 64, 'objective': 'binary', \n",
    "             'metric': 'auc', 'seed': 7}\n",
    "    num_round = 1000\n",
    "    print(\"Training model!\")\n",
    "    bst = lgb.train(param, dtrain, num_round, valid_sets=[dvalid], \n",
    "                    early_stopping_rounds=20, verbose_eval=False)\n",
    "    \n",
    "    valid_pred = bst.predict(valid[feature_cols])\n",
    "    valid_score = metrics.roc_auc_score(valid['is_attributed'], valid_pred)\n",
    "    print(f\"Validation AUC score: {valid_score}\")\n",
    "    \n",
    "    if test is not None: \n",
    "        test_pred = bst.predict(test[feature_cols])\n",
    "        test_score = metrics.roc_auc_score(test['is_attributed'], test_pred)\n",
    "        return bst, valid_score, test_score\n",
    "    else:\n",
    "        return bst, valid_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run this cell to get a baseline score. If your encodings do better than this, you can keep them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Baseline model\")\n",
    "train, valid, test = get_data_splits(clicks)\n",
    "_ = train_model(train, valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Categorical encodings and leakage\n",
    "\n",
    "These encodings are all based on statistics calculated from the dataset like counts and means. Considering this, what data should you be using to calculate the encodings?\n",
    "\n",
    "Run the following line after you've decided your answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check your answer (Run this code cell to receive credit!)\n",
    "q_1.solution()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) Count encodings\n",
    "\n",
    "Here, encode the categorical features `['ip', 'app', 'device', 'os', 'channel']` using the count of each value in the data set. Using `CountEncoder` from the `category_encoders` library, fit the encoding using the categorical feature columns defined in `cat_features`. Then apply the encodings to the train and validation sets, adding them as new columns with names suffixed `\"_count\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import category_encoders as ce\n",
    "\n",
    "cat_features = ['ip', 'app', 'device', 'os', 'channel']\n",
    "train, valid, test = get_data_splits(clicks)\n",
    "\n",
    "# Create the count encoder\n",
    "count_enc = ____\n",
    "\n",
    "# Learn encoding from the training set\n",
    "____\n",
    "\n",
    "# Apply encoding to the train and validation sets as new columns\n",
    "# Make sure to add `_count` as a suffix to the new columns\n",
    "train_encoded = ____\n",
    "valid_encoded = ____\n",
    "\n",
    "# Check your answer\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment if you need some guidance\n",
    "# q_2.hint()\n",
    "# q_2.solution()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#%%RM_IF(PROD)%%\n",
    "cat_features = ['ip', 'app', 'device', 'os', 'channel']\n",
    "train, valid, test = get_data_splits(clicks)\n",
{'fraction_non_alphanumeric': 0.17294864582496183, 'fraction_numerical': 0.0033753917865466526, 'mean_word_length': 2.710196779964222, 'pattern_counts': {'":': 102, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4253399', 'n_tokens_mistral': 4311, 'n_tokens_neox': 4198, 'n_words': 1103}