<jupyter_start><jupyter_text>Basic model to test extent before advanced models<jupyter_code>train_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 201917 entries, 0 to 201916
Data columns (total 6 columns):
first_active_month    201917 non-null object
card_id               201917 non-null object
feature_1             201917 non-null int64
feature_2             201917 non-null int64
feature_3             201917 non-null int64
target                201917 non-null float64
dtypes: float64(1), int64(3), object(2)
memory usage: 9.2+ MB<jupyter_text>no missing data<jupyter_code>test_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 123623 entries, 0 to 123622
Data columns (total 5 columns):
first_active_month    123622 non-null object
card_id               123623 non-null object
feature_1             123623 non-null int64
feature_2             123623 non-null int64
feature_3             123623 non-null int64
dtypes: int64(3), object(2)
memory usage: 4.7+ MB<jupyter_text>1 data point missing filling NAs<jupyter_code>train_df['first_active_month'].value_counts()
test_df['first_active_month'] = test_df['first_active_month'].fillna('2018-01-01')
train_df.head()<jupyter_output><empty_output><jupyter_text>partition of features<jupyter_code>plt.figure(figsize=(10, 7))
sns.countplot(train_df['feature_1'])
plt.show()
plt.figure(figsize=(10, 7))
sns.countplot(train_df['feature_2'])
plt.show()
plt.figure(figsize=(10, 7))
sns.countplot(train_df['feature_3'])
plt.show()<jupyter_output><empty_output><jupyter_text>Features seem pretty balanced.  First active month<jupyter_code>train_df['first_active_month'] = pd.to_datetime(train_df['first_active_month'])
test_df['first_active_month'] = pd.to_datetime(test_df['first_active_month'])
train_df['first_active_month'].head()
print("Newest client :", max( max(train_df['first_active_month']), max(test_df['first_active_month'])))
first_active_day = pd.to_timedelta(train_df['first_active_month']).apply(lambda x: x.days)
first_active_day_test = pd.to_timedelta(test_df['first_active_month']).apply(lambda x: x.days)
first_active_day.head()
newest = max(max(first_active_day), max(first_active_day_test))
newest
ancienety = (newest - first_active_day).astype(int) // 30
ancienety_test = (newest - first_active_day_test).astype(int) // 30
plt.figure(figsize=(12, 8))
sns.distplot(ancienety)
plt.xlabel('Ancienety (in month)')
plt.show()
train_df['ancienety'] = ancienety
test_df['ancienety'] = ancienety_test<jupyter_output><empty_output><jupyter_text>distribution of the target<jupyter_code>plt.figure(figsize=(10, 7))
sns.distplot(train_df['target'])
plt.show()<jupyter_output><empty_output><jupyter_text>look at that cute outlier on far left<jupyter_code>df_outlier = train_df[train_df['target'] < -30]
print('Number of outliers :', len(df_outlier))
df_outlier.head()
df_inlier = train_df[train_df['target'] > -30]<jupyter_output><empty_output><jupyter_text>Nothing can be said so far, except that they'll play an important role in the future.  Base Model  Linear Regression<jupyter_code>X_train = np.array(train_df[['feature_1', 'feature_2', 'feature_3', 'ancienety']])
X_test = np.array(test_df[['feature_1', 'feature_2', 'feature_3', 'ancienety']])
y_train = np.array(train_df['target'])
lin_reg = LinearRegression()
lin_reg.fit(X_train, y_train)
y_pred = lin_reg.predict(X_train)
print('RMSE on train data :', np.sqrt(mean_squared_error(y_train, y_pred)))
y_test = lin_reg.predict(X_test)<jupyter_output><empty_output><jupyter_text>Linear Regression without Outliers<jupyter_code>X_inlier = np.array(df_inlier[['feature_1', 'feature_2', 'feature_3', 'ancienety']])
y_inlier = np.array(df_inlier['target'])
lin_reg = LinearRegression()
lin_reg.fit(X_inlier, y_inlier)
y_pred = lin_reg.predict(X_inlier)
print('RMSE on inliers :', np.sqrt(mean_squared_error(y_inlier, y_pred)))
y_test = lin_reg.predict(X_test)<jupyter_output><empty_output><jupyter_text>submission<jupyter_code>submission = pd.DataFrame({"card_id": test_df["card_id"].values})
submission["target"] = y_test
submission.to_csv("submission.csv", index=False)<jupyter_output><empty_output><jupyter_text>simple model scored 3.925, which is good  lgbm<jupyter_code>from sklearn.model_selection import train_test_split
import lightgbm
x = df_inlier[['feature_1', 'feature_2', 'feature_3', 'ancienety']]
y = df_inlier[['target']]
x, x_test, y, y_test = train_test_split(x, y, test_size=0.2, random_state=42)
train = df_inlier
categorical_features = [c for c, col in enumerate(train.columns) if 'cat' in col]
train_data = lightgbm.Dataset(x, label=y, categorical_feature=categorical_features)
test_data = lightgbm.Dataset(x_test, label=y_test)
parameters = {
    'application': 'binary',
    'objective': 'binary',
    'metric': 'auc',
    'is_unbalance': 'true',
    'boosting': 'gbdt',
    'num_leaves': 31,
    'feature_fraction': 0.5,
    'bagging_fraction': 0.5,
    'bagging_freq': 20,
    'learning_rate': 0.05,
    'verbose': 0
}
model = lightgbm.train(parameters,
                       train_data,
                       valid_sets=test_data,
                       num_boost_round=5000,
                       early_stopping_rounds=100)<jupyter_output>[1]	valid_0's auc: 0.514554
Training until validation scores don't improve for 100 rounds.
[2]	valid_0's auc: 0.516777
[3]	valid_0's auc: 0.51683
[4]	valid_0's auc: 0.516701
[5]	valid_0's auc: 0.517061
[6]	valid_0's auc: 0.517277
[7]	valid_0's auc: 0.517334
[8]	valid_0's auc: 0.517149
[9]	valid_0's auc: 0.517283
[10]	valid_0's auc: 0.517308
[11]	valid_0's auc: 0.517419
[12]	valid_0's auc: 0.517204
[13]	valid_0's auc: 0.517255
[14]	valid_0's auc: 0.517272
[15]	valid_0's auc: 0.51734
[16]	valid_0's auc: 0.517152
[17]	valid_0's auc: 0.517146
[18]	valid_0's auc: 0.516998
[19]	valid_0's auc: 0.517046
[20]	valid_0's auc: 0.517018
[21]	valid_0's auc: 0.516967
[22]	valid_0's auc: 0.516934
[23]	valid_0's auc: 0.51692
[24]	valid_0's auc: 0.516897
[25]	valid_0's auc: 0.516823
[26]	valid_0's auc: 0.51685
[27]	valid_0's auc: 0.516837
[28]	valid_0's auc: 0.516817
[29]	valid_0's auc: 0.516853
[30]	valid_0's auc: 0.516925
[31]	valid_0's auc: 0.516897
[32]	valid_0's auc: 0.516932
[33]	valid_0's auc: 0.[...]<jupyter_text>create a submission<jupyter_code>X_test = test_df[['feature_1', 'feature_2', 'feature_3', 'ancienety']]
y_test = model.predict(X_test)
submission = pd.DataFrame({"card_id": test_df["card_id"].values})
submission["target"] = y_test
submission.to_csv("submission.csv", index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14078706528370957, 'fraction_numerical': 0.07931665649786455, 'mean_word_length': 6.035407725321888, 'pattern_counts': {'":': 2, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '101879', 'license': 'MIT', 'n_tokens_mistral': 2954, 'n_tokens_neox': 2576, 'n_words': 487, 'path': 'model.py.ipynb', 'repo_name': 'ds-suyog/Elo-Merchant-Category-Recommendation', 'size': '6556'}