<jupyter_start><jupyter_text>Prepare datasetThis dataset has nearly 10 million rows and 182 columns. There is not much information on what each column means other than its name. The target variable 'clicks' has only 1869 rows with value of one, and the rest is zero. I chose to randomly sample 100,000 rows out of every 1 million rows as df_zero, and concat with df_clicks for modeling.To prepare dataframe for modeling, first drop all columns with null values. Separate columns into numerical variables (num_var) and categorical variables (cat_var). From preliminary modeling, I noticed both num_var and cat_var have some columns with many unique values and/or has 'id' in column names, indicating they could be unique identifiers. I chose to throw out such columns in cat_var with >100 unique values, and columns in num_var with >100 unique values and has 'id' in its name.<jupyter_code>def prepare_df(df_clicks,df_zero):
    df = pd.concat([df_clicks,df_zero],axis=0,ignore_index=True)
    #remove columns with null
    df = df.dropna(axis=1)
    print df.shape

    #separate numerical and categorical variables
    cols_total = df.columns
    num_var = []
    cat_var = []
    for col in cols_total:
        if df[col].dtype =='object':
            cat_var.append(col)
        else:
            num_var.append(col)
    print len(num_var), len(cat_var)

    #check unique values in cat_var
    for cat in cat_var:
        print cat, len(df[cat].unique())

    #threw away columns with more than 100 unique values
    cat_var2=[]
    for cat in cat_var:
        if len(df[cat].unique())<100:
            cat_var2.append(cat)
    print len(cat_var2),len(cat_var)

    #drop columns not in cat_Var2 and convert the rest with LabelEncoder()
    df_new = df.copy()
    for cat in cat_var:
        if cat in cat_var2:
            le = LabelEncoder()
            unq_vals = df_new[cat].unique()
            le.fit(unq_vals)
            df_new[cat] = le.transform(df_new[cat])
        else:
            df_new.drop(cat,axis=1,inplace=True)

    #drop numerical variables with 'id' in name and >100 unique values
    for col in num_var:
        if 'id' in col and len(df_new[col].unique())>100:
            df_new.drop(col,axis=1,inplace=True)
            print col

    print df_new.shape
    return df_new
#sample 1/10 or 100,000 for the first 1 million rows

num = 500000
temp = pd.read_csv('dataset/dataset.csv',skiprows=500000,\
                   nrows=num, names=df_clicks.columns,low_memory=False)
print temp.shape
df_zero1 = temp[temp['clicks']==0].sample(100000)
print df_zero1.shape
df_new1 = prepare_df(df_clicks,df_zero1)<jupyter_output>(101869, 105)
52 53
Unnamed: 0 101869
transaction_id 1871
city_id 3737
postal_code 1034
geo_fence_recency_id 12
viewability 13
view_conversions_type40 32
date 38
wday 14
tday_id 4
tday 19
exchange 16
exchange_margin 51
campaign 79
model_enabled 4
campaign_status 13
platform 22
platform_id 28
company 33
company_id 36
client 38
client_id 39
advertiser 40
advertiser_id 20
cost_type 2
cost_type_id 2
spend_type 5
recency_config 8
recency_config_id 5
adx_category 2
adx_category_id 2
adx_product 2
adx_product_category_id 2
adx_product_category 2
openx_category 2
sifi_product_category_id 2
sifi_product_category 2
campaign_type 2
spend_vs_perf 2
domain 422
data_provider 47
is_advertiser 177
ad 141
ad_file_type 10
ad_position 4
segment 7
browser 20
os 18
day_spend_flag 13
conversions 7268
keyword_with_data_provider_id 8008
keyword_with_data_provider 1465
dtime 571
42 53
campaign_id
ad_id
domain_id
keyword_id
context_id
exchange_id
segment_id
mobile_app_id
(101869, 86)<jupyter_text>Feature Selection with skfeatureUse skfeature.FCBF for feature selection. This is a fast correlation-based filter solution, suitable for high-dimension data. Split data into 5 fold, run FCBF to select features, model with randomforest and get average of roc_auc_score as metrics. To my surprise, with less than 5 features, the roc_auc_score is already 1.0.Randomly sample another df_zero from row number 1 million to 2 million and run the model again. Got similar features and perfect score.Randomly sample another df_zero from row number 2 million to 3 million and run one of the model to get test score. Perfect score.<jupyter_code>def test_FCBF(df):
    df_X = df.drop('clicks',axis=1)
    X = df_X.values    # data
    y = df['clicks']    # label
    n_samples, n_features = X.shape
    # split data into 5 folds
    ss = KFold(n_samples, n_folds=5, shuffle=True)

    # perform evaluation on classification task
    num_fea = 20    # number of selected features
    clf = RandomForestClassifier(n_estimators=100,n_jobs=-1)

    score = 0.
    for train, test in ss:
        # obtain the index of each feature on the training set
        idx = FCBF.fcbf(X[train], y[train], n_selected_features=num_fea)
        print idx
        # obtain the dataset on the selected features
        features = X[:, idx[0:num_fea]]
        features_names = df_X.columns[idx]

        clf.fit(features[train], y[train])
        #print clf.feature_importances_, cannot use df because dropped clicks
        feature_importance = [format(x,'.2f') for x in clf.feature_importances_]
        zip_feature = zip(features_names, feature_importance)
        print sorted(zip_feature, key = lambda x: x[1], reverse=True)

        # predict the class labels of test data
        y_predict = clf.predict(features[test])

        # obtain the classification accuracy on the test data
        score+= metrics.roc_auc_score(y[test], y_predict)

    # output the average classification accuracy over all 5 folds
    print '\nROC_AUC_Score:', float(score)/5
test_FCBF(df_new1)
#sample 1/10 or 100,000 for row number after 1 million

num = 500000
temp = pd.read_csv('dataset/dataset.csv',skiprows=1000000,\
                   nrows=num, names=df_clicks.columns,low_memory=False)
print temp.shape
df_zero2 = temp[temp['clicks']==0].sample(100000)
print df_zero2.shape
df_new2 = prepare_df(df_clicks,df_zero2)
test_FCBF(df_new2)
#sample 1/10 or 100,000 for row number after 2 million as test set

num = 500000
temp = pd.read_csv('dataset/dataset.csv',skiprows=2000000,\
                   nrows=num, names=df_clicks.columns,low_memory=False)
print temp.shape
df_zero3 = temp[temp['clicks']==0].sample(100000)
print df_zero3.shape
df_new3 = prepare_df(df_clicks,df_zero3)
#pick a simple three-feature model from df_new2
cols_use = ['spend_hours','imps','advertiser_id']
X_train = df_new2[cols_use].values
y_train = df_new2['clicks']
clf = RandomForestClassifier(n_estimators=100,n_jobs=-1)
clf.fit(X_train,y_train)
X_test = df_new3[cols_use].values
y_test = df_new3['clicks']
y_pred = clf.predict(X_test)
print metrics.roc_auc_score(y_test,y_pred)
print len(y_test==y_pred),len(y_test)<jupyter_output>1.0
101869 101869
{'fraction_non_alphanumeric': 0.07660698561784561, 'fraction_numerical': 0.047989433519225126, 'mean_word_length': 3.952761627906977, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '504128', 'license': 'MIT', 'n_tokens_mistral': 2492, 'n_tokens_neox': 2217, 'n_words': 712, 'path': '12GB_feature_selection.ipynb', 'repo_name': 'lingcheng99/random-flies', 'size': '6814'}