<jupyter_start><jupyter_text>Featurized data is loaded Here and splitted into train and test data<jupyter_code>feature_data = pd.read_csv('feature_data').drop(['Unnamed: 0'],axis=1)
#test_data =  pd.read_csv('feature_data_test.csv')
feature_data.shape<jupyter_output><empty_output><jupyter_text>Train Test Splitting<jupyter_code>#for final model
y_train=feature_data.iloc[0:5932188]['reordered']
train_users=feature_data.iloc[0:5932188]['user_id']
x_train=feature_data.iloc[0:5932188].drop(['reordered','user_id','product_id'],axis = 1)
y_test=feature_data.iloc[5932188:]['reordered']
test_users=feature_data.iloc[5932188:]['user_id']
x_test=feature_data.iloc[5932188:].drop(['reordered','user_id','product_id'],axis = 1)
pred_true_data=feature_data.iloc[5932188:][['user_id','product_id','reordered']]
master_train=pd.read_csv('master_train.csv')
pred_true_data=pd.merge(pred_true_data,master_train[['user_id','order_id']].drop_duplicates(),on=['user_id'],how='inner')
pred_true_data['y_true']=pred_true_data['reordered']
pred_true_data.drop(['reordered'],axis=1,inplace=True)
pred_true_data=pd.read_csv('pred_true_data.csv')<jupyter_output><empty_output><jupyter_text>MODELING<jupyter_code>feature_data['reordered'].value_counts().plot(kind='bar')<jupyter_output><empty_output><jupyter_text>Looks like class is imbalanced we will try to take care of it<jupyter_code>feature_data['reordered'].value_counts()<jupyter_output><empty_output><jupyter_text>common functions  The below function measures micro and macro F1 score when probability estimates are given<jupyter_code>def measure_best_F1_score(y_pred_proba):
    y_pred=[]
    for j in range(len(y_pred_proba)):
        if(y_pred_proba[j][1]>0.2):
            y_pred.append(1)
        else:
            y_pred.append(0)
    pred_true_data['y_pred']=y_pred
    temp_test_order_id=pd.unique(pred_true_data['order_id'])
    f_test_products=dict()
    f_pred_products=dict()
    for oid in tqdm(temp_test_order_id):
        temp_data=pred_true_data[pred_true_data['order_id']==oid]
        temp_true=temp_data[temp_data['y_true']==1]['product_id'].tolist()
        if temp_true:
            f_test_products[oid] = temp_true
        else:
            f_test_products[oid]=['None']
        temp_pred=temp_data[temp_data['y_pred']==1]['product_id'].tolist()
        if temp_pred:
            f_pred_products[oid]=temp_pred
        else:
            f_pred_products[oid]=["None"]
    
    f_score=[]
    for key in tqdm(f_test_products):
        f_score.append(multilabel_fscore(f_test_products[key],f_pred_products[key]))
    mean_micro_f1_score=np.mean(f_score)
    for_macro_score=[]
    for i in f_test_products.values():
        for_macro_score.append(len(i))
    score_len=zip(f_score,for_macro_score)
    temp=0
    for score,length in score_len:
        temp+=score*length
    macro_f1_score=temp/sum(for_macro_score)
    return mean_micro_f1_score,macro_f1_score<jupyter_output><empty_output><jupyter_text>The below function Calucates multilabel F1_score which will be user in our previous function credits https://www.kaggle.com/onodera/multilabel-fscore<jupyter_code>def multilabel_fscore(y_true, y_pred):
    """
    ex1:
    y_true = [1, 2, 3]
    y_pred = [2, 3]
    return: 0.8
    
    ex2:
    y_true = ["None"]
    y_pred = [2, "None"]
    return: 0.666
    
    ex3:
    y_true = [4, 5, 6, 7]
    y_pred = [2, 4, 8, 9]
    return: 0.25
    
    """
    y_true, y_pred = set(y_true), set(y_pred)
    
    precision = sum([1 for i in y_pred if i in y_true]) / len(y_pred)
    
    recall = sum([1 for i in y_true if i in y_pred]) / len(y_true)
    
    if precision + recall == 0:
        return 0
    return (2 * precision * recall) / (precision + recall)<jupyter_output><empty_output><jupyter_text>The F1 score maximization technique. Specifically designed for kaggle competion. Not be applicable in the real world  credits https://www.kaggle.com/mmueller/f1-score-expectation-maximization-in-o-n/<jupyter_code># -*- coding: utf-8 -*-
"""
@author: Faron
"""
import numpy as np
from operator import itemgetter

'''
This kernel implements the O(n²) F1-Score expectation maximization algorithm presented in
"<NAME>., <NAME>., <NAME>., and <NAME>.  Optimizing F-measures: A Tale of Two Approaches. In ICML, 2012."

'''


class F1Optimizer():
    def __init__(self):
        pass

    @staticmethod
    def get_expectations(P, pNone=None):
        expectations = []
        
        P = np.sort(P)[::-1]
       
        n = np.array(P).shape[0]
        
        DP_C = np.zeros((n + 2, n + 1))
        if pNone is None:
            pNone = (1.0 - P).prod()

        DP_C[0][0] = 1.0
        for j in range(1, n):
            DP_C[0][j] = (1.0 - P[j - 1]) * DP_C[0, j - 1]
        

        for i in range(1, n + 1):
            DP_C[i, i] = DP_C[i - 1, i - 1] * P[i - 1]
            for j in range(i + 1, n + 1):
                DP_C[i, j] = P[j - 1] * DP_C[i - 1, j - 1] + (1.0 - P[j - 1]) * DP_C[i, j - 1]

        DP_S = np.zeros((2 * n + 1,))
        DP_SNone = np.zeros((2 * n + 1,))
        for i in range(1, 2 * n + 1):
            DP_S[i] = 1. / (1. * i)
            DP_SNone[i] = 1. / (1. * i + 1)
        for k in range(n + 1)[::-1]:
            f1 = 0
            f1None = 0
            for k1 in range(k + 1):
                
                f1 += 2 * k1 * DP_C[k1][k] * DP_S[k + k1]
                f1None += 2 * k1 * DP_C[k1][k] * DP_SNone[k + k1]
            for i in range(1, 2 * k - 1):
                DP_S[i] = (1 - P[k - 1]) * DP_S[i] + P[k - 1] * DP_S[i + 1]
                DP_SNone[i] = (1 - P[k - 1]) * DP_SNone[i] + P[k - 1] * DP_SNone[i + 1]
            expectations.append([f1None + 2 * pNone / (2 + k), f1])
        return np.array(expectations[::-1]).T

    @staticmethod
    def maximize_expectation(P, pNone=None):
        expectations = F1Optimizer.get_expectations(P, pNone)

        ix_max = np.unravel_index(expectations.argmax(), expectations.shape)
        
        max_f1 = expectations[ix_max]

        predNone = True if ix_max[0] == 0 else False
        best_k = ix_max[1]

        return best_k, predNone, max_f1

    

def get_best_prediction(items, preds, pNone=None):
#    print("Maximize F1-Expectation")
#    print("=" * 23)
    items_preds = sorted(list(zip(items, preds)), key=itemgetter(1), reverse=True)
    P = [p for i,p in items_preds]
    L = [i for i,p in items_preds]
    
    opt = F1Optimizer.maximize_expectation(P, pNone)
   
    best_prediction = ['None'] if opt[1] else []
    
    best_prediction += (L[:opt[0]])
   
#    f1_max = opt[2]
    
#    print("Prediction {} yields best E[F1] of {}\n".format(best_prediction, f1_max))
    return ' '.join(list(map(str,best_prediction)))

if __name__ == '__main__':
    get_best_prediction(['a', 'b'], [0.01, 0.1],0.6)<jupyter_output><empty_output><jupyter_text>logistic regresssion with SGD classifier  data is scaled below as logistic regression,linear svm ,naive baye's  work better with normalized or scaled data Data is balance with class_weight='balanced' parameter in SGD classifier<jupyter_code>x_train=scale(x_train,axis=0)
x_test=scale(x_test,axis=0)
alpha = [10 ** x for x in range(-5, 1)] 
f1_scores_logistic=[]
for i in tqdm(alpha):
    clf = SGDClassifier(alpha=i, penalty='l2', loss='log', random_state=42,class_weight='balanced',n_jobs=-1)
    #Here we have taken care of imbalanced dataset.
    clf.fit(x_train, y_train)
    sig_clf = CalibratedClassifierCV(clf, method="sigmoid")
    sig_clf.fit(x_train, y_train)
    predict_y = sig_clf.predict_proba(x_test)
    micro_f1_score,macro_f1_score = measure_best_F1_score(predict_y)
    f1_scores_logistic.append([alpha,micro_f1_score,macro_f1_score])
np.save('f1_scores_logistic.npy',f1_scores_logistic)
f1_scores_logistic=np.load('f1_scores_logistic.npy',allow_pickle=True)
print('for hyper parameter alpha={},'.format(0.001))
print('logistic regression best micro f1 score is {}'.format(np.round(f1_scores_logistic[2][1],4)))
print('logistic regression best macro f1 score is {}'.format(np.round(f1_scores_logistic[2][2],4)))<jupyter_output>for hyper parameter alpha=0.001,
logistic regression best micro f1 score is 0.3714
logistic regression best macro f1 score is 0.438<jupyter_text>SVM with SGD classifier<jupyter_code># Train a Logistic regression+Calibration model using text features whicha re on-hot encoded
alpha = [10 ** x for x in range(-5, 1)]

f1_scores_svm=[]
for i in tqdm(alpha):
    clf = SGDClassifier(alpha=i, penalty='l2', loss='hinge', random_state=42,class_weight='balanced',n_jobs=-1)
    clf.fit(x_train, y_train)
    
    sig_clf = CalibratedClassifierCV(clf, method="sigmoid")
    sig_clf.fit(x_train, y_train)
    predict_y = sig_clf.predict_proba(x_test)
    
    micro_f1_score,macro_f1_score = measure_best_F1_score(predict_y)
    f1_scores_svm.append([alpha,micro_f1_score,macro_f1_score])
np.save('f1_scores_svm.npy',f1_scores_svm)



f1_scores_svm=np.load('f1_scores_svm.npy',allow_pickle=True)
print('for hyper parameter alpha={},'.format(0.00001))
print('linear svm best micro f1 score is {}'.format(np.round(f1_scores_svm[0][1],4)))
print('linear svm best macro f1 score is {}'.format(np.round(f1_scores_svm[0][2],4)))<jupyter_output>for hyper parameter alpha=1e-05,
linear svm best micro f1 score is 0.3674
linear svm best macro f1 score is 0.4452<jupyter_text>Naive_baye's method<jupyter_code>train_users=feature_data.iloc[0:5932188]['user_id']
x_train=feature_data.iloc[0:5932188].drop(['user_id','product_id'],axis = 1)

x_train_majority=x_train[x_train.reordered==0]
x_train_minority=x_train[x_train.reordered==1]

#https://elitedatascience.com/imbalanced-classes
#### Upsample minority class
x_train_minority_upsampled = resample(x_train_minority, 
                                 replace=True,     # sample with replacement
                                 n_samples=5352199,    # to match majority class
                                 random_state=123)
x_train = pd.concat([x_train_majority, x_train_minority_upsampled])

del x_train_majority,x_train_minority,x_train_minority_upsampled,feature_data,master_train

#https://www.geeksforgeeks.org/garbage-collection-python/
collected = gc.collect() 
print("Garbage collector: collected", 
          "%d objects." % collected) 

y_train=x_train['reordered']
x_train.drop(['reordered'],axis=1,inplace=True)

x_train=scale(x_train,axis=0)
x_test=scale(x_test,axis=0)
Parameter_list=[1e-9,1e-4,1e-3,1e-2,1e-1,1e0,1e1,1e2]

f1_scores_naive=[]
for alpha in tqdm(Parameter_list):
    GB = GaussianNB(var_smoothing=alpha)
    GB.fit(x_train, y_train)
    y_predict_proba=GB.predict_proba(x_test)
    micro_f1_score,macro_f1_score = measure_best_F1_score(y_predict_proba)
    f1_scores_naive.append([alpha,micro_f1_score,macro_f1_score])
f1_scores_naive = np.array(f1_scores_naive)
np.save('f1_scores_naive.npy',f1_scores_naive)<jupyter_output><empty_output><jupyter_text>Naive Baye's method with calibration<jupyter_code>Parameter_list=[1]

f1_scores_naive=[]
for alpha in tqdm(Parameter_list):
    GB = GaussianNB(var_smoothing=alpha)
    GB.fit(x_train, y_train)
    cab_clf = CalibratedClassifierCV(GB, method="sigmoid")
    cab_clf.fit(x_train, y_train)
    y_predict_proba=cab_clf.predict_proba(x_test)
    micro_f1_score,macro_f1_score = measure_best_F1_score(y_predict_proba)
    f1_scores_naive.append([alpha,micro_f1_score,macro_f1_score])
f1_scores_naive = np.array(f1_scores_naive)
np.save('f1_scores_naive_calibrated.npy',f1_scores_naive)
f1_scores_naive=np.load('f1_scores_naive.npy',allow_pickle=True)
print('for variable smoothing hyper parameter ={},'.format(1))
print('naive baye\'s best micro f1 score is {}'.format(np.round(f1_scores_naive[5][1],4)))
print('naive baye\'s best macro f1 score is {}'.format(np.round(f1_scores_naive[5][2],4)))
f1_scores_naive<jupyter_output><empty_output><jupyter_text>Random Forest(bagging)<jupyter_code>R_cfl=RandomForestClassifier(n_jobs=-1)

prams={
    
     'n_estimators':[10,50,100,500],
     'max_depth':[3,5,10],
    'min_samples_split':[2,4,8],
    'min_samples_leaf':[1,2,4,6]
}
random_cfl=RandomizedSearchCV(R_cfl,param_distributions=prams,verbose=10,scoring='neg_log_loss',)
random_cfl.fit(x_train,y_train)
parameters={'n_estimators': 50,
 'min_samples_split': 8,
 'min_samples_leaf': 4,
 'max_depth': 10}
random_cfl.best_params_
r_cfl = RandomForestClassifier(random_state=42,n_jobs=-1,n_estimators=parameters['n_estimators'],max_depth=parameters['max_depth'],min_samples_split=parameters['min_samples_split'],min_samples_leaf=parameters['min_samples_leaf'],class_weight='balanced')
r_cfl.fit(x_train,y_train)
cab_clf = CalibratedClassifierCV(r_cfl, method="sigmoid")
cab_clf.fit(x_train, y_train)
predict_y = cab_clf.predict_proba(x_test)

micro_f1_score,macro_f1_score = measure_best_F1_score(predict_y)
print('Random Forest Micro_f1_score is: ',micro_f1_score)
print('Random Forest Macro_f1_score is: ',macro_f1_score)<jupyter_output><empty_output><jupyter_text>XGBoost<jupyter_code>x_cfl=xgb.XGBClassifier(n_jobs=-1,use_label_encoder =False,eval_metric ='logloss')

prams={
    'learning_rate':[0.01,0.03,0.05,0.1],
     'n_estimators':[10,50,100,200],
     'max_depth':[3,5,10],
    'colsample_bytree':[0.1,0.3,0.5,1],
    'subsample':[0.1,0.3,0.5,1]
}
xgb_cfl=RandomizedSearchCV(x_cfl,param_distributions=prams,verbose=10,scoring='neg_log_loss',)
xgb_cfl.fit(x_train.iloc[0:1000000],y_train.iloc[0:1000000])
parameters=xgb_cfl.best_params_
parameters={'subsample': 0.5,
 'n_estimators': 100,
 'max_depth': 5,
 'learning_rate': 0.1,
 'colsample_bytree': 0.1}
x_cfl = xgb.XGBClassifier(random_state=42,use_label_encoder =False,eval_metric ='logloss',n_jobs=-1,learning_rate=parameters['learning_rate'],subsample=parameters['subsample'],n_estimators=parameters['n_estimators'],max_depth=parameters['max_depth'],colsample_bytree=parameters['colsample_bytree'],scale_pos_weight=9)
x_cfl.fit(x_train,y_train)
cab_clf = CalibratedClassifierCV(x_cfl, method="sigmoid")
cab_clf.fit(x_train, y_train)
predict_y = cab_clf.predict_proba(x_test)
micro_f1_score,macro_f1_score = measure_best_F1_score(predict_y)

print('xgboost Micro_f1_score is: ',micro_f1_score)
print('xgboost Macro_f1_score is: ',macro_f1_score)<jupyter_output><empty_output><jupyter_text>light gbm model and hyper tuning<jupyter_code>x_cfl=lightgbm.LGBMClassifier(n_jobs = -1)

prams={
    'learning_rate':[0.01,0.03,0.05,0.1,0.15,0.2],
     'n_estimators':[50,100,200,500,1000,1500],
     'max_depth':[5,10,-1],
    'colsample_bytree':[0.1,0.3,0.5,1],
    'subsample':[0.1,0.3,0.5,1],
    'min_child_samples':[5,10,20,50],
    'min_child_weight':[0.0001,0.001,0.01,0.1],
    'num_leaves':[25,30,60,100,500],
    'subsample_freq':[0,1,2,5],
    'subsample_for_bin':[10,100000,200000,400000]
}
random_cfl1=RandomizedSearchCV(x_cfl,param_distributions=prams,verbose=10,n_jobs=1,scoring='neg_log_loss',)
random_cfl1.fit(x_train,y_train)
parameters=random_cfl1.best_params_
parameters={'subsample_freq': 0,
 'subsample_for_bin': 200000,
 'subsample': 0.1,
 'num_leaves': 25,
 'n_estimators': 1500,
 'min_child_weight': 0.001,
 'min_child_samples': 10,
 'max_depth': 5,
 'learning_rate': 0.05,
 'colsample_bytree': 1}<jupyter_output><empty_output><jupyter_text>training with best hyperparameters<jupyter_code>clf_lgb=lightgbm.LGBMClassifier(boosting_type='gbdt',objective='binary',n_jobs=-1,n_estimators=parameters['n_estimators'],subsample_freq=parameters['subsample_freq'],\
                                subsample_for_bin=parameters['subsample_for_bin'],subsample=parameters['subsample'],min_child_weight=parameters['min_child_weight'],min_child_samples=parameters['min_child_samples'],\
                                max_depth=parameters['max_depth'],learning_rate=parameters['learning_rate'], colsample_bytree=parameters['colsample_bytree'], num_leaves=parameters['num_leaves'],is_unbalance=True,random_state=2)
clf_lgb.fit(x_train,y_train)
cab_cfl=CalibratedClassifierCV(clf_lgb,method='sigmoid')
cab_cfl.fit(x_train,y_train)
y_pred_proba=cab_cfl.predict_proba(x_test)
filename='lgbmmodel.sav'
pickle.dump(cab_cfl,open(filename,'wb'))
#cab_cfl=pickle.load(open(filename,'rb'))
micro_f1_score,macro_f1_score = measure_best_F1_score(y_pred_proba)

print('lightgbm Micro_f1_score is: ',micro_f1_score)
print('lightgbm Macro_f1_score is: ',macro_f1_score)<jupyter_output><empty_output><jupyter_text>Testing Various models  ends here<jupyter_code>clf_lgb.feature_importances_<jupyter_output><empty_output><jupyter_text>plotting feature importance of lightgbm model<jupyter_code>lightgbm.plot_importance(clf_lgb,max_num_features=25,figsize=(12,8))<jupyter_output><empty_output><jupyter_text>F1 Score calculation  Calculating f1 score based on diffferent probability threshold<jupyter_code>y_pred_proba = cab_cfl.predict_proba(x_test)
threshold = [0.1,0.2,0.3,0.4,0.5]
threshold = [0.1,0.2,0.3,0.4,0.5]
y_pred_of_pred=[]
for i in threshold:
    y_pred=[]
    for j in range(len(y_pred_proba)):
        if(y_pred_proba[j][1]>i):
            y_pred.append(1)
        else:
            y_pred.append(0)
    y_pred_of_pred.append(y_pred)<jupyter_output><empty_output><jupyter_text>0.2 found to be the best threshold  Preparing data format  for kaggle competion (this f1 score we will specifically use in kaggle , not applicable to real world ), For real world seperate f1 score is calculated<jupyter_code>pred_true_data=pd.read_csv('pred_true_data.csv')
temp_test_order_id=pd.unique(pred_true_data['order_id'])
f_test_pro=dict()
f_pred_proba=dict()
f_pred_products=dict()
for oid in tqdm(temp_test_order_id):
    
    temp_data=pred_true_data[pred_true_data['order_id']==oid]
    temp_true=temp_data[temp_data['y_true']==1]['product_id'].tolist()
    if temp_true:
        f_test_pro[oid]=[str(i) for i in temp_true]
    else:
        f_test_pro[oid]=['None']
    
    temp_pred_list=temp_data['deep_proba'].tolist()
    products_proba =temp_data['product_id'].tolist()
    f_pred_proba[oid]    = [round(i,4) for i in temp_pred_list]
    f_pred_products[oid] = products_proba
    
pred_list=dict()
for oid in tqdm(temp_test_order_id):
    a=get_best_prediction(f_pred_products[oid],f_pred_proba[oid])
    pred_list[oid]=a
#f_score=[]
#for key in tqdm(f_test_pro):
#    f_score.append(multilabel_fscore(f_test_pro[key],pred_list[key].split(' ')))
f_score=[]
for key in tqdm(f_test_pro):
    f_score.append(multilabel_fscore(f_test_pro[key],pred_list[key].split(' ')))
np.mean(f_score)
micro_f1_score,macro_f1_score = measure_best_F1_score(y_pred_proba)

print('lightgbm Micro_f1_score is: ',micro_f1_score)
print('lightgbm Macro_f1_score is: ',macro_f1_score)<jupyter_output><empty_output><jupyter_text>Kaggle related code ends here  Final score table with different hypertuned models and for best probability threshold of 0.2<jupyter_code>#credits: https://zetcode.com/python/prettytable/
from prettytable import PrettyTable
    
x = PrettyTable()

x.field_names = ["Model Name", "micro_f1_score", "macro_f1_score"]

x.add_row(["Logistic Regression_balanced", 0.3714, 0.4379])
x.add_row(["Linear SVM_class_balanced", 0.3674, 0.4452])
x.add_row(["Naive Baye's class balanced ", 0.3092, 0.3997])
x.add_row(["Naive Baye's class balanced calibrated", 0.21, 0.27])
x.add_row(["Random Forest", 0.37007, 0.4297])
x.add_row(["Random Forest_class_balanced", 0.3781, 0.4578])
x.add_row(["Xgboost", 0.3685, 0.4268])
x.add_row(["Xgboost_class_balanced",0.3779 ,0.4561])
x.add_row(["lightgbm", 0.3707, 0.428])
x.add_row(["lightgbm_class_balanced",0.3796,0.4572])
x.add_row(["average_probability_models",0.3806,0.4579])

print(x)<jupyter_output>+----------------------------------------+----------------+----------------+
|               Model Name               | micro_f1_score | macro_f1_score |
+----------------------------------------+----------------+----------------+
|      Logistic Regression_balanced      |     0.3714     |     0.4379     |
|       Linear SVM_class_balanced        |     0.3674     |     0.4452     |
|      Naive Baye's class balanced       |     0.3092     |     0.3997     |
| Naive Baye's class balanced calibrated |      0.21      |      0.27      |
|             Random Forest              |    0.37007     |     0.4297     |
|      Random Forest_class_balanced      |     0.3781     |     0.4578     |
|                Xgboost                 |     0.3685     |     0.4268     |
|         Xgboost_class_balanced         |     0.3779     |     0.4561     |
|                lightgbm                |     0.3707     |     0.428      |
|        lightgbm_class_balanced         |     0.3796     |     0.4572     |[...]<jupyter_text>Notemicro_f1_score:Here I calculated f1_scores for each user product list and took average across all users.(This will be used in kaggle)macro_f1_score:Here I calculated f1_scores for each user product list and taking weighted average of products with each user.  Procedure After EDA I processed the data . Only 'days_since_prior_order' had null values. Took care of it while featurizing . There were no outliers or missing values .Data was clean with every other aspects. Combined prior data with training data for creating training label.After featurizing found that 'reordered data' is much lower than 'not reordered data'. So there was data imbalance. Took care of this issue while creating models.Tried linear classification techniques like logistic regression , linear svm with hypertuning and found the f1_scores. Data imbalance issue was taken care.Tried non linear classification techniques like naive baye's, random forest,xgboost and lightgbm with hypertuning and found the f1_scores. F1 scores with class balanced and not balanced found out.Data balancing improved the performance random forest , xgboost, lightgbm performed and scores were very close. Logistic regression and linear svm scores were very competitive to these advanced non_linear methods. Naive baye's was the worst performer random forest , xgboost, lightgbm performed and scores were very close. Logistic regression and linear svm scores were very competitive to these advanced non_linear methods. Naive baye's was the worst performer I checked the accuracy of the models for different probability thresholds , o.2 found to be the best threshold  Highest score recorded was for lightgbm with 0.3796 micro_f1 score and 0.4572 macro_f1 score Feature importance graph is plotted for lightgbm , just to check important features Then took average probabilities of 4 top performed models and calculated the scores again . This time micro and macro f1 scores of 0.3806 and 0.4579 was obtained  Final top scores<jupyter_code>final_data_scores=pd.read_csv('final_data_scores.csv')
final_data_scores[['model','micro_f1_score','macro_f1_score','F1_maximization_technique(for kaggle)']]<jupyter_output><empty_output><jupyter_text>Micro and macro f1 scores are applicable to real world scoring . Here I used a special technique called F1_score_maximization technique. This will be used to boost the kaggle score.  Highest F1_score calculated with F1_score_maximization technique is 0.3930, which is my probable kaggle score  Kaggle scorecard screenshot of different models<jupyter_code>#https://stackoverflow.com/questions/32370281/how-to-embed-image-or-picture-in-jupyter-notebook-either-from-a-local-machine-o
from IPython.display import Image
Image("kaggleresult1.PNG")

Image("kaggleresult2.PNG")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13229888005471488, 'fraction_numerical': 0.04163460716423015, 'mean_word_length': 4.5902031063321385, 'pattern_counts': {'":': 0, '<': 90, '<?xml version=': 0, '>': 92, 'https://': 6, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '22', 'dir': 'jupyter-structured-clean-dedup', 'id': '192386', 'license': 'MIT', 'n_tokens_mistral': 9378, 'n_tokens_neox': 8744, 'n_words': 1535, 'path': 'instacart_Models.ipynb', 'repo_name': 'Anand310892/Instacart-market-basket-analysis', 'size': '23396'}