<jupyter_start><jupyter_text>Training The xgboost using cross-validation method<jupyter_code>i=1
means=[]
from sklearn.cross_validation import KFold
kf = KFold(len(Y_prevalid), n_folds=5, shuffle=True)
for training,testing in kf:
    print("iteration "+str(i))
    clf.fit(X_prevalid[training], Y_prevalid[training])
    prediction = clf.predict(X_prevalid[testing]) 
    curmean = np.mean(prediction == Y_prevalid[testing])
    means.append(curmean)
    i+=1
    print(curmean)
print("Mean accuracy: ".format(np.mean(means)))<jupyter_output>iteration 1
0.852552659764
iteration 2
0.846959419255
iteration 3
0.850291562537
iteration 4
0.849101511365
iteration 5
0.849678647941
Mean accuracy:<jupyter_text>plotting the prediction<jupyter_code>prediction_all = clf.predict(X)
plt.figure(figsize=(25,10))
plt.scatter(indexes, prediction_all, color='r', label='prediction', marker="x")
plt.legend()<jupyter_output><empty_output><jupyter_text>plotting the real labels<jupyter_code>plt.figure(figsize=(20,10))
plt.scatter(indexes, Y, color='g', label='real', marker="v")
plt.legend()<jupyter_output><empty_output><jupyter_text>Now try on validation dataset<jupyter_code>clf.fit(X_prevalid, Y_prevalid)
prediction = clf.predict(X_valid) 
curmean = np.mean(prediction == Y_valid)
print(curmean)<jupyter_output>0.85662604722<jupyter_text>confusing matrix<jupyter_code>from sklearn.metrics import confusion_matrix
print(confusion_matrix(Y_valid, prediction))<jupyter_output>[[5044    4  210   70]
 [  12   56   85  244]
 [ 121   10 3141  219]
 [ 131   57  343  757]]<jupyter_text>trying an oversampling method<jupyter_code>from imblearn.over_sampling import RandomOverSampler
ros = RandomOverSampler()
X_prevalid, Y_prevalid = ros.fit_sample(X_prevalid, Y_prevalid)
i=1
means=[]
from sklearn.cross_validation import KFold
kf = KFold(len(Y_prevalid), n_folds=5, shuffle=True)
for training,testing in kf:
    print("iteration "+str(i))
    clf.fit(X_prevalid[training], Y_prevalid[training])
    prediction = clf.predict(X_prevalid[testing]) 
    curmean = np.mean(prediction == Y_prevalid[testing])
    means.append(curmean)
    i+=1
    print(curmean)
print("Mean accuracy: ".format(np.mean(means)))
clf.fit(X_prevalid, Y_prevalid)
prediction = clf.predict(X_valid) 
curmean = np.mean(prediction == Y_valid)
print(curmean)
from sklearn.metrics import confusion_matrix
print(confusion_matrix(Y_valid, prediction))<jupyter_output>[[4988   35  185  120]
 [   8  165   50  174]
 [  99   97 2936  359]
 [ 106  273  220  689]]<jupyter_text>Voting : combining mutiple classifiers<jupyter_code>from sklearn.ensemble import RandomForestClassifier
clf1=RandomForestClassifier(n_estimators=100, n_jobs=-1, criterion='gini')
from sklearn.neighbors import KNeighborsClassifier
clf2 = KNeighborsClassifier(n_neighbors=14)
from sklearn.ensemble import VotingClassifier
eclf1 = VotingClassifier(estimators=[('xgb', clf), ('rf', clf1), ('knn', clf2)], voting='soft', weights=[2,1,1])
eclf1 = eclf1.fit(X_prevalid, Y_prevalid)
prediction=eclf1.predict(X_valid)
curmean = np.mean(prediction == Y_valid)
print(curmean)<jupyter_output>0.797410510282
{'fraction_non_alphanumeric': 0.12251443232841565, 'fraction_numerical': 0.06799230275817832, 'mean_word_length': 7.122395833333333, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '585448', 'license': 'MIT', 'n_tokens_mistral': 1333, 'n_tokens_neox': 1183, 'n_words': 207, 'path': 'notebooks/xgboost.ipynb', 'repo_name': 'yafeunteun/wikipedia-spam-classifier', 'size': '3118'}