<jupyter_start><jupyter_text>Opis skupa:<jupyter_code>df_original.sample(10)
df_original.describe()<jupyter_output><empty_output><jupyter_text>Atributi:<jupyter_code>for c in df_original.columns:
    print(c)<jupyter_output>Administrative
Administrative_Duration
Informational
Informational_Duration
ProductRelated
ProductRelated_Duration
BounceRates
ExitRates
PageValues
SpecialDay
Month
OperatingSystems
Browser
Region
TrafficType
VisitorType
Weekend
Revenue<jupyter_text>Vizualizacija raspodele podataka<jupyter_code>pd.DataFrame.hist(df_original, column = df_original.columns[:12], figsize = [15,15])
print(f'Broj instanci: {df_original.shape[0]}\nBroj atributa: {df_original.shape[1]}')<jupyter_output>Broj instanci: 12330
Broj atributa: 18<jupyter_text>Ciljna promenljiva biće Revenue (prihod)<jupyter_code>all_revenue = df_original['Revenue']
revenue = df_original['Revenue'].unique()
n_classes = revenue.shape[0]<jupyter_output><empty_output><jupyter_text>Klase:<jupyter_code>for c in revenue:
    print(c)
print('False: {}\nTrue: {}'.format(np.sum(all_revenue == False), np.sum(all_revenue == True)))
changes = dict( zip(revenue, range(n_classes)))
binary_revenue = all_revenue.replace(changes)
plt.title('Podaci')
plt.xticks([0,1])
plt.hist(binary_revenue)

#plt.savefig('klase.png')
print('Nedostajuce vrednosti u podacima:\n')
print(df_original.isnull().sum())<jupyter_output>Nedostajuce vrednosti u podacima:

Administrative             0
Administrative_Duration    0
Informational              0
Informational_Duration     0
ProductRelated             0
ProductRelated_Duration    0
BounceRates                0
ExitRates                  0
PageValues                 0
SpecialDay                 0
Month                      0
OperatingSystems           0
Browser                    0
Region                     0
TrafficType                0
VisitorType                0
Weekend                    0
Revenue                    0
dtype: int64<jupyter_text>Koje atribute zadržavamo?<jupyter_code>plt.figure(figsize = (20,15))
correlation_matrix = df_original.corr()
sns.heatmap(correlation_matrix, annot = True, cmap = plt.cm.Blues)

#plt.savefig('korelacija.png')<jupyter_output><empty_output><jupyter_text>Možemo izbaciti kolone koje su visoko korelirane ( >= 0.85 )<jupyter_code>df_original = df_original.drop(['ExitRates', 'ProductRelated_Duration'], axis = 1)<jupyter_output><empty_output><jupyter_text>Na osnovu onoga što nam je vratio XGBoost izbacujemo nerelevantne atribute ( eksperimentalno )<jupyter_code># Pod komentarom je jer je korišćeno samo eksperimentalno
#df_original = df_original.drop(['Informational_Duration', 'Weekend', 'Browser', 'SpecialDay'], axis = 1)<jupyter_output><empty_output><jupyter_text>Enkodiranje kategoričkih atributa<jupyter_code>df = pd.get_dummies(df_original, columns = ['OperatingSystems', 'Browser', 'Region', 'TrafficType', 'VisitorType', 'Month', 'Weekend'])

# Ako izbacimo atribute na osnovu XGBoost algoritma, enkodiramo sledeće atribute:
#df = pd.get_dummies(df_original, columns = ['OperatingSystems', 'Region', 'TrafficType', 'VisitorType', 'Month'])
df = df.replace(changes)
print(df.columns)<jupyter_output>Index(['Administrative', 'Administrative_Duration', 'Informational',
       'Informational_Duration', 'ProductRelated', 'BounceRates', 'PageValues',
       'SpecialDay', 'Revenue', 'OperatingSystems_1', 'OperatingSystems_2',
       'OperatingSystems_3', 'OperatingSystems_4', 'OperatingSystems_5',
       'OperatingSystems_6', 'OperatingSystems_7', 'OperatingSystems_8',
       'Browser_1', 'Browser_2', 'Browser_3', 'Browser_4', 'Browser_5',
       'Browser_6', 'Browser_7', 'Browser_8', 'Browser_9', 'Browser_10',
       'Browser_11', 'Browser_12', 'Browser_13', 'Region_1', 'Region_2',
       'Region_3', 'Region_4', 'Region_5', 'Region_6', 'Region_7', 'Region_8',
       'Region_9', 'TrafficType_1', 'TrafficType_2', 'TrafficType_3',
       'TrafficType_4', 'TrafficType_5', 'TrafficType_6', 'TrafficType_7',
       'TrafficType_8', 'TrafficType_9', 'TrafficType_10', 'TrafficType_11',
       'TrafficType_12', 'TrafficType_13', 'TrafficType_14', 'TrafficType_15',
       'TrafficType_16', 'Traff[...]<jupyter_text>Novi izgled skupa:<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>Podela na skup atributa i specijalni atribut klase<jupyter_code>X = df.drop('Revenue', axis = 1)
Y = df['Revenue']
print(f'Atributi koji ce biti korisceni:')
for c in X.columns:
    print(c)<jupyter_output>Atributi koji ce biti korisceni:
Administrative
Administrative_Duration
Informational
Informational_Duration
ProductRelated
BounceRates
PageValues
SpecialDay
OperatingSystems_1
OperatingSystems_2
OperatingSystems_3
OperatingSystems_4
OperatingSystems_5
OperatingSystems_6
OperatingSystems_7
OperatingSystems_8
Browser_1
Browser_2
Browser_3
Browser_4
Browser_5
Browser_6
Browser_7
Browser_8
Browser_9
Browser_10
Browser_11
Browser_12
Browser_13
Region_1
Region_2
Region_3
Region_4
Region_5
Region_6
Region_7
Region_8
Region_9
TrafficType_1
TrafficType_2
TrafficType_3
TrafficType_4
TrafficType_5
TrafficType_6
TrafficType_7
TrafficType_8
TrafficType_9
TrafficType_10
TrafficType_11
TrafficType_12
TrafficType_13
TrafficType_14
TrafficType_15
TrafficType_16
TrafficType_17
TrafficType_18
TrafficType_19
TrafficType_20
VisitorType_New_Visitor
VisitorType_Other
VisitorType_Returning_Visitor
Month_Aug
Month_Dec
Month_Feb
Month_Jul
Month_June
Month_Mar
Month_May
Month_Nov
Month_Oct
Month_Sep
Weekend_Fal[...]<jupyter_text>Podela na trening i test skup i standardizacija podataka<jupyter_code>x_train, x_test, y_train, y_test = train_test_split(X, Y, test_size = 0.3, stratify = Y, random_state = 7)

x_train_before = pd.DataFrame(x_train, columns = X.columns)

scaler = preprocessing.StandardScaler()
scaler.fit(x_train)

x_train_scaled = scaler.transform(x_train)
x_test_scaled = scaler.transform(x_test)

x_train_scaled = pd.DataFrame(x_train_scaled, columns = X.columns)
x_test_scaled = pd.DataFrame(x_test_scaled, columns = X.columns)
fig, (ax1, ax2) = plt.subplots(ncols = 2, figsize = (15, 5))

ax1.set_title('Pre standardizacije')
sns.kdeplot(x_train_before['Administrative'], ax = ax1)
sns.kdeplot(x_train_before['Informational'], ax = ax1)
sns.kdeplot(x_train_before['BounceRates'], ax = ax1)

ax2.set_title('Posle standardizacije')
sns.kdeplot(x_train_scaled['Administrative'], ax = ax2)
sns.kdeplot(x_train_scaled['Informational'], ax = ax2)
sns.kdeplot(x_train_scaled['BounceRates'], ax = ax2)

#plt.savefig('standardizacija.png')<jupyter_output><empty_output><jupyter_text>Balansiranje klasa<jupyter_code>print('Broj instanci u klasi "1" pre balansiranja: {}'.format(sum(y_train == 1)))
print('Broj instanci u klasi "0" pre balansiranja: {}'.format(sum(y_train == 0)))

sm = SMOTE(sampling_strategy = 'minority', random_state = 2)
x_train_balanced, y_train = sm.fit_sample(x_train_scaled, y_train.ravel())

print()
print('Broj instanci u klasi "1" nakon balansiranja: {}'.format(sum(y_train == 1)))
print('Broj instanci u klasi "0" pre balansiranja: {}'.format(sum(y_train == 0)))

x_train_balanced = pd.DataFrame(x_train_balanced, columns = X.columns)<jupyter_output>Broj instanci u klasi "1" pre balansiranja: 1336
Broj instanci u klasi "0" pre balansiranja: 7295

Broj instanci u klasi "1" nakon balansiranja: 7295
Broj instanci u klasi "0" pre balansiranja: 7295<jupyter_text>Skup podataka nakon preprocesiranja ( bez PCA jer on neće biti korišćen svuda )<jupyter_code>X_new = pd.concat([x_train_balanced, x_test_scaled])
df1 = pd.DataFrame(y_train)
df1.columns = ['Revenue']
df2 = pd.DataFrame(y_test)
Y_new = pd.concat([df1, df2])

X_new = X_new.loc[~X_new.index.duplicated(keep = 'first')]
Y_new = Y_new.loc[~Y_new.index.duplicated(keep = 'first')]
final = pd.concat([X_new, Y_new], axis = 1)

#final.to_csv(r'preprocesiran_skup.csv')<jupyter_output><empty_output><jupyter_text>Algoritam XGBoost - korišćen za odabir atributa<jupyter_code>xg_clf = xgb.XGBClassifier()
xg_clf.fit(x_train_balanced, y_train)
xgb.plot_importance(xg_clf, ylabel = 'Atributi', title = 'Značaj atributa', xlabel = 'F mera')

# plt.savefig('atr.png')<jupyter_output><empty_output><jupyter_text>Na osnovu onoga što nam je vratio XGBoost možemo odrediti bitne atribute  Primena algoritma PCA za smanjenje dimenzionalnosti<jupyter_code>pca = PCA(0.95)
pca.fit(x_train_balanced)

x_train_pca = pca.transform(x_train_balanced)
x_test_pca = pca.transform(x_test_scaled)<jupyter_output><empty_output><jupyter_text>Učenje modela  Logistička regresija<jupyter_code>parameters = {
    'C': np.linspace(1, 10, 10),
}

lr = LogisticRegression()
clf = GridSearchCV(lr, parameters, verbose = 8, cv = 5, n_jobs = 3)
clf.fit(x_train_balanced, y_train)

clf.best_params_
lr = LogisticRegression(C = 1.0, penalty = 'l2', solver = 'lbfgs', max_iter = 200)
lr.fit(x_train_balanced, y_train)<jupyter_output><empty_output><jupyter_text>Evaluacija<jupyter_code>def evaluate(model, x_tr, x_tst):
    y_test_predicted = model.predict(x_tst)
    test_score = metrics.accuracy_score(y_test, y_test_predicted)
    y_train_predicted = model.predict(x_tr)
    train_score = metrics.accuracy_score(y_train, y_train_predicted)
    print("Train score: {train}\nTest score: {test}".format(train = train_score, test = test_score))
    
    print()
    print(metrics.classification_report(y_test, y_test_predicted))
    
    return y_test_predicted
def draw_matrix(y_tst_predicted, cmap):
    confusion_matrix = metrics.confusion_matrix(y_test, y_tst_predicted)
    
    fig, ax = plt.subplots()
    img = ax.imshow(confusion_matrix, interpolation = 'nearest', cmap = cmap)
    ax.figure.colorbar(img, ax = ax)

    ax.set(xticks = np.arange(confusion_matrix.shape[1]),
           yticks = np.arange(confusion_matrix.shape[0]),
           xticklabels = ['False', 'True'], yticklabels = ['False', 'True'],
           title = '<NAME>',
           ylabel = 'Prava klasa',
           xlabel = 'Predviđena klasa')

    fmt = '.2f' 
    thresh = confusion_matrix.max() / 2.
    for i in range(confusion_matrix.shape[0]):
        for j in range(confusion_matrix.shape[1]):
            ax.text(j, i, format(confusion_matrix[i, j], fmt),
                    ha = "center", va = "center",
                    color = "white" if confusion_matrix[i, j] > thresh else "black")

    fig.tight_layout()

    #plt.savefig('konf3.png')
y_test_predicted = evaluate(lr, x_train_balanced, x_test_scaled)
draw_matrix(y_test_predicted, plt.cm.Reds)<jupyter_output>Train score: 0.8513365318711447
Test score: 0.8605028386050284

              precision    recall  f1-score   support

           0       0.95      0.89      0.91      3127
           1       0.54      0.72      0.62       572

    accuracy                           0.86      3699
   macro avg       0.74      0.80      0.77      3699
weighted avg       0.88      0.86      0.87      3699<jupyter_text>Stabla odlučivanja<jupyter_code># params = {
#     'max_depth' : [3, 5, 7, 10, 15, 20],
#     'criterion' : ['gini', 'entropy']
# }
# dt = DecisionTreeClassifier()
# grid = GridSearchCV(dt, params, cv = 5, verbose = 8)
# grid.fit(x_train_balanced, y_train)
# grid.best_params_
dt = DecisionTreeClassifier(criterion = 'entropy', max_depth = 7)
dt.fit(x_train_balanced, y_train)
y_test_predicted = evaluate(dt, x_train_balanced, x_test_scaled)
draw_matrix(y_test_predicted, plt.cm.Greens)<jupyter_output>Train score: 0.9229609321453051
Test score: 0.8780751554474182

              precision    recall  f1-score   support

           0       0.94      0.91      0.93      3127
           1       0.59      0.68      0.63       572

    accuracy                           0.88      3699
   macro avg       0.77      0.80      0.78      3699
weighted avg       0.89      0.88      0.88      3699<jupyter_text>SVM  SVM bez kernela<jupyter_code># params = {
#     'C' : range(1, 3, 2),
# }
# lin = svm.LinearSVC()
# grid = GridSearchCV(lin, params, cv = 5, verbose = 8)
# grid.fit(x_train_balanced, y_train)
# grid.best_params_
lin_svm = svm.LinearSVC(loss = 'hinge', C = 1.0)
lin_svm.fit(x_train_balanced, y_train)
y_test_predicted = evaluate(lin_svm, x_train_balanced, x_test_scaled)
draw_matrix(y_test_predicted, plt.cm.Blues)<jupyter_output>Train score: 0.8605209047292666
Test score: 0.8767234387672344

              precision    recall  f1-score   support

           0       0.95      0.90      0.93      3127
           1       0.58      0.72      0.64       572

    accuracy                           0.88      3699
   macro avg       0.76      0.81      0.79      3699
weighted avg       0.89      0.88      0.88      3699<jupyter_text>SVM sa kernelom<jupyter_code># params = {
#     'kernel' : ['rbf', 'poly', 'sigmoid'],
#     'C' : [1.0, 2.0]
# }
# lin_ker = svm.SVC(kernel = 'rbf')
# grid = GridSearchCV(lin_ker, params, cv = 5, verbose = 8)
# grid.fit(x_train_balanced, y_train)
# grid.best_params_
kernelized_svm = svm.SVC(kernel = 'rbf', C = 2.0)
kernelized_svm.fit(x_train_balanced, y_train)
y_test_predicted = evaluate(kernelized_svm, x_train_balanced, x_test_scaled)
draw_matrix(y_test_predicted, plt.cm.Purples)<jupyter_output>Train score: 0.9372172721041809
Test score: 0.8640173019735063

              precision    recall  f1-score   support

           0       0.93      0.91      0.92      3127
           1       0.55      0.64      0.59       572

    accuracy                           0.86      3699
   macro avg       0.74      0.77      0.76      3699
weighted avg       0.87      0.86      0.87      3699<jupyter_text>Random decision forest<jupyter_code># params = {
#     'n_estimators' : range(5, 20, 5),
#     'criterion' : ['gini', 'entropy']
# }
# rfc = RandomForestClassifier()
# grid = GridSearchCV(rfc, params, cv = 5, verbose = 8)
# grid.fit(x_train_balanced, y_train)
# grid.best_params_
rfc = RandomForestClassifier(n_estimators = 15, criterion = 'entropy')
rfc.fit(x_train_balanced, y_train)
y_test_predicted = evaluate(rfc, x_train_balanced, x_test_scaled)
draw_matrix(y_test_predicted, plt.cm.Oranges)<jupyter_output>Train score: 0.9984921178889651
Test score: 0.8842930521762639

              precision    recall  f1-score   support

           0       0.93      0.93      0.93      3127
           1       0.63      0.63      0.63       572

    accuracy                           0.88      3699
   macro avg       0.78      0.78      0.78      3699
weighted avg       0.88      0.88      0.88      3699<jupyter_text>Konačni model<jupyter_code>X = pd.concat([x_train_balanced, x_test_scaled])

y_train = pd.DataFrame(y_train, columns = ['Revenue'])
y_test = pd.DataFrame(y_test, columns = ['Revenue'])
Y = pd.concat([y_train, y_test])
rfc.fit(X, Y.values.ravel())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11190330685136145, 'fraction_numerical': 0.05147008895226455, 'mean_word_length': 3.992542372881356, 'pattern_counts': {'":': 0, '<': 83, '<?xml version=': 0, '>': 85, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 20, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '506513', 'license': 'MIT', 'n_tokens_mistral': 6035, 'n_tokens_neox': 5580, 'n_words': 1037, 'path': 'kodovi.ipynb', 'repo_name': 'popovic-olivera/data-mining-term-paper', 'size': '14738'}