<jupyter_start><jupyter_text>Warm-up(a) In a one-vs-one fashion, for each pairs of classes, train a linear SVM classifier using scikit-learn's function LinearSVC, with the default value for the regularization parameter. Compute the multi-class misclassification error obtained using these classifiers trained in a one-vs-one fashion.<jupyter_code>X_train = pickle.load(open('features_train_all','rb'))
y_train = pickle.load(open('labels_train_all','rb'))
X_train1, X_test1, y_train1, y_test1 = train_test_split(X_train, y_train, test_size=0.2, random_state=42)
LinearSVC_ovo = SVC(C=1.0, kernel='linear', max_iter=1000, decision_function_shape = 'ovo')
LinearSVC_ovo.fit(X_train1, y_train1)
y_lrSVC_ovo = LinearSVC_ovo.predict(X_test1)
accuracy_lrSVC_ovo = accuracy_score(y_test1, y_lrSVC_ovo)
misclassification_error = 1 - accuracy_lrSVC_ovo
print("The multi-class misclassification error obtained using classifiers trained in a one-vs-one fashion is ", + misclassification_error)<jupyter_output>The multi-class misclassification error obtained using classifiers trained in a one-vs-one fashion is  0.327546296296<jupyter_text>(b) In a one-vs-rest fashion, for each class, train a linear SVM classifier using scikit-learn's function LinearSVC, with the default value for $\lambda_c$. Compute the multi-class misclassification error obtained using these classifiers trained in a one-vs-rest fashion.<jupyter_code>linearSVC_ovr = LinearSVC(C=1.0, loss='squared_hinge', penalty='l2',multi_class='ovr')
linearSVC_ovr.fit(X_train1, y_train1)
y_lrSVC_ovr = linearSVC_ovr.predict(X_test1)
accuracy_lrSVC_ovr = accuracy_score(y_test1, y_lrSVC_ovr)
misclassification_error1 = 1 - accuracy_lrSVC_ovr
print("The multi-class misclassification error obtained using classifiers trained in a one-vs-rest fashion is ", + misclassification_error1)<jupyter_output>The multi-class misclassification error obtained using classifiers trained in a one-vs-rest fashion is  0.298611111111<jupyter_text>(c) Using the option multi class='crammer singer' in scikitlearn's function LinearSVC, train a multi-class linear SVM classifier using the default value for the regularization parameter. Compute the multi-class misclassification error obtained using this multi-class linear SVM classifier.<jupyter_code>linearSVC_cs = LinearSVC(C=1.0, loss='squared_hinge', penalty='l2',multi_class='crammer_singer')
linearSVC_cs.fit(X_train1, y_train1)
y_lrSVC_cs = linearSVC_cs.predict(X_test1)
accuracy_lrSVC_cs = accuracy_score(y_test1, y_lrSVC_cs)
misclassification_error2 = 1 - accuracy_lrSVC_cs
print("The multi-class misclassification error obtained using multi-class linear SVM classifier is ", + misclassification_error2)<jupyter_output>The multi-class misclassification error obtained using multi-class linear SVM classifier is  0.295138888889<jupyter_text>Linear SVMs for multi-class classification- Redo all questions above now tuning the regularization parameters using cross-validation.<jupyter_code>X_train_sub = X_train[:500]
y_train_sub = y_train[:500]
#Redo Model one: linearSVC with one-vs-one
ovo_svm = SVC(kernel='linear', max_iter=1000, decision_function_shape = 'ovo')
parameters = {'C':[10**i for i in range(-4, 5)]}
clf_ovo = GridSearchCV(ovo_svm, parameters)
clf_ovo.fit(X_train_sub, y_train_sub)
clf_ovo.best_params_
LinearSVC_ovo_opt = SVC(C=0.1, kernel='linear', max_iter=1000, decision_function_shape = 'ovo')
LinearSVC_ovo_opt.fit(X_train1, y_train1)
y_lrSVC_ovo_opt = LinearSVC_ovo_opt.predict(X_test1)
accuracy_lrSVC_ovo_opt = accuracy_score(y_test1, y_lrSVC_ovo_opt)
misclassification_error_opt = 1 - accuracy_lrSVC_ovo_opt
print("The multi-class misclassification error obtained using classifiers trained in a one-vs-one fashion with lambda=0.1 is ", + misclassification_error_opt)
#Redo model 2: LinearSVC with one-vs-rest
ovr_svm = LinearSVC(loss='squared_hinge', penalty='l2',multi_class='ovr')
parameters = {'C':[10**i for i in range(-4, 5)]}
clf_ovr = GridSearchCV(ovr_svm, parameters)
clf_ovr.fit(X_train_sub, y_train_sub)
clf_ovr.best_params_
linearSVC_ovr_opt = LinearSVC(C=0.01, loss='squared_hinge', penalty='l2',multi_class='ovr')
linearSVC_ovr_opt.fit(X_train1, y_train1)
y_lrSVC_ovr_opt = linearSVC_ovr_opt.predict(X_test1)
accuracy_lrSVC_ovr_opt = accuracy_score(y_test1, y_lrSVC_ovr_opt)
misclassification_error1_opt = 1 - accuracy_lrSVC_ovr_opt
print("The multi-class misclassification error obtained using classifiers trained in a one-vs-rest fashion with lambda=0.01 is ", + misclassification_error1_opt)
#Redo model 3: multi-class linear SVM
cs_svm = LinearSVC(loss='squared_hinge', penalty='l2',multi_class='crammer_singer')
parameters = {'C':[10**i for i in range(-4, 5)]}
clf_cs = GridSearchCV(cs_svm, parameters)
clf_cs.fit(X_train_sub, y_train_sub)
clf_cs.best_params_
linearSVC_cs_opt = LinearSVC(C=0.1, loss='squared_hinge', penalty='l2',multi_class='crammer_singer')
linearSVC_cs_opt.fit(X_train1, y_train1)
y_lrSVC_cs_opt = linearSVC_cs_opt.predict(X_test1)
accuracy_lrSVC_cs_opt = accuracy_score(y_test1, y_lrSVC_cs_opt)
misclassification_error2_opt = 1 - accuracy_lrSVC_cs_opt
print("The multi-class misclassification error obtained using multi-class linear SVM with lambda=0.1 is ", + misclassification_error2_opt)<jupyter_output>The multi-class misclassification error obtained using multi-class linear SVM with lambda=0.1 is  0.293981481481<jupyter_text>Kernel SVMs for multi-class classification- Redo all questions above now using the polynomial kernel of order 2 (and tuning the regularization parameters using cross-validation).<jupyter_code>#Redo Model 1: polynomial kernel SVM of order 2 with one-vs-one
ovo_svm_poly = SVC(kernel='poly', degree=2, max_iter=1000, decision_function_shape = 'ovo')
parameters = {'C':[10**i for i in range(-4, 5)], 'coef0': [0, 1e-1, 1e-2, 1e-3, 1e-4]}
clf_ovo_poly = GridSearchCV(ovo_svm_poly, parameters)
clf_ovo_poly.fit(X_train_sub, y_train_sub)
clf_ovo_poly.best_params_
polySVC_ovo_opt = SVC(C=1000, coef0=0.1, kernel='poly', degree=2, max_iter=1000, decision_function_shape = 'ovo')
polySVC_ovo_opt.fit(X_train1, y_train1)
y_ovo_poly = polySVC_ovo_opt.predict(X_test1)
accuracy_poly_ovo_opt = accuracy_score(y_test1, y_ovo_poly)
misclassification_error_poly1 = 1 - accuracy_poly_ovo_opt
print("The multi-class misclassification error obtained using polynomial kernel SVM in one-vs-one with lambda=1000 is ", + misclassification_error_poly1)
#Redo Model 2: polynomial kernel SVM of order 2 with one-vs-rest
ovr_svm_poly = SVC(kernel='poly', degree=2, max_iter=1000, decision_function_shape = 'ovr')
parameters = {'C':[10**i for i in range(-4, 5)], 'coef0': [0, 1e-1, 1e-2, 1e-3, 1e-4]}
clf_ovr_poly = GridSearchCV(ovo_svm_poly, parameters)
clf_ovr_poly.fit(X_train_sub, y_train_sub)
clf_ovr_poly.best_params_
polySVC_ovr_opt = SVC(C=1000, coef0=0.1, kernel='poly', degree=2, max_iter=1000, decision_function_shape = 'ovr')
polySVC_ovr_opt.fit(X_train1, y_train1)
y_ovr_poly = polySVC_ovr_opt.predict(X_test1)
accuracy_poly_ovr_opt = accuracy_score(y_test1, y_ovr_poly)
misclassification_error_poly2 = 1 - accuracy_poly_ovr_opt
print("The multi-class misclassification error obtained using polynomial kernel SVM in one-vs-rest with lambda=1000 is ", + misclassification_error_poly2)<jupyter_output>The multi-class misclassification error obtained using polynomial kernel SVM in one-vs-rest with lambda=1000 is  0.327546296296
{'fraction_non_alphanumeric': 0.09530890901716912, 'fraction_numerical': 0.0356901446532378, 'mean_word_length': 7.870503597122302, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '10490', 'license': 'MIT', 'n_tokens_mistral': 2829, 'n_tokens_neox': 2602, 'n_words': 604, 'path': 'Data Milestone hw7.ipynb', 'repo_name': 'jingyany/Machine-Learning-Algorithms', 'size': '7397'}