<jupyter_start><jupyter_text>支持向量机(Support Vector Machine)  支持向量机对于数据的缩放很敏感   在使用SVM算法前,需要对数据进行缩放 C值控制分类器的容许误差  C越小,容许误差越大,正则化越强,越不容易过度拟合  C越大,容许误差越小,正则化越弱,越容易造成过度拟合 使用LinearSVC进行训练, 速度较快  1.必须设置loss="hinge",因为其并非默认选项  2.当n_samples > n_features,推荐使用dual=False来提升运算效率,其默认选项为True<jupyter_code>import numpy as np
from sklearn import datasets
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import LinearSVC

iris = datasets.load_iris()
X = iris["data"][:, (2, 3)]  # petal length, petal width
y = (iris["target"] == 2).astype(np.float64)  # Iris-Virginica

svm_clf = Pipeline([
        ("scaler", StandardScaler()),
        ("linear_svc", LinearSVC(C=1, loss="hinge", random_state=42)),    #必须设置loss="hinge",因为其并非默认项
    ])

svm_clf.fit(X, y)<jupyter_output><empty_output><jupyter_text>一种替代方案是使用SVC,但运行速度过慢,不推荐<jupyter_code>from sklearn.svm import SVC
SVC(kernel="linear",c=1)<jupyter_output><empty_output><jupyter_text>另一种替代方案是SGDClassifier  该方法虽然没有LinearSVC快,但可以对数据进行在线更新,可以应用于庞大数据集中<jupyter_code>from sklearn.linear_model import SGDClassifier
SGDClassifier(loss="hinge",alpha=1/(m*C))<jupyter_output><empty_output><jupyter_text>非线性SVM分类器(Nonlinear SVM Classification)  添加多项式(高阶)特征<jupyter_code>from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures

polynomial_svm_clf = Pipeline([
        ("poly_features", PolynomialFeatures(degree=3)),
        ("scaler", StandardScaler()),
        ("svm_clf", LinearSVC(C=10, loss="hinge", random_state=42))
    ])

polynomial_svm_clf.fit(X, y)<jupyter_output><empty_output><jupyter_text>先运用PolynomialFeatures建立高阶多项式,然后代入进线性模型进行训练  多项式核(Polynomial Kernel) 取得和添加多项式特征一样的结果  但在实际应用中,不需要添加这些特征,因此避免了特征数量和阶次上升时造成的特征数骤增<jupyter_code>from sklearn.svm import SVC

poly_kernel_svm_clf = Pipeline([
        ("scaler", StandardScaler()),
        ("svm_clf", SVC(kernel="poly", degree=3, coef0=1, C=5))
    ])
poly_kernel_svm_clf.fit(X, y)<jupyter_output><empty_output><jupyter_text>其中,coef0控制模型受到高阶多项式和低阶多项式影响的大小  Gaussian RBF Kernel<jupyter_code>rbf_kernel_svm_clf = Pipeline([
        ("scaler", StandardScaler()),
        ("svm_clf", SVC(kernel="rbf", gamma=5, C=0.001))
    ])
rbf_kernel_svm_clf.fit(X, y)<jupyter_output><empty_output><jupyter_text>gamma($\gamma$) :  gamma越大,决策边界的形状越不规则    gamma越小,决策边界的形状越平滑  因此,gamma可以作为一个正则化的超参数来使用  如果模型过度拟合,则降低gamma; 如果欠拟合,则增加gamma (类似$C$)  SVM Regression SVM同样也支持线性和非线性回归  线性回归<jupyter_code>from sklearn.svm import LinearSVR

svm_reg = LinearSVR(epsilon=1.5, random_state=42)
svm_reg.fit(X, y)<jupyter_output><empty_output><jupyter_text>epsilon($\epsilon$)控制确定回归线的street的宽度  $\epsilon$越大,则宽度越大,越小则宽度越小  非线性回归<jupyter_code>from sklearn.svm import SVR  #SVR支持kernel trick

svm_poly_reg = SVR(kernel="poly", degree=2, C=100, epsilon=0.1)
svm_poly_reg.fit(X, y)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1256140350877193, 'fraction_numerical': 0.012982456140350877, 'mean_word_length': 7.881619937694704, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '249498', 'license': 'Apache-2.0', 'n_tokens_mistral': 1436, 'n_tokens_neox': 1537, 'n_words': 150, 'path': 'SVM.ipynb', 'repo_name': 'CatoneCat/machine_learning', 'size': '2850'}