<jupyter_start><jupyter_text>SVR Concept Support Vectore Regression is a version of Support Vector Machine for regression.we can use SVR for working with continuous Values instead of Classification which is SVM  [More about SVR](https://medium.com/coinmonks/support-vector-regression-or-svr-8eb3acf6d0ff) [SVR explained](http://www.saedsayad.com/support_vector_machine_reg.htm)   TERMS Two Dashed lines are called Boundary Line    Line in between is called Hyperplane    Data points which are closest to the Boundary Line are called Support Vectors    Distances between Boundary lines and Hyperplane are "+e" and "-e"(negative and positive epsilon)  Concept     SVR is about deciding a boundary at ‘e’ distance from the original hyper plane such that data points closest to the hyper plane or the support vectors are within that boundary line    Decision boundary is Margin of tolerance and only data points which are within boundary are taken in consideration.    To make explanation simpler, that we are going to take only those data points which have least error rate. Thus giving us a better fitting model.  SVR in python Let's import all the libraries first!<jupyter_code>import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
from sklearn.svm import SVR #Our today's topic<jupyter_output><empty_output><jupyter_text>Let's get the Data!<jupyter_code>dataset = pd.read_csv('50_Startups.csv') #Loading the data into DataFrame
dataset.head() #Displaying the data to make sure DataFrame is alright
###  dataframe.iloc requires 1)ROW(s) and 2)Column(s)
x = dataset.iloc[:, 2:3].values #Put a colon to choose all rows. 
y = dataset.iloc[:, 4:].values #Profit column is at 4 index(start counting from 0 like a programmer!)<jupyter_output><empty_output><jupyter_text>Let's create the model and Fit the data<jupyter_code>svr_regressor_model = SVR().fit(x, y)<jupyter_output>C:\Users\PC\AppData\Local\Enthought\Canopy\edm\envs\User\lib\site-packages\sklearn\utils\validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>Let's visualize the values and model<jupyter_code>plt.scatter(x, y, c='r', label="Values")
plt.plot(x, svr_regressor_model.predict(x), c='b', label="Support Vector Regression")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>The model does not look so effective as we expected.    WHY?    Maybe because we forgot to do feature scaling. Let's try  Feature Scaling is a method used to standartize data and it is also referred as "Normalization"<jupyter_code>from sklearn.preprocessing import StandardScaler
scalerx = StandardScaler().fit(x)
scalery = StandardScaler().fit(y)
X = scalerx.transform(x)
Y = scalery.transform(y)<jupyter_output><empty_output><jupyter_text>Fitting data to the new model<jupyter_code>#Let's name this model a bit differently to differenciate with previous model
svr_regressor_rbf = SVR().fit(X, Y)<jupyter_output>C:\Users\PC\AppData\Local\Enthought\Canopy\edm\envs\User\lib\site-packages\sklearn\utils\validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>Visualizing<jupyter_code>X_grid = np.arange(min(X), max(X), 0.1)
X_grid = X_grid.reshape((len(X_grid), 1))
plt.scatter(X,Y, c='b', label="Values")
plt.plot(X_grid, svr_regressor_rbf.predict(X_grid), c='r', label="SVR",linewidth=3)
plt.title("SVR with RBF(default) kernel")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>This model looks much better than previous. Which means we need to scale the data before fitting it into the SVR     model SVR model has many optional parameters, but the most important one is Kernel    There are 3 kernels for SVR models: Linear, Polynomial and RBF(default) which we used above SVR model with linear kernel  Fitting data to the SVR model with Linear kernel<jupyter_code>#Let's name this model a bit differently to differenciate with previous model
svr_regressor_linear = SVR(kernel='linear').fit(X, Y)<jupyter_output>C:\Users\PC\AppData\Local\Enthought\Canopy\edm\envs\User\lib\site-packages\sklearn\utils\validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>Visualizing<jupyter_code>plt.scatter(X, Y, c='r', label="Values")
plt.plot(X, svr_regressor_linear.predict(X), c='b', label="SVR")
plt.title("SVR with linear kernel")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>This model with Linear kernel is worse than previous model with RBF kernel    To prove it, I will use Score function of SVR model    Formula of computing the score is following $$ Score= 1- \frac { \sum\limits_{k = 1}^n {(y_{obs_k}-y_{pri_k})^2} } {\sum\limits_{k = 1}^n {(y_{obs_k}-y_{obs_ave})^2 }}$$     Let's compute scores of each model<jupyter_code>svr_regressor_rbf.score(X,Y) #rbf kernel with scaling(rbf)
svr_regressor_linear.score(X,Y) #linear kernel with scaling<jupyter_output><empty_output><jupyter_text>As we can see, the model with RBF kernel fits better than with Linear kernel  Let's try to create a model with Polynomial kernel SVR model with Polynomial kernel  Fitting data to the SVR model with Polynomial kernel<jupyter_code>#Let's name this model a bit differently to differenciate with previous model
svr_regressor_model_poly = SVR(kernel='poly').fit(X,Y)<jupyter_output>C:\Users\PC\AppData\Local\Enthought\Canopy\edm\envs\User\lib\site-packages\sklearn\utils\validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>Visualizing<jupyter_code>X_grid = np.arange(min(X), max(X), 0.1)
X_grid = X_grid.reshape((len(X_grid), 1))
plt.scatter(X, Y, c='r', label="Values")
plt.plot(X_grid, svr_regressor_model_poly.predict(X_grid), c='b', label="SVR")
plt.title("SVR with Polynomial kernel")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Let's get the score and compare<jupyter_code>svr_regressor_model_poly.score(X,Y) #Polynomial kernel with scaling<jupyter_output><empty_output><jupyter_text>We have created 4 models and have 4 different scores for each model. Let's summarize:    1. With default kernel. Score = 0.60    2. With Linear kernel. Score = 0.53    3. With Polynomial kernel. Score = 0.51  Conclusion:    Best model for our data was the Model with RBF(default) Kernel and     Obviously, for SVR models we need to Scale(normalize, standardize) data  prediction with best Model   Let's predict of value with best model(rbf svr).we have selected first Marketing Spend as x and wants to predict the benefit that is y, for checking the prediction of rbf svr. x=471784(marketing spend) and y=192261.83(benefit). after prediction the value of y_new by using rbf svr is 0.2385 that is because of scaling data so we should use inverse of scaling that is (inverse_transform).so you can see the predicted value is y_new_inverse is 121531.592 but if you see the real data for this x=471784.1 is y=192261.83 but as we calculated the rbf score=0.6 so this accuracy of prediction is acceptable.Now you can find the benefit of any marketing spend with this accuracy!you can try another x values and find y_new_inverse<jupyter_code>y_new=svr_regressor_rbf.predict(471784.10)
print(y_new)
y_new_inverse = scalery.inverse_transform(y_new)
y_new_inverse<jupyter_output>[ 0.23856378]<jupyter_text>Questions 1. Take R&D Spend as Independent variable(X)<jupyter_code><font size=4> 2. Create a model with Linear kernel and fit the data(X,Y)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09376966645689112, 'fraction_numerical': 0.015733165512901194, 'mean_word_length': 5.779863481228669, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '372194', 'license': 'MIT', 'n_tokens_mistral': 2707, 'n_tokens_neox': 2536, 'n_words': 871, 'path': 'SVM/SVR/SVR_1d/SVR(1d).ipynb', 'repo_name': 'rasulrukhilloev/ML_tutorials', 'size': '7945'}