<jupyter_start><jupyter_text>Simple Linear Regression  Importing the libraries The objective of a linear regression model is to find a relationship between one or more features(independent variables) and a continuous target variable(dependent variable). When there is only feature it is called Uni-variate Linear Regression and if there are multiple features, it is called Multiple Linear Regression. Hypothesis of Linear RegressionThe linear regression model can be represented by the following equation![alt text](https://miro.medium.com/max/626/1*A32yDrkQKIU0Z-Nf7JD9Pg.png)Y is the predicted valueθ₀ is the bias term.θ₁,…,θₙ are the model parametersx₁, x₂,…,xₙ are the feature values. Best Fit LineThe line for which the the error between the predicted values and the observed values is minimum is called the best fit line or the regression line. These errors are also called as residuals.![alt text](https://miro.medium.com/max/788/1*6lzTBeDt_J39XPvhI4NAMg.png) To define and measure the error of our model we define the cost function as the sum of the squares of the residuals. The cost function is denoted by![alt text](https://miro.medium.com/max/315/1*cUnU05zgo8BEzjm22oszLw.png)where the hypothesis function h(x) is denoted by![alt text](https://miro.medium.com/max/331/1*U918Cc_opofphkgbP9Kkpg.png)and m is the total number of training examples in our data-set.Why do we take the square of the residuals and not the absolute value of the residuals ? We want to penalize the points which are farther from the regression line much more than the points which lie close to the line. **Gradient descent**Gradient descent is a generic optimization algorithm used in many machine learning algorithms. It iteratively tweaks the parameters of the model in order to minimize the cost function.1. random weight intilaization 2. compute partial derivative w.r.t. paramter.![alt text](https://miro.medium.com/max/311/1*1DbFTHQGx1yW_bygxQjDCw.png)3. After computing the derivative we update the parameters as given below![alt text](https://miro.medium.com/max/321/1*eCIJ2D7iEzr16XjNIIAX-Q.png)![alt text](https://miro.medium.com/max/554/1*Gv_g5WHAGZUIeZLBmYncLw.png)here a in learning parameter. We repeat the steps 2,3 until the cost function converges to the minimum value. If the value of α is too small, the cost function takes larger time to converge. If α is too large, gradient descent may overshoot the minimum and may finally fail to converge. ![alt text](https://miro.medium.com/max/1400/1*n79s9gvd0E8ALe9dLUEKAw.png)<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score<jupyter_output><empty_output><jupyter_text>Importing the dataset<jupyter_code>dataset = pd.read_csv('https://raw.githubusercontent.com/Uttam580/Machine_learning/master/datasets./Salary_Data.csv')
X = dataset.iloc[:, :-1].values
y = dataset.iloc[:, -1].values<jupyter_output><empty_output><jupyter_text>Splitting the dataset into the Training set and Test set<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 1/3, random_state = 0)<jupyter_output><empty_output><jupyter_text>Training the Simple Linear Regression model on the Training set<jupyter_code>from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Predicting the Test set results<jupyter_code>y_pred = regressor.predict(X_test)<jupyter_output><empty_output><jupyter_text>Visualising the Training set results<jupyter_code>plt.scatter(X_train, y_train, color = 'red')
plt.plot(X_train, regressor.predict(X_train), color = 'blue')
plt.title('Salary vs Experience (Training set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()<jupyter_output><empty_output><jupyter_text>Visualising the Test set results<jupyter_code>plt.scatter(X_test, y_test, color = 'red')
plt.plot(X_train, regressor.predict(X_train), color = 'blue')
plt.title('Salary vs Experience (Test set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()<jupyter_output><empty_output><jupyter_text>**Model Evaluation** **Mean Absolute Error (MAE)**It is simply the average of the squared difference between the target value and the value predicted by the regression model. As it squares the differences, it penalizes even a small error which leads to over-estimation of how bad the model is**Root Mean Squared Error:** it is the square root of the averaged squared difference between the target value and the value predicted by the model. RMSE is highly affected by outlier values.As compared to mean absolute error, RMSE gives higher weightage and punishes large errors.![alt text](https://4.bp.blogspot.com/-wG7IbjTfE6k/XGUvqm7TCVI/AAAAAAAAAZU/vpH1kuKTIooKTcVlnm1EVRCXLVZM9cPNgCLcBGAs/s1600/formula-MAE-MSE-RMSE-RSquared.JPG) **Mean Absolute Error:**MAE is the absolute difference between the target value and the value predicted by the model. The MAE is more robust to outliers and does not penalize the errors as extremely as mse. MAE is a linear score which means all the individual differences are weighted equally. It is not suitable for applications where you want to pay more attention to the outliers.**R² Error:**Coefficient of Determination or R² is another metric used for evaluating the performance of a regression model. the R² will always be less than or equal to 1.Nearer the value to 1 means model is good.R-Squared does not penalize for adding features that add no value to the model.**Adjusted R2 error:**![alt text](https://miro.medium.com/max/792/0*WkdWEm2993yhYvUA.png)R2 shows how well terms (data points) fit a curve or line. Adjusted R2 also indicates how well terms fit a curve or line, but adjusts for the number of terms in a model. If you add more and more useless variables to a model, adjusted r-squared will decrease. If you add more useful variables, adjusted r-squared will increase.Adjusted R2 will always be less than or equal to R2.<jupyter_code># model evaluation
rmse = mean_squared_error(y_test, y_pred)
r2 =r2_score(y_test, y_pred)
# printing values
print('Slope:' ,regressor.coef_)
print('Intercept:', regressor.intercept_)
print('Root mean squared error: ', rmse)
print('R2 score: ', r2)<jupyter_output>Slope: [9345.94244312]
Intercept: 26816.192244031183
Root mean squared error:  21026037.329511296
R2 score:  0.9749154407708353
{'fraction_non_alphanumeric': 0.07857142857142857, 'fraction_numerical': 0.026139817629179333, 'mean_word_length': 5.927368421052631, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 11, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '618740', 'license': 'MIT', 'n_tokens_mistral': 2170, 'n_tokens_neox': 1987, 'n_words': 747, 'path': 'supervised_learning/simple_linear_regression.ipynb', 'repo_name': 'Uttam580/basic_ml', 'size': '6580'}