<jupyter_start><jupyter_text>Assumptions for Linear Regression1. LinearityLinear regression needs the relationship between the independent and dependent variables to be linear. Let's use a pair plot to check the relation of independent variables with the actual_performance variable<jupyter_code>p = sns.pairplot(df, x_vars=['no_of_workers','over_time','incentive','idle_men'], y_vars='actual_productivity', size=7, aspect=0.7)<jupyter_output>C:\ProgramData\Anaconda3\lib\site-packages\seaborn\axisgrid.py:1969: UserWarning: The `size` parameter has been renamed to `height`; please update your code.
  warnings.warn(msg, UserWarning)<jupyter_text>As observed, there is no linearity between the dependent and independent variablers.So it shows that a linear regression fitting might not be the best model for it. A linear model might not be able to efficiently explain the data in terms of variability, prediction accuracy etc.Thus, we cannot carry forward with the linear regression model.<jupyter_code>df2=pd.get_dummies(df)
# Fitting the linear model
x = df2.drop(["actual_productivity"],axis=1)
y = df2.actual_productivity
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X = sc.fit_transform(x)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,random_state = 0,test_size=0.25)
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error
from sklearn.metrics import r2_score
from sklearn import linear_model

regr = linear_model.LinearRegression()
regr.fit(X_train,y_train)
y_pred = regr.predict(X_train)
print("R squared: {}".format(r2_score(y_true=y_train,y_pred=y_pred)))<jupyter_output>R squared: 0.3412174424967326<jupyter_text>2. Mean of ResidualsResiduals as we know are the differences between the true value and the predicted value. One of the assumptions of linear regression is that the mean of the residuals should be zero. So let's find out.<jupyter_code>residuals = y_train.values-y_pred
mean_residuals = np.mean(residuals)
print("Mean of Residuals {}".format(mean_residuals))<jupyter_output>Mean of Residuals -0.00047661176360436284<jupyter_text>Mean of residual is very close to 0, so we are good here. 3. Check for HomoscedasticityHomoscedasticity means that the residuals have equal or almost equal variance across the regression line. By plotting the error terms with predicted terms we can check that there should not be any pattern in the error terms.Detecting heteroscedasticity!Graphical Method: Firstly do the regression analysis and then plot the error terms against the predicted values( Yi^). If there is a definite pattern (like linear or quadratic or funnel shaped) obtained from the scatter plot then heteroscedasticity is present.<jupyter_code>p = sns.scatterplot(y_pred,residuals)
plt.xlabel('y_pred/predicted values')
plt.ylabel('Residuals')
plt.ylim(-10,10)
plt.xlim(0,26)
p = sns.lineplot([0,26],[0,0],color='blue')
p = plt.title('Residuals vs fitted values plot for homoscedasticity check')<jupyter_output>C:\ProgramData\Anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(
C:\ProgramData\Anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(<jupyter_text>Goldfeld Quandt Test-------------------------------------Checking heteroscedasticity : Using Goldfeld Quandt we test for heteroscedasticity.Null Hypothesis: Error terms are homoscedasticAlternative Hypothesis: Error terms are heteroscedastic.<jupyter_code>import statsmodels.stats.api as sms
from statsmodels.compat import lzip
name = ['F statistic', 'p-value']
test = sms.het_goldfeldquandt(residuals, X_train)
lzip(name, test)<jupyter_output><empty_output><jupyter_text>Since p value is more than 0.05 in Goldfeld Quandt Test, we can't reject it's null hypothesis that error terms are homoscedastic. Bartlett’s test tests the null hypothesis that all input samples are from populations with equal variances.<jupyter_code>from scipy.stats import bartlett
test = bartlett(X_train[0],residuals)
print(test)

test = bartlett(X_train[1],residuals)
print(test)

test = bartlett(X_train[2],residuals)
print(test)<jupyter_output>BartlettResult(statistic=1236.433485650819, pvalue=7.368585254791994e-271)
BartlettResult(statistic=1251.3691876832208, pvalue=4.1834487291423117e-274)
BartlettResult(statistic=1204.1849048257968, pvalue=7.512835591327921e-264)<jupyter_text>If p value is more than 0.05 in Bartlett, it's null hypothesis that error terms are homoscedastic gets accepted. 4. Check for Normality of error terms/residuals<jupyter_code>p = sns.distplot(residuals,kde=True)
p = plt.title('Normality of error terms/residuals')<jupyter_output>C:\ProgramData\Anaconda3\lib\site-packages\seaborn\distributions.py:2557: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `histplot` (an axes-level function for histograms).
  warnings.warn(msg, FutureWarning)<jupyter_text>The residual terms are pretty much normally distributed for the number of test points we took. A skew is also visible from the plot. It's very difficult to get perfect curves, distributions in real life data. 5. No autocorrelation of residualsWhen the residuals are autocorrelated, it means that the current value is dependent of the previous (historic) values and that there is a definite unexplained pattern in the Y variable that shows up in the error terms. Though it is more evident in time series data.There should not be autocorrelation in the data so the error terms should not form any pattern.<jupyter_code>plt.figure(figsize=(10,5))
p = sns.lineplot(y_pred,residuals,marker='o',color='blue')
plt.xlabel('y_pred/predicted values')
plt.ylabel('Residuals')
plt.ylim(-10,10)
plt.xlim(0,26)
p = sns.lineplot([0,26],[0,0],color='red')
p = plt.title('Residuals vs fitted values plot for autocorrelation check')<jupyter_output>C:\ProgramData\Anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(
C:\ProgramData\Anaconda3\lib\site-packages\seaborn\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  warnings.warn(<jupyter_text>Checking for autocorrelation To ensure the absence of autocorrelation we use Ljungbox test.Null Hypothesis: Autocorrelation is present.Alternative Hypothesis: Autocorrelation is absent.<jupyter_code>from statsmodels.stats import diagnostic as diag
min(diag.acorr_ljungbox(residuals , lags = 40)[1])
#Since p value is mpre than 0.05 we accept the null hypothesis that error terms are not autocorrelated.
import statsmodels.api as sm
# autocorrelation
sm.graphics.tsa.plot_acf(residuals, lags=40)
plt.show()
# partial autocorrelation
sm.graphics.tsa.plot_pacf(residuals, lags=40)
plt.show()<jupyter_output><empty_output><jupyter_text>The results do not show signs of autocorelation since there are no spikes outside the blue confidence interval region. 6. No perfect multicollinearityIn regression, multicollinearity refers to the extent to which independent variables are correlated. Multicollinearity affects the coefficients and p-values, but it does not influence the predictions, precision of the predictions, and the goodness-of-fit statistics. If your primary goal is to make predictions, and you don’t need to understand the role of each independent variable, you don’t need to reduce severe multicollinearity<jupyter_code>plt.figure(figsize=(20,20))
p=sns.heatmap(df.corr(), annot=True,cmap='RdYlGn',square=True)<jupyter_output><empty_output><jupyter_text>Look for correlation of rows where the dependent variable (actual_performance in this case) is not involved because if a variable is correlated with the dependent variable then this would be a good sign for our model. Correlation within dependent variables is what we need to look for and avoid. This data doesn't contain perfect multicollinearity among independent variables. In case there was any then we would try to remove one of the correlated variables depending on which was more important to our regression model.<jupyter_code>## Decision Tree

from sklearn.tree import DecisionTreeRegressor

dec_tree = DecisionTreeRegressor(random_state=0)
dec_tree.fit(X_train,y_train)
dec_tree_y_pred = dec_tree.predict(X_train)
print("Accuracy: {}".format(dec_tree.score(X_train,y_train)))
print("R squared: {}".format(r2_score(y_true=y_train,y_pred=dec_tree_y_pred)))
# SVM

from sklearn.svm import SVR

svr = SVR()
svr.fit(X_train,y_train)
svr_y_pred = svr.predict(X_train)
print("Accuracy: {}".format(svr.score(X_train,y_train)))
print("R squared: {}".format(r2_score(y_true=y_train,y_pred=svr_y_pred)))<jupyter_output>Accuracy: 0.5164469302618175
R squared: 0.5164469302618175
{'fraction_non_alphanumeric': 0.08208415176263827, 'fraction_numerical': 0.029670216065336503, 'mean_word_length': 6.418711656441718, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '348261', 'license': 'Apache-2.0', 'n_tokens_mistral': 3143, 'n_tokens_neox': 2886, 'n_words': 1007, 'path': 'MLP - Linear Regression Assignment - Group 9.ipynb', 'repo_name': 'sonagaur10/ML-Projects', 'size': '9673'}