<jupyter_start><jupyter_text>Exploring Dataset<jupyter_code>list(wine.columns)
wine.dtypes
wine.describe()<jupyter_output><empty_output><jupyter_text>We see that, there is big difference between Mean and 50% for column - 'Free Sulphur Dioxide' and 'Total Sulphur'. Means there is skewness.Also, we see big difference between Max and 75% for columns except - 'Quality' and Alcohol, denoting outliers in the dataset<jupyter_code>wine.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 1599 entries, 0 to 1598
Data columns (total 12 columns):
 #   Column                Non-Null Count  Dtype  
---  ------                --------------  -----  
 0   fixed acidity         1599 non-null   float64
 1   volatile acidity      1599 non-null   float64
 2   citric acid           1599 non-null   float64
 3   residual sugar        1599 non-null   float64
 4   chlorides             1599 non-null   float64
 5   free sulfur dioxide   1599 non-null   float64
 6   total sulfur dioxide  1599 non-null   float64
 7   density               1599 non-null   float64
 8   pH                    1599 non-null   float64
 9   sulphates             1599 non-null   float64
 10  alcohol               1599 non-null   float64
 11  quality               1599 non-null   int64  
dtypes: float64(11), int64(1)
memory usage: 150.0 KB<jupyter_text>There are no missing value. It's a great thing! Phew!  Checking Outliers<jupyter_code>f, axes = plt.subplots(3, 4, figsize=(10, 7), sharex=True)

sns.boxplot('fixed acidity',data=wine,orient='v',ax=axes[0,0],color='Blue')

sns.boxplot('volatile acidity',data=wine,orient='v',ax=axes[0,1],color='orange')

sns.boxplot('citric acid',data=wine,orient='v',ax=axes[0,2],color='yellow')

sns.boxplot('residual sugar',data=wine,orient='v',ax=axes[0,3],color='red')

sns.boxplot('chlorides',data=wine,orient='v',ax=axes[1,0],color='pink')

sns.boxplot('free sulfur dioxide',data=wine,orient='v',ax=axes[1,1],color='white')

sns.boxplot('total sulfur dioxide',data=wine,orient='v',ax=axes[1,2],color='green')

sns.boxplot('density',data=wine,orient='v',ax=axes[1,3],color='purple')

sns.boxplot('pH',data=wine,orient='v',ax=axes[2,0],color='brown')

sns.boxplot('sulphates',data=wine,orient='v',ax=axes[2,1],color='pink')

sns.boxplot('alcohol',data=wine,orient='v',ax=axes[2,2],color='red')

sns.boxplot('quality',data=wine,orient='v',ax=axes[2,3],color='orange')<jupyter_output><empty_output><jupyter_text>Plotting graph for Correlation<jupyter_code>corr = wine.corr()
mask = np.zeros_like(corr)
mask[np.triu_indices_from(mask)] = True
with sns.axes_style("white"):
     f, ax = plt.subplots(figsize=(12, 12))
     ax = sns.heatmap(corr, mask=mask, vmax=.3, square=True, cmap="YlGnBu",annot=True)<jupyter_output><empty_output><jupyter_text>Plotting Graphs to check Dependency of features on Quality<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Fixed Acidity")
sns.barplot(x='quality',y='fixed acidity',data=wine,palette='rocket')<jupyter_output><empty_output><jupyter_text>On increasing FIXED ACIDITY, QUALITY INCREASE<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Volatile Acidity")
sns.barplot(x='quality',y='volatile acidity',data=wine,palette='vlag')<jupyter_output><empty_output><jupyter_text>On INCREASING VOLATILE ACIDITY, QUALITY DECREASES<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Citric Acid")
sns.barplot(x='quality',y='citric acid',data=wine,palette='deep')<jupyter_output><empty_output><jupyter_text>On INCREASING CITRIC ACID, QUALITY INCREASE<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Residual Sugar")
sns.barplot(x='quality',y='residual sugar',data=wine,palette='husl')<jupyter_output><empty_output><jupyter_text>There is a small Positive correlation between Residual SUgar and Quality which means it doesn't affect the quality much<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Free Sulphur dioxide")
sns.barplot(x='quality',y='free sulfur dioxide',data=wine,palette='Set1')<jupyter_output><empty_output><jupyter_text>SMALL NEGATIVE CORRELATION<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Total Sulphur dioxide")
sns.barplot(x='quality',y='total sulfur dioxide',data=wine,palette='Set3')<jupyter_output><empty_output><jupyter_text>QUALITY DECREASES with INCREASE in TOTAL SULFUR DIOXIDE<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Density")
sns.barplot(x='quality',y='density',data=wine,palette='Set2')<jupyter_output><empty_output><jupyter_text>SMALL DECREASE in QUALITY with INCREASE in DENSITY<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs pH")
sns.barplot(x='quality',y='pH',data=wine,palette='colorblind')<jupyter_output><empty_output><jupyter_text>SMALL NEGATIVE CORRELATION BETWEEN pH and Quality. Doesn't seem to affect much!<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Sulphates")
sns.barplot(x='quality',y='sulphates',data=wine,palette='bright')<jupyter_output><empty_output><jupyter_text>QUALITY INCREASES with INCREASE in SULPHATES<jupyter_code>plt.figure(figsize=(7,5))
plt.title("Quality vs Alcohol")
sns.barplot(x='quality',y='alcohol',data=wine,palette='deep')<jupyter_output><empty_output><jupyter_text>QUALITY INCREASES with INCREASE of ALCOHOL  Plotting and calculating Skewness<jupyter_code>print("Skewness and Kurtosis of all the columns")
print("++++++++++++++++++++++++++++++++++++++++++")
for col in list(wine.columns):
    print("{0} - Skewness : {1:.3f} and Kurtosis : {2:.3f}".format(col,wine[col].skew(),wine[col].kurt()))
    print("-----------------------------------------------")
ax = sns.distplot(wine['fixed acidity'])
ax.set_xlabel('Fixed Acidity',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['volatile acidity'])
ax.set_xlabel('Volatile Acidity',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['citric acid'])
ax.set_xlabel('Citric Acid',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['residual sugar'])
ax.set_xlabel('Residual Sugar',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['chlorides'])
ax.set_xlabel('Chlorides',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['free sulfur dioxide'])
ax.set_xlabel('Free Sulphur Dioxide',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['total sulfur dioxide'])
ax.set_xlabel('Total Sulphur Dioxide',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['density'])
ax.set_xlabel('Density',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['pH'])
ax.set_xlabel('pH',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['sulphates'])
ax.set_xlabel('Sulphates',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wine['alcohol'])
ax.set_xlabel('Alcohol',fontsize=14)
ax.set_ylabel('Count',fontsize=14)<jupyter_output><empty_output><jupyter_text>Note - Skewness is caused by Outliers present in the dataset. So by removing outliers we remove the SKewness also  Removing Skewness by calculating IQR value (Interquartile Range)<jupyter_code>wineCopy = wine
#wineCopy.drop(['quality'],axis=1,inplace=True)
from scipy import stats

q1 = wineCopy.quantile(0.25)
q3 = wineCopy.quantile(0.75)
iqr = q3 - q1

Q1 = q1
Q3 = q3
IQR = iqr

print(iqr)
print(wineCopy < (11 - 1.5 * iqr)) or (wineCopy > (q3 + 1.5 * iqr))
wineCopy = wineCopy[~((wineCopy < (Q1 - 1.5 * IQR)) |(wineCopy > (Q3 + 1.5 * IQR))).any(axis=1)]
wineCopy.shape
print("Skewness and Kurtosis of all the columns after Removing Outliers")
print("++++++++++++++++++++++++++++++++++++++++++")
for col in list(wineCopy.columns):
    print("{0} - Skewness : {1:.3f} and Kurtosis : {2:.3f}".format(col,wineCopy[col].skew(),wineCopy[col].kurt()))
    print("-----------------------------------------------")<jupyter_output>Skewness and Kurtosis of all the columns after Removing Outliers
++++++++++++++++++++++++++++++++++++++++++
fixed acidity - Skewness : 0.738 and Kurtosis : 0.063
-----------------------------------------------
volatile acidity - Skewness : 0.270 and Kurtosis : -0.237
-----------------------------------------------
citric acid - Skewness : 0.312 and Kurtosis : -0.907
-----------------------------------------------
residual sugar - Skewness : 0.610 and Kurtosis : 0.399
-----------------------------------------------
chlorides - Skewness : 0.222 and Kurtosis : 0.185
-----------------------------------------------
free sulfur dioxide - Skewness : 0.832 and Kurtosis : 0.055
-----------------------------------------------
total sulfur dioxide - Skewness : 0.980 and Kurtosis : 0.329
-----------------------------------------------
density - Skewness : 0.052 and Kurtosis : -0.025
-----------------------------------------------
pH - Skewness : 0.061 and Kurtosis : -0.203
------------------------[...]<jupyter_text>Now we see that Skewness has reduced a lot compared to previous values of Skewness of each Column. Now we can proceed with Training our model for prediction of Quality. Outliers may tamper our model's prediction as they are values which have been miscalculated or values which have been recorded wrong  Plotting Graphs to Compare Skewness<jupyter_code>ax = sns.distplot(wineCopy['fixed acidity'])
ax.set_xlabel('Fixed Acidity',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['volatile acidity'])
ax.set_xlabel('Volatile Acidity',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['citric acid'])
ax.set_xlabel('Citric Acid',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['residual sugar'])
ax.set_xlabel('Residual Sugar',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['free sulfur dioxide'])
ax.set_xlabel('Free Sulphur Dioxide',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['total sulfur dioxide'])
ax.set_xlabel('Total Sulphur Dioxide',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['density'])
ax.set_xlabel('Density',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['pH'])
ax.set_xlabel('pH',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['sulphates'])
ax.set_xlabel('Sulphates',fontsize=14)
ax.set_ylabel('Count',fontsize=14)
ax = sns.distplot(wineCopy['alcohol'])
ax.set_xlabel('Alcohol',fontsize=14)
ax.set_ylabel('Count',fontsize=14)<jupyter_output><empty_output><jupyter_text>Now you can compare each graphs after removing outliers to that which had outliers. A great difference we see. No Positive or Negative skewness is observed. Now we are sure that skewness has reduced a lot. Hence we can start training of out data<jupyter_code>#wine = pd.read_csv('./winequality-red.csv')
wineCopy['quality'].value_counts()<jupyter_output><empty_output><jupyter_text>After Removing Outliers we see that Alcohol of Quality 3 and 9 are gone. So since the dataset is very small, we will proceed without removing Outliers  Preparing Dataset for Training<jupyter_code>#FOR DIVIDING QUALITY IN 2 CLASS - 0 and 1

bins = (2, 6.5, 8)
labels = ['bad', 'good']
wine['quality'] = pd.cut(x = wine['quality'], bins = bins, labels = labels)

from sklearn.preprocessing import LabelEncoder
labelencoder_y = LabelEncoder()
wine['quality'] = labelencoder_y.fit_transform(wine['quality'])

wine.head()

wine['quality'].value_counts()
wine['quality'].value_counts()<jupyter_output><empty_output><jupyter_text>StratifiedShuffleSplit is used to maintain the correct ratio of value counts while splitting the data for Train and Test. Assume we have 10 'Ones' and 6 'Zeros'. If we randomly split, it may happen that Train set will have 8 'Ones' and 2 'Zeros'. If this happends, out model will be biased towards 'Ones'. Hence the above mentioned method is used to make sure that our model is not bias<jupyter_code>from sklearn.model_selection import StratifiedShuffleSplit
split = StratifiedShuffleSplit(n_splits=1,test_size=0.2,random_state=42)

for train_index, test_index in split.split(wine,wine['quality']):
    strat_train_set = wine.loc[train_index]
    strat_test_set = wine.loc[test_index]

strat_test_set.head()
wine.shape
print("Test size : {0}, Train size : {1}".format(strat_train_set.shape,strat_test_set.shape))
X_train_set = strat_train_set.drop('quality',axis=1).values
y_train_set = strat_train_set['quality'].values.reshape(-1,1)

X_test_set = strat_test_set.drop('quality',axis=1).values
y_test_set = strat_test_set['quality'].values.reshape(-1,1)
print("Size of X_train : {0}, y_train : {1}".format(X_train_set.shape, y_train_set.shape))
print("Size of X_test : {0}, y_test : {1}".format(X_test_set.shape, y_test_set.shape))
def countValue(data):
    zero = 0
    one = 0
    for i in data:
        if(i==0):
            zero +=1
        else:
            one +=1
    s = "Zeros : "+str(zero)+" Ones : "+str(one)
    return s
print("In Train Set \'quality\' : "+countValue(y_train_set))
print("In Test Set \'quality\' : "+countValue(y_test_set))<jupyter_output>In Train Set 'quality' : Zeros : 1105 Ones : 174
In Test Set 'quality' : Zeros : 277 Ones : 43<jupyter_text>StandardScaler is used to scale data as we see there is big difference between minimum and maximum values on few columns. TO improve accuracy we should scale the data. It takes input as Numpy.array<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_set = scaler.fit_transform(X_train_set)
#Y_train_set = StandardScaler(Y_train_set)
X_test_set = scaler.fit_transform(X_test_set)
#Y_test_set = StandardScaler(Y_test_set)<jupyter_output><empty_output><jupyter_text>TRAINING VARIOUS ALGORITHMS<jupyter_code>from sklearn.metrics import accuracy_score,confusion_matrix,precision_score,classification_report
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import RandomizedSearchCV, cross_val_score
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier<jupyter_output><empty_output><jupyter_text>Linear Regression<jupyter_code>logisticRegModel = LogisticRegression(C=1, fit_intercept=True, max_iter=1000,penalty='l2',solver='liblinear',random_state=42)
logisticRegModel.fit(X_train_set,y_train_set.ravel())
y_pred_logistic = logisticRegModel.predict(X_test_set)
accuracy_lr_test = accuracy_score(y_test_set,y_pred_logistic)

print("Accuracy on Test Set : "+str(accuracy_lr_test*100)+str("%"))
print("\nClassification Report is as follows : ")
print(classification_report(y_test_set,y_pred_logistic))
print('Confusion Matrix')
confusion_matrix(y_test_set,y_pred_logistic)

cvs_lr = cross_val_score(logisticRegModel,X_train_set,y_train_set,cv=5)

print("Cross Validation score : "+str(cvs_lr.mean()))<jupyter_output>Accuracy on Test Set : 89.375%

Classification Report is as follows : 
              precision    recall  f1-score   support

           0       0.91      0.97      0.94       277
           1       0.68      0.40      0.50        43

    accuracy                           0.89       320
   macro avg       0.80      0.68      0.72       320
weighted avg       0.88      0.89      0.88       320

Confusion Matrix
Cross Validation score : 0.8733486519607844<jupyter_text>Performing Hyperparameter Tuning<jupyter_code>dual=[True,False]
max_iter=[100,200,500,800,1000,1200]
solver = ['lbfgs', 'liblinear']
penalty = ['l1', 'l2']
param_grid = dict(dual=dual,max_iter=max_iter,solver=solver,penalty=penalty)
logisticRegRSCV = RandomizedSearchCV(logisticRegModel,param_grid, random_state=42, n_iter=100, cv=10, verbose=0, n_jobs=-1)
logisticRegRSCV.fit(X_train_set,y_train_set)
logisticRegRSCV.best_score_
logisticRegRSCV.best_params_<jupyter_output><empty_output><jupyter_text>Now we got the best parameters for our model! Now We will again create our model<jupyter_code>logisticRegModel_tuned = LogisticRegression(solver= 'liblinear', penalty='l1',max_iter= 100, dual= False)
logisticRegModel_tuned.fit(X_train_set,y_train_set)
y_pred_logistic_tuned = logisticRegModel_tuned.predict(X_test_set)
accuracy_lr_test_tuned = accuracy_score(y_test_set,y_pred_logistic_tuned)
print("Accuracy after Hyperparameter Tuning : "+str(accuracy_lr_test_tuned*100)+str("%"))
print("\nClassification Report is as follows : ")
print(classification_report(y_test_set,y_pred_logistic_tuned))
print("Confusion Matrix :")
confusion_matrix(y_test_set,y_pred_logistic_tuned)

cvs_lr_tuned = cross_val_score(logisticRegModel_tuned,X_train_set,y_train_set,cv=5)

print("Cross Validation score : "+str(cvs_lr_tuned.mean()))<jupyter_output>Accuracy after Hyperparameter Tuning : 89.6875%

Classification Report is as follows : 
              precision    recall  f1-score   support

           0       0.91      0.97      0.94       277
           1       0.71      0.40      0.51        43

    accuracy                           0.90       320
   macro avg       0.81      0.69      0.72       320
weighted avg       0.88      0.90      0.88       320

Confusion Matrix :
Cross Validation score : 0.8772549019607844<jupyter_text>So, the accuracy of Logistic Regression model Increased from 89.375% to 89.68% and Precision increased from 0.6 to 0.65  K-Nearest Neighbours<jupyter_code>knnModel = KNeighborsClassifier(leaf_size=1, metric='minkowski', metric_params=None,n_jobs=None,n_neighbors=32, p=2,weights='distance')
knnModel.fit(X_train_set,y_train_set)
y_pred_knn = knnModel.predict(X_test_set)
accuracy_knn_test = accuracy_score(y_test_set,y_pred_knn)
print("Accuracy of KNN Classifer is : "+str(accuracy_knn_test*100)+str("%"))
print("\nClassification Report is as follows : ")
print(classification_report(y_test_set,y_pred_knn))
print("Confusion Matrix")
print(confusion_matrix(y_test_set,y_pred_knn))

cvs_knn = cross_val_score(knnModel,X_train_set,y_train_set,cv=5)
print("Cross Validation score : "+str(cvs_knn.mean()))
params = {'n_neighbors':[5,15,20,25,30,25,40,45,50],
          'leaf_size':[1,2,3,4,5,8,10],
          'weights':['uniform', 'distance'],
          'algorithm':['auto', 'ball_tree','kd_tree','brute'],
          'n_jobs':[-1]}
knnModelRSCV = RandomizedSearchCV(knnModel,params,n_jobs=-1,cv=10,n_iter=100,random_state=42)
knnModelRSCV.fit(X_train_set,y_train_set)
knnModelRSCV.best_score_
knnModelRSCV.best_estimator_
knnModel_tuned = KNeighborsClassifier(algorithm='brute', leaf_size=5, metric='minkowski',
                     metric_params=None, n_jobs=-1, n_neighbors=20, p=2,
                     weights='distance')
knnModel_tuned.fit(X_train_set,y_train_set)
y_pred_knn_tuned = knnModel_tuned.predict(X_test_set)
accuracy_knn_test_tuned = accuracy_score(y_test_set,y_pred_knn_tuned)
print("Accuracy of KNN after Hyperparameter Tuning : "+str(accuracy_knn_test_tuned*100)+str("%"))
print("\nClassification Report is as follows : ")
print(classification_report(y_test_set,y_pred_knn_tuned))
print("Confusion Matrix")
print(confusion_matrix(y_test_set,y_pred_knn_tuned))

cvs_knn_tuned = cross_val_score(knnModel_tuned,X_train_set,y_train_set,cv=5)

print("Cross Validation score : "+str(cvs_knn_tuned.mean()))<jupyter_output>Accuracy of KNN after Hyperparameter Tuning : 92.1875%

Classification Report is as follows : 
              precision    recall  f1-score   support

           0       0.93      0.99      0.96       277
           1       0.85      0.51      0.64        43

    accuracy                           0.92       320
   macro avg       0.89      0.75      0.80       320
weighted avg       0.92      0.92      0.91       320

Confusion Matrix
[[273   4]
 [ 21  22]]
Cross Validation score : 0.8952267156862745<jupyter_text>Before Hypertuing, accuracy was 92.81% and after Hypertuning is 92.1875%  Random Forest Classifier<jupyter_code>randomForestModel = RandomForestClassifier(criterion = 'entropy',max_features=4,n_estimators = 500, random_state=42)
randomForestModel.fit(X_train_set,y_train_set)
y_pred_rfc = randomForestModel.predict(X_test_set)
accuracy_rfc_test = accuracy_score(y_test_set,y_pred_rfc)
print("Accuracy on Test set : {0:.2f} %".format(accuracy_rfc_test*100))
print("Classification Report as follows")
print(classification_report(y_test_set,y_pred_rfc))
print("Confusion Matrix ")
print(confusion_matrix(y_test_set,y_pred_rfc))

cvs_rfc = cross_val_score(randomForestModel,X_train_set,y_train_set,cv=5)

print("Cross Validation score : "+str(cvs_rfc.mean()))<jupyter_output>Accuracy on Test set : 93.75 %
Classification Report as follows
              precision    recall  f1-score   support

           0       0.94      0.99      0.96       277
           1       0.93      0.58      0.71        43

    accuracy                           0.94       320
   macro avg       0.93      0.79      0.84       320
weighted avg       0.94      0.94      0.93       320

Confusion Matrix 
[[275   2]
 [ 18  25]]<jupyter_text>Performing Hypertuning<jupyter_code>n_estimators = [int(x) for x in np.linspace(start=100, stop=1500,num=12)]

max_features = ['auto','sqrt']

max_depth = [int(x) for x in np.linspace(10,110,num=13)]
max_depth.append('None')

min_samples_split = [2,5,10,15,20]

min_samples_leaf = [1,2,4,6,8,10]
bootstrap = [True,False]

random_grid = {'n_estimators': n_estimators,
               'max_features': max_features,
               'max_depth': max_depth,
               'min_samples_split': min_samples_split,
               'min_samples_leaf': min_samples_leaf,
               'bootstrap': bootstrap}
randomforestRSCV = RandomizedSearchCV(estimator=randomForestModel, param_distributions=random_grid, n_iter=100,cv=5,verbose=2,random_state=42,n_jobs=-1)
randomforestRSCV.fit(X_train_set,y_train_set)
randomforestRSCV.best_estimator_
randomforestRSCV.best_score_
randomForestModel_tuned = RandomForestClassifier(bootstrap=True, ccp_alpha=0.0, class_weight=None,
                       criterion='entropy', max_depth=101, max_features='auto',
                       max_leaf_nodes=None, max_samples=None,
                       min_impurity_decrease=0.0, min_impurity_split=None,
                       min_samples_leaf=2, min_samples_split=2,
                       min_weight_fraction_leaf=0.0, n_estimators=1245,
                       n_jobs=None, oob_score=False, random_state=42, verbose=0,
                       warm_start=False)

randomForestModel_tuned.fit(X_train_set,y_train_set)
y_pred_rfc_tuned = randomForestModel_tuned.predict(X_test_set)
accuracy_rfc_test_tuened = accuracy_score(y_test_set,y_pred_rfc_tuned)
print("Accuracy on Test set : {0:.2f} %".format(accuracy_rfc_test_tuened*100))
print("Classification Report as follows")
print(classification_report(y_test_set,y_pred_rfc_tuned))
print("Confusion Matrix ")
print(confusion_matrix(y_test_set,y_pred_rfc_tuned))
cvs_rfc_tuned = cross_val_score(randomForestModel_tuned,X_train_set,y_train_set,cv=5)

print("Cross Validation score : "+str(cvs_rfc_tuned.mean()))
models = [('Logistic Regression', accuracy_lr_test, cvs_lr.mean()),
          ('Logistic Regression Tuned', accuracy_lr_test_tuned, cvs_lr_tuned.mean()),
          ('KNN', accuracy_knn_test, cvs_knn.mean()),
          ('KNN Tuned', accuracy_knn_test_tuned, cvs_knn_tuned.mean()),
          ('Random Forest Classifier', accuracy_rfc_test, cvs_rfc.mean()),
          ('Logistic Regression', accuracy_rfc_test_tuened, cvs_rfc_tuned.mean()),
         
         ]<jupyter_output><empty_output><jupyter_text>Visualizing Models<jupyter_code>predictions = pd.DataFrame(data=models,columns=['Model','Accuracy of Test Data','Cross Validation Score'])
predictions
f,ax = plt.subplots(1,1,figsize=(18,10))
predictions.sort_values(by=['Cross Validation Score'],ascending=False,inplace=True)
sns.barplot(x='Cross Validation Score', y='Model',data=predictions,palette='Reds_d')
#axes[0].set(xlabel='Region', ylabel='Charges')
ax.set_xlabel('Cross-Validaton Score', size=16)
ax.set_ylabel('Model')
ax.set_xlim(0,1.0)
ax.set_xticks(np.arange(0, 1.1, 0.1))
plt.show()
f,ax = plt.subplots(1,1,figsize=(18,10))
predictions.sort_values(by=['Accuracy of Test Data'],ascending=False,inplace=True)
sns.barplot(x='Accuracy of Test Data', y='Model',data=predictions,palette='Blues_d')
#axes[0].set(xlabel='Region', ylabel='Charges')
ax.set_xlabel('Accuracy of Test Data', size=16)
ax.set_ylabel('Model')
ax.set_xlim(0,1.0)
ax.set_xticks(np.arange(0, 1.1, 0.1))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14980504822491278, 'fraction_numerical': 0.03935973732813462, 'mean_word_length': 4.783527177783053, 'pattern_counts': {'":': 0, '<': 113, '<?xml version=': 0, '>': 113, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '29', 'dir': 'jupyter-structured-clean-dedup', 'id': '468021', 'license': 'MIT', 'n_tokens_mistral': 9505, 'n_tokens_neox': 8602, 'n_words': 1598, 'path': 'Red Wine EDA and Prediction.ipynb', 'repo_name': 'SahilChachra/Redwine-Quality-Analysis-and-Prediction', 'size': '24365'}