<jupyter_start><jupyter_text>Histogram plots for various attributes<jupyter_code>fig = px.histogram(data_frame,x='quality')
fig.show()
fig = px.histogram(data_frame,x='fixed acidity')
fig.show()

fig = px.histogram(data_frame,x='volatile acidity')
fig.show()

fig = px.histogram(data_frame,x='citric acid')
fig.show()

fig = px.histogram(data_frame,x='residual sugar')
fig.show()

fig = px.histogram(data_frame,x='alcohol')
fig.show()

fig = px.histogram(data_frame,x='pH')
fig.show()<jupyter_output><empty_output><jupyter_text>Box Plots for various attributes<jupyter_code>sns.boxplot(x=data_frame['quality'],y=data_frame['fixed acidity'])
sns.boxplot(x=data_frame ['quality'],y=data_frame['volatile acidity'])
sns.boxplot(x=data_frame ['quality'],y=data_frame['citric acid'])
sns.boxplot(x=data_frame ['quality'],y=data_frame['residual sugar'])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,4])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,5])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,6])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,7])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,8])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,9])
sns.boxplot(x=data_frame ['quality'],y=data_frame.iloc[:,10])
sns.boxplot(x=data_frame['quality'], y=data_frame['chlorides'])<jupyter_output><empty_output><jupyter_text>Correlation Matrix<jupyter_code>corr = data_frame.corr()
plt.subplots(figsize=(15,10))
sns.heatmap(corr, xticklabels=corr.columns, yticklabels=corr.columns, annot=True, cmap=sns.diverging_palette(220, 20, as_cmap=True))<jupyter_output><empty_output><jupyter_text>Convert to a Classification Problem> The objective is to compare the effectiveness of different classification techniques, so we needed to change the output variable to a binary output. For this problem, we have defined a bottle of wine as ‘1’ if it had a quality score of 7 or higher, and if it had a score of less than 7, it was deemed ‘0’.<jupyter_code>reviews = []
for i in data_frame['quality']:
    if i >= 3 and i <= 6:
        reviews.append('0')
    else:
        reviews.append('1')
data_frame['Reviews'] = reviews
data_frame.head() #shows the top 5 rows of dataset
X1= data_frame.drop('quality',1)
X=X1.drop("Reviews",1)
y=data_frame['Reviews']<jupyter_output><empty_output><jupyter_text>Proportion of Good(1) vs Bad Wines(0)>We wanted to make sure that there was a reasonable number of good quality wines. Based on the results below, it seemed like a fair enough number. In some applications, resampling may be required if the data was extremely imbalanced, but we assumed that it was okay for this purpose.<jupyter_code>plt.figure(figsize=(10,6))
sns.countplot(data_frame["Reviews"],palette="muted")
data_frame["Reviews"].value_counts()<jupyter_output><empty_output><jupyter_text>Preparing Data for Modelling Standardizing Feature Variables>To prepare the data for modeling the first thing that we should do is standardize the data. Standardizing the data means that it will transform the data so that its distribution will have a mean of 0 and a standard deviation of 1. It’s important to standardize your data in order to equalize the range of the data.For example, imagine a dataset with two input features: height in millimeters and weight in pounds. Because the values of ‘height’ are much higher due to its measurement, a greater emphasis will automatically be placed on height than weight, creating a bias.<jupyter_code># Normalize feature variables
from sklearn.preprocessing import StandardScaler
X_features = X
X = StandardScaler().fit_transform(X)
pd.DataFrame(X).head()<jupyter_output><empty_output><jupyter_text>Split the data >25% of the data was chosen as test data and the rest 75% was chosen as training data. Werandomly selected rows for dividing the data. We chose the partition percentage in a randommanner to avoid any bias.<jupyter_code># Splitting the data
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.25, random_state=0)<jupyter_output><empty_output><jupyter_text>Dimensionality Reduction>Principal Component Analysis(PCA) is one of the most popular linear dimension reduction. Sometimes, it is used alone and sometimes as a starting solution for other dimension reduction methods.>In this data set we had 11 features so it would be impossible to plot decision boundaries of classifiers.Hence we have used this to reduce the dimensions to only 2.>It reduce the dimensionality of a dataset, while preserving as much ‘variability’ (i.e. statistical information) as possible.<jupyter_code>from sklearn.decomposition import PCA
pca = PCA(n_components=2)
X_train = pca.fit_transform(X_train)
X_test = pca.transform(X_test)
explained_variance = pca.explained_variance_ratio_<jupyter_output><empty_output><jupyter_text>MODELLINGFor this project, we wanted to compare four different machine learning models:Random Forest ClassificationDecision TreesSVMK-Nearest Neighbours  Random Forest Classification<jupyter_code>from sklearn.metrics import classification_report,confusion_matrix
from sklearn.ensemble import RandomForestClassifier
model1 = RandomForestClassifier(random_state=1)
model1.fit(X_train, y_train)
y_pred1 = model1.predict(X_test)
print(classification_report(y_test, y_pred1))
print(confusion_matrix(y_test, y_pred1))
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test,y_pred1)
print('accuracy of random forest classifier:',accuracy)
from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 2, stop = X_set[:, 0].max() + 2, step = 0.005),
                     np.arange(start = X_set[:, 1].min() - 2, stop = X_set[:, 1].max() + 2, step = 0.005))
plt.contourf(X1, X2, model1.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.5, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Random Forest (Training set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()

from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 2, stop = X_set[:, 0].max() + 2, step = 0.005),
                     np.arange(start = X_set[:, 1].min() - 2, stop = X_set[:, 1].max() + 2, step = 0.005))
plt.contourf(X1, X2, model1.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.5, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Random Forest (Test set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()<jupyter_output>*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.
*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.<jupyter_text>Observations:    Precision rate of class 1 is less than 50 percent it is because our data set has less values of class 1 as compared to class 0.    We can see the decision boundaries of classifiers , there is no over fitting of boundaries.     Accuracy : 91%  Decision Tree Classification<jupyter_code>from sklearn.metrics import classification_report
from sklearn.tree import DecisionTreeClassifier
model = DecisionTreeClassifier(random_state=1)
model.fit(X_train, y_train)
y_pred= model.predict(X_test)
print(classification_report(y_test, y_pred))
print(confusion_matrix(y_test, y_pred))
from sklearn.metrics import accuracy_score
accuracy = accuracy_score(y_test,y_pred)
print('accuracy of decision tree classifier:',accuracy)
from sklearn import tree
text_representation = tree.export_text(model)
print(text_representation)<jupyter_output>|--- feature_1 <= -1.38
|   |--- feature_1 <= -1.82
|   |   |--- feature_0 <= 2.45
|   |   |   |--- feature_0 <= 2.14
|   |   |   |   |--- feature_1 <= -2.67
|   |   |   |   |   |--- feature_0 <= 0.75
|   |   |   |   |   |   |--- feature_1 <= -2.78
|   |   |   |   |   |   |   |--- class: 1
|   |   |   |   |   |   |--- feature_1 >  -2.78
|   |   |   |   |   |   |   |--- feature_1 <= -2.72
|   |   |   |   |   |   |   |   |--- class: 0
|   |   |   |   |   |   |   |--- feature_1 >  -2.72
|   |   |   |   |   |   |   |   |--- class: 1
|   |   |   |   |   |--- feature_0 >  0.75
|   |   |   |   |   |   |--- class: 0
|   |   |   |   |--- feature_1 >  -2.67
|   |   |   |   |   |--- feature_1 <= -2.43
|   |   |   |   |   |   |--- feature_1 <= -2.58
|   |   |   |   |   |   |   |--- feature_1 <= -2.59
|   |   |   |   |   |   |   |   |--- class: 0
|   |   |   |   |   |   |   |--- feature_1 >  -2.59
|   |   |   |   |   |   |   |   |--- class: 1
|   |   |   |   |   |   |--- feature_1 >  -2.58
|   |   [...]<jupyter_text>Visualisation of classifier on test and training set<jupyter_code>from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 2, stop = X_set[:, 0].max() + 2, step = 0.005),
                     np.arange(start = X_set[:, 1].min() - 2, stop = X_set[:, 1].max() + 2, step = 0.005))
plt.contourf(X1, X2, model.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.5, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Decision Tree (Training set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()
from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
                     np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, model.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('Decision Tree (Test set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()<jupyter_output>*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.
*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.<jupyter_text>Observations:     Accuracy is 88.5%.    We faced the problem of overfitting in decision tree classification. A decision tree can be easily        over-fitted generating too many branches.      SVM<jupyter_code>from sklearn.svm import SVC
svc = SVC()
svc.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Hyper parameter tunning for SVM>SVM also has some hyper-parameters (like what C or gamma values to use) and finding optimal hyper-parameter is a very hard task to solve. But it can be found by just trying all combinations and see what parameters work best. The main idea behind it is to create a grid of hyper-parameters and just try all of their combinations (hence, this method is called Gridsearch).Scikit-learn has this functionality built-in with GridSearchCV.<jupyter_code>from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV 
#Finding best parameters for our SVC model
param = {
    'C': [0.1,0.8,0.9,1,1.1,1.2,1.3,1.4],
    'kernel':['linear', 'rbf'],
    'gamma' :[0.1,0.8,0.9,1,1.1,1.2,1.3,1.4]
}
grid_svc = GridSearchCV(svc, param_grid=param, scoring='accuracy', cv=10)
grid_svc.fit(X_train, y_train)

#Best parameters for our svc model
grid_svc.best_params_
#fitting best parameters
svc = SVC(C =0.8, gamma = 1.4, kernel= 'rbf')
svc.fit(X_train, y_train)
pred_svc = svc.predict(X_test)
print(classification_report(y_test, pred_svc))
print(confusion_matrix(y_test, pred_svc))
accuracy = accuracy_score(y_test,pred_svc)
print('accuracy of svm classifier:',accuracy)<jupyter_output>precision    recall  f1-score   support

           0       0.90      0.97      0.93       355
           1       0.39      0.16      0.22        45

    accuracy                           0.88       400
   macro avg       0.64      0.56      0.58       400
weighted avg       0.84      0.88      0.85       400

[[344  11]
 [ 38   7]]
accuracy of svm classifier: 0.8775<jupyter_text>Visualisation of classifier on test and training set<jupyter_code>from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 2, stop = X_set[:, 0].max() + 2, step = 0.005),
                     np.arange(start = X_set[:, 1].min() - 2, stop = X_set[:, 1].max() + 2, step = 0.005))
plt.contourf(X1, X2, svc.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.5, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('SVC (Training set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()
from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 2, stop = X_set[:, 0].max() + 2, step = 0.005),
                     np.arange(start = X_set[:, 1].min() - 2, stop = X_set[:, 1].max() + 2, step = 0.005))
plt.contourf(X1, X2, svc.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('svc (Test set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()<jupyter_output>*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.
*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.<jupyter_text>Observations:    Accuracy 87.75%    In classification report as well as in confusion matrix we can observe that the recall percent of class 1 is only 16 %, which implies it is not a very well trained model.     The poor recall rate of class 1 can even be seen in the graph   KNN  Hyper parameter tunning for KNN<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
leaf_size = list(range(1,50))
n_neighbors = list(range(1,30))
p=[1,2]
#Convert to dictionary
hyperparameters = dict(leaf_size=leaf_size, n_neighbors=n_neighbors, p=p)
#Create new KNN object
knn_2 = KNeighborsClassifier()
#Use GridSearch
clf= GridSearchCV(knn_2, hyperparameters, cv=10)
#Fit the model
best_model = clf.fit(X_train,y_train)
#Print The value of best Hyperparameters
print('Best leaf_size:', best_model.best_estimator_.get_params()['leaf_size'])
print('Best p:', best_model.best_estimator_.get_params()['p'])
print('Best n_neighbors:', best_model.best_estimator_.get_params()['n_neighbors'])
# Fitting K-NN to the Training set
classifier = KNeighborsClassifier(leaf_size=1,n_neighbors = 19, metric = 'minkowski', p = 2)
classifier.fit(X_train, y_train)

# Predicting the Test set results
y_predknn = classifier.predict(X_test)
print(classification_report(y_test, y_predknn))
print(confusion_matrix(y_test, y_predknn))
accuracy = accuracy_score(y_test,y_predknn)
print('accuracy of KNN classifier:',accuracy)<jupyter_output>accuracy of KNN classifier: 0.88<jupyter_text>Visualisation of knn classifier on test and training set<jupyter_code>from matplotlib.colors import ListedColormap
X_set, y_set = X_train, y_train
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
                     np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('KNN (Training set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()
from matplotlib.colors import ListedColormap
X_set, y_set = X_test, y_test
X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:, 0].max() + 1, step = 0.01),
                     np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1, step = 0.01))
plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(), X2.ravel()]).T).reshape(X1.shape),
             alpha = 0.75, cmap = ListedColormap(('red', 'green')))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i, j in enumerate(np.unique(y_set)):
    plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],
                c = ListedColormap(('red', 'green'))(i), label = j)
plt.title('KNN (Test set)')
plt.xlabel('PC1')
plt.ylabel('PC2')
plt.legend()
plt.show()<jupyter_output>*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.
*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2-D array with a single row if you intend to specify the same RGB or RGBA value for all points.<jupyter_text>Observations:     Accuracy is 88%     In KNN also the recall percentage of class 1 is only 22 %, which implies it is not a very well trained model.      Conclusion  The data we have obtained is a classification related dataset. We got to know this by using various visualization techniques.We used various classification techniques and compared them. On evaluating the results we can conclude that Random Forest is the best classification technique according to the dataset. Random Forest is able to outperform the other two because of its ability to use many trees to classify the data. Thus random forest becomes more generalized than the other two and thus is performing better than the other two classifying algorithms.<jupyter_code>conclusion = pd.DataFrame({'models': ["SVC","Random Forest","Decision Trees","KNN"],
                           'accuracies': [accuracy_score(y_test,pred_svc),accuracy_score(y_test,y_pred1),accuracy_score(y_test,y_pred),accuracy_score(y_test,y_predknn)]})
conclusion<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13741117726137891, 'fraction_numerical': 0.026454772421739964, 'mean_word_length': 4.146775389177169, 'pattern_counts': {'":': 0, '<': 74, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '530076', 'license': 'MIT', 'n_tokens_mistral': 7694, 'n_tokens_neox': 7230, 'n_words': 2099, 'path': 'IDS_PROJECT.ipynb', 'repo_name': 'Akhil-Kashyap/IDS_Project', 'size': '20828'}