<jupyter_start><jupyter_text>Supervised Machine LearningUses previously **labeled** data to train an algorithm (i.e the output is known). The algorithm learns if it is doing right by comparing the predicted vs the real output.Simplified steps:1. Data collection and processing.2. Division of training data in Train and Test sets.3. Use the train set to train the model.4. Predict an output for the test set and compare the predicted vs real results.5. Improve the model until we are satisfied with the performance on the test set Types of supervised ML models ClassificationClassification problems are characterized by having categorical output (i.e: active, inactive), so the model tries to predict to which class the input belongs. It can include several classes, is not limited to a binary classification. RegressionRegression problems are characterized by continuous variables, where the model tries to predict the exact value of the input (i.e: IC50 of a specific compound) Evaluation of supervised ML models Classification metricsWe obtain the following data to evaluate the model:- Y_pred: predictions on the test set- Y_real: real outcome of the test set**Accuracy:** number of correct predictions divided by the total number of predictions (TP/(lenY)). For example, if we have predicted correctly 5 out of 10 data points --> Accuracy = 50%**Precision**: identification of only real positives (with a 100% precision, a model does not classify any negatives as positive) --> TP/(TP+FP)**Recall**: identification of all positives (with a 100% recall, no positive is classified as negative, but some negatives might be included in the positives) --> TP / (TP+FN)**Confusion matrix**: plots the real vs the predicted values in a table, to easily obtain the FP, TP, TN, FN values. Regression metricsIn a regression task, we obtain as error the difference between the predicted value and the real value (i.e: IC50real=0.1, IC50pred = 0.5 --> error of 0.4).**Mean Absolute Error:** mean of the absolute values of errors. **Mean Squared Error:** mean of the square error. By squaring, larger errors are contributing more and therefore the model punishes them.**Root Mean Squared Error:** root of the mean of square error to simplify interpretation (by using MSE, we also square the units which makes them difficult to interpret).**R-square**: coefficient of determination, the amount of variance explained by the model (from 0 to 1, the closer to 1, the better our model is)  Sklearn PackageSklearn is the most popular package for machine learning in python. We are going to go over the basic features of sklearn to train a model that predicts the quality of the Portuguese Wines from *<NAME>, <NAME>, <NAME>, <NAME> and <NAME>. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009.*First, install the sklearn package by either :- `$pip install -U scikit-learn` on the command line- `!pip install -U scikit-learn` on a jupyter notebook cellAnd then, import on the notebook only the modules that we are going to use. The package is very large, so we will only import specific chunks of it. Run the code below to import the necessary modules to perform a basic classification problem:- `from sklearn.model_selection import train_test_split`- `from sklearn.linear_model import LogisticRegression`<jupyter_code>!pip install -U scikit-learn
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression<jupyter_output><empty_output><jupyter_text>Logistic regression on a classification task Data collectionLoad the wine dataset into a pandas dataframe, as we have been working with it in session 3 we already know its characteristics.<jupyter_code>import pandas as pd
df = pd.read_csv("../session3/winequality-red.csv")
#check the labeled output: 
import matplotlib.pyplot as plt

y = df.groupby("quality", as_index=False).nunique()
plt.bar(x = y["quality"], height=y["name"])
plt.xlabel("quality")
plt.ylabel("number of wines")<jupyter_output><empty_output><jupyter_text>We could use the discrete values as a multiclass classification problem, but, to simplify our first analysis, we will define a cutoff:- Wines of quality => 7 --> good (1)- Wines of quality  bad (0)To this end, we will add a classification (clf) column to our dataset, containing only 0 or 1<jupyter_code>#add a classification column to the dataframe:

clf = [] #we create an empty list where we will add the clf values

quality = df["quality"].tolist() #pass the quality column to a list 

#iterate over the quality list and append values to the clf list
for item in quality:
    if item < 7:
        clf.append(0)
    else:
        clf.append(1)

#add the clf list as a column in the dataframe:
df["clf"]=clf
#check how many values of each type we have
df["clf"].value_counts()
#assert that the classification was right
assert df["clf"].value_counts()[1] == (len(df[df["quality"]>=7]))

#the assert function is very useful as a checkpoint to understand if our data was correct<jupyter_output><empty_output><jupyter_text>Train test splitNow, we want to use the physicochemical properties of each wine to try and predict its quality (1 or 0). The dataset has already been cleaned for us, so we do not need to process it further (eliminate empty columns, deal with outliers etc).The next step is to divide the data into train and test. For this, we use the built-in function train_test_split in sklearn.First, we need to determine what are the X and Y of our model:X - Element to predictY - OutputOf course, the element to predict (x) is the wine, but, the algorithm needs at least one property of the wine to be able to predict its quality (the output, y). Let's start by identifying a property that correlates with the wine quality:<jupyter_code>#Check if there is a property with different mean for good vs bad wine

good = df[df["clf"]==1]
bad = df[df["clf"]==0]

print(good.mean(axis=0))
print(bad.mean(axis=0))

#Sulfur dioxide looks quite different between good and bad, so let's start here:

x = df[["total sulfur dioxide","free sulfur dioxide"]] #we use the total sulfur dioxide as the wine representation
y = df["clf"] #we use the classification 0 or 1 as prediction outcome
#let's call the sklearn function train_test_split using the x and y values we have defined above
#test_size indicates the % of the total data that will be kept to test the model prediction

X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
# to better understand what train and test split does, lets print the four outcomes one by one:
X_train.head()
y_train.head()
X_test.head()
y_test.head()<jupyter_output><empty_output><jupyter_text>Model TrainingWe have now created a train set with its associated outcome (0 or 1) and a test set with its associated outcome as well (1) and it is time to train a model.One of the simplest classification algorithms is based on a logistic regression, or sigmoid function. The sigmoid takes any value and outputs a value between 0 and 1, basically a probability that a point belongs to the 0 or the 1 class<jupyter_code>#we instanciate the model as a Logistic Regression from sklearn

mdl = LogisticRegression()

#and then we train (fit) the model using our training data
mdl.fit(X_train, y_train)
#we then predict the probability of the wines in the test set being 0 (first column) or 1 (second column)
pred = mdl.predict(X_test)
pred<jupyter_output><empty_output><jupyter_text>Model performanceWe have trained a Logistic Regression model using a train set, and predicted the outcome on a test set for which we have the actual values. Let's check how good our model did.We can use the classification report and the confusion matrix methods built-in in sklearn.<jupyter_code>from sklearn import metrics
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
print(classification_report(y_test,pred))
confusion_matrix(y_test, pred)
import seaborn as sns

cf_matrix = confusion_matrix(y_test, pred)

ax = sns.heatmap(cf_matrix, annot=True, cmap='Greens', cbar=False, annot_kws={"size": 16})
ax.set_title("Predicted vs Real Quality", fontsize=14)
ax.set_xlabel('Predicted', fontsize=14)
ax.set_ylabel('Real', fontsize=14);

## Ticket labels - List must be in alphabetical order
ax.xaxis.set_ticklabels(['Bad','Good'], fontsize=14)
ax.yaxis.set_ticklabels(['Bad','Good'], fontsize=14)<jupyter_output><empty_output><jupyter_text>So, our model cannot very accurately predict if a wine is good using only the Sulfur Dioxide content of each wine as input. Let's try to add all the variables we have as input:<jupyter_code>df.columns
x = df[['fixed acidity', 'volatile acidity', 'citric acid',
       'residual sugar', 'chlorides', 'free sulfur dioxide',
       'total sulfur dioxide', 'density', 'pH', 'sulphates', 'alcohol']]
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
X_train #X is a dataframe containing the relevant columns for prediction
mdl = LogisticRegression(max_iter=10000, verbose=3)
mdl.fit(X_train, y_train)
pred = mdl.predict(X_test)
cf_matrix = confusion_matrix(y_test, pred)

ax = sns.heatmap(cf_matrix, annot=True, cmap='Greens', cbar=False, annot_kws={"size": 16})
ax.set_title("Predicted vs Real Quality", fontsize=14)
ax.set_xlabel('Predicted', fontsize=14)
ax.set_ylabel('Real', fontsize=14);

## Ticket labels - List must be in alphabetical order
ax.xaxis.set_ticklabels(['Bad','Good'], fontsize=14)
ax.yaxis.set_ticklabels(['Bad','Good'], fontsize=14)<jupyter_output><empty_output><jupyter_text>ROC-AUC curveUsing the predict_proba function of sklearn, we can create a ROC curve to evaluate the model performance<jupyter_code>pred_prob = mdl.predict_proba(X_test)

fpr, tpr, _ = metrics.roc_curve(y_test,  pred_prob[:,1]) #we are interested in the probability of being 1 (good)
auc = metrics.roc_auc_score(y_test, pred_prob[:,1])
plt.plot(fpr,tpr,label="auc="+str(auc))
plt.legend(loc=4)
plt.show()<jupyter_output><empty_output><jupyter_text>Training the model with a better description of the wines has improved the performance (of the 24 predicted good wines, 7 were actually good), but we might want to refine it further. In this situation, there is a number of options to continue:- Increase the training sample- Increase the number of wine descriptors- Choose another algorithm  Regression taskWe have seen how to train a classification. We can use the same dataset to train a regression, if we consider the quality of the wine to be a continuous variable (from 0 to 10).First, we need to create a train/test split where the y's, this time, will be the `quality` column instead of the `clf` column.<jupyter_code>x = df[['fixed acidity', 'volatile acidity', 'citric acid',
       'residual sugar', 'chlorides', 'free sulfur dioxide',
       'total sulfur dioxide', 'density', 'pH', 'sulphates', 'alcohol']]

y=df["quality"]

X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)<jupyter_output><empty_output><jupyter_text>Now, we will import a sklearn model for Linear Regression, the simplest regression type.<jupyter_code>from sklearn.linear_model import LinearRegression
mdl = LinearRegression()
mdl.fit(X_train, y_train)
pred = mdl.predict(X_test)
pred<jupyter_output><empty_output><jupyter_text>Metrics for regressionIn this case, we will use the MAE, MSE and RMSE values. We can import the MAE and MSE formulas from sklearn and then calculate the RMSE and R<jupyter_code>mae = metrics.mean_absolute_error(y_test, pred)
mse = metrics.mean_squared_error(y_test, pred)
rmse = mse**(0.5)  
r2 = metrics.r2_score(y_test, pred)

print("MAE:"+ str(mae))
print("MSE:" + str(mse))
print("RMSE:" + str(rmse))
print("R2:" + str(r2))
plt.scatter(y_test, pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0845809016324114, 'fraction_numerical': 0.010572612704051425, 'mean_word_length': 5.46120218579235, 'pattern_counts': {'":': 2, '<': 51, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '594814', 'license': 'MIT', 'n_tokens_mistral': 3565, 'n_tokens_neox': 3351, 'n_words': 1534, 'path': 'session4/Python101_Session4.ipynb', 'repo_name': 'ersilia-os/python101', 'size': '11838'}