<jupyter_start><jupyter_text>Using Data Science to Understand What Makes Wine Taste Good  Section 1: Data ExplorationIn this section, we'll do some exploratory analysis to understand the nature of our data and the underlying distribution.  First, import some necessary libraries.  Click the below cell block and run it.<jupyter_code># Import libraries necessary for this project
import numpy as np
import pandas as pd
import seaborn as sns
import warnings 


from time import time
from IPython.display import display # Allows the use of display() for displaying DataFrames
from pathlib import Path

import matplotlib.pyplot as plt

# Pretty display for notebooks
%matplotlib inline
warnings.filterwarnings('ignore')
pd.set_option('precision', 3)<jupyter_output><empty_output><jupyter_text>Next, we'll load the dataset for white wines, and display the first 5 columns. Run the below cell block<jupyter_code># Load the Red Wines dataset
data = pd.read_csv('../data/raw/winequality-white.csv')

# Display the first five records
data.head()<jupyter_output><empty_output><jupyter_text>Now, let's do some basic preliminary analysis of our data:  We'll begin by first seeing if our data has any missing information<jupyter_code># buscar nulos
data.isnull().sum()
sns.heatmap(data.isnull().T, xticklabels=False, cmap='gray')<jupyter_output><empty_output><jupyter_text>No se tienen valores nulos  Examine the features in the data-set and their data types<jupyter_code># información de tipos de variables
data.info()
# analitica descriptiva básica
data.describe()<jupyter_output><empty_output><jupyter_text>Section 2: Exploring Relationships between features<jupyter_code># Explorar las relaciones entre variables
# Matriz de correlaciones
data.corr()

# sugerencia 2: sns.heatmap()
fig = plt.figure(figsize=(10, 5))
sns.heatmap(data.corr(), linewidths=0.1, cmap='gray', annot=True, fmt='.2f' )
#Create a new dataframe containing only pH and fixed acidity columns to visualize their co-relations
fixedAcidity_pH = data[['pH', 'fixed acidity']]
fixedAcidity_pH.head()
sns.jointplot('fixed acidity', 'pH', data=fixedAcidity_pH, 
              kind='reg', color='black', marker='+',
              joint_kws = {'scatter_kws':dict(alpha=0.15)})<jupyter_output><empty_output><jupyter_text>Citric_acid and fixed acidity<jupyter_code>data.columns
fixedAcidity_citric = data[['citric acid', 'fixed acidity']]
sns.jointplot('fixed acidity', 'citric acid', data=fixedAcidity_citric, 
              kind='reg', color='black', marker='+',
              joint_kws = {'scatter_kws':dict(alpha=0.15,)})<jupyter_output><empty_output><jupyter_text>density and fixed acidity<jupyter_code>fixedAcidity_density = data[['density', 'fixed acidity']]
sns.jointplot('fixed acidity', 'density', data=fixedAcidity_density, 
              kind='reg', color='black', marker='+',
              joint_kws = {'scatter_kws':dict(alpha=0.15)})<jupyter_output><empty_output><jupyter_text>quality and volatile acidity<jupyter_code>quality_volatileacidity = data[['quality', 'volatile acidity']]
sns.jointplot('volatile acidity', 'quality', data=quality_volatileacidity, 
              kind='reg', color='black', marker='+',
              joint_kws = {'scatter_kws':dict(alpha=0.15)})<jupyter_output><empty_output><jupyter_text>alcohol and quality<jupyter_code>quality_alcohol = data[['alcohol', 'quality']]
sns.jointplot('alcohol', 'quality', data=quality_alcohol, 
              kind='reg', color='black', marker='+',
              joint_kws = {'scatter_kws':dict(alpha=0.15)})<jupyter_output><empty_output><jupyter_text>Barplot quality and volatile acidity<jupyter_code>#We can visualize relationships of discreet values better with a bar plot

fig, axs = plt.subplots(ncols=1,figsize=(10,6))
sns.barplot(x='quality', y='volatile acidity', data=quality_volatileacidity, ax=axs)
plt.title('quality VS volatile acidity')

plt.tight_layout()
plt.show()
plt.gcf().clear()<jupyter_output><empty_output><jupyter_text>Barplot quality and alcohol<jupyter_code>fig, axs = plt.subplots(ncols=1,figsize=(10,6))
sns.barplot(x='quality', y='alcohol', data=quality_alcohol, ax=axs)
plt.title('quality VS alcohol')

plt.tight_layout()
plt.show()
plt.gcf().clear()<jupyter_output><empty_output><jupyter_text>Outlier Detection:Detecting outliers in the data is extremely important in the data preprocessing step of any analysis. The presence of outliers can often skew results which take into consideration these data points. There are many "rules of thumb" for what constitutes an outlier in a dataset. Here, we will use [Tukey's Method for identfying outliers](http://datapigtechnologies.com/blog/index.php/highlighting-outliers-in-your-data-with-the-tukey-method/): An **outlier step** is calculated as **1.5** times the **interquartile range (IQR)**. A data point with a feature that is beyond an outlier step outside of the IQR for that feature is considered abnormal.In the code block below:* Assign the value of the 25th percentile for the given feature to Q1. Use np.percentile for this.* Assign the value of the 75th percentile for the given feature to Q3. Again, use np.percentile.* Assign the calculation of an outlier step for the given feature to step.* Optionally remove data points from the dataset by adding indices to the outliers list.**NOTE:** If you choose to remove any outliers, ensure that the sample data does not contain any of these points!Once you have performed this implementation, the dataset will be stored in the variable good_data.<jupyter_code># For each feature find the data points with extreme high or low values


for feature in data.keys():
    
    # TODO: Calculate Q1 (25th percentile of the data) for the given feature
    Q1 = np.percentile(data[feature], q=25)
    
    # TODO: Calculate Q3 (75th percentile of the data) for the given feature
    Q3 = np.percentile(data[feature], q=75)
    
    # TODO: Use the interquartile range to calculate an outlier step (1.5 times the interquartile range)
    interquartile_range = Q3 - Q1
    step = 1.5 * interquartile_range
    
    # Display the outliers
    print("Data points considered outliers for the feature '{}':".format(feature))
    display(data[~((data[feature] >= Q1 - step) & (data[feature] <= Q3 + step))])
    aux = ~((data[feature] >= Q1 - step) & (data[feature] <= Q3 + step))
    
    
# OPTIONAL: Select the indices for data points you wish to remove
outliers  = []
# Remove the outliers, if any were specified
good_data = data.drop(data.index[outliers]).reset_index(drop = True)<jupyter_output>Data points considered outliers for the feature 'fixed acidity':<jupyter_text>Part 2: Using Machine Learning to Predict the Quality of WinesFirst we are going to try using linear regression to see what happen<jupyter_code>#Displays the first 2 columns
data.iloc[:, :2]
# Split the data into features and target label
features = data.iloc[:, :-1].values
target = data.iloc[:, -1].values<jupyter_output><empty_output><jupyter_text>Next, shuffle and split our data-set into training and testing subsets:<jupyter_code>sns.countplot(data['quality'])
data['quality'].value_counts()
# Import train_test_split
from sklearn.model_selection import train_test_split

# Split the 'features' and 'income' data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(features, 
                                                    target, 
                                                    test_size=0.3, 
                                                    random_state=88)

# Show the results of the split
print(f' X_train.shape : {X_train.shape}, y_train.shape : {y_train.shape}')
print(f' X_test.shape : {X_test.shape}, y_test.shape : {y_test.shape}')<jupyter_output>X_train.shape : (3428, 11), y_train.shape : (3428,)
 X_test.shape : (1470, 11), y_test.shape : (1470,)<jupyter_text>Implementation: Initial Model Evaluation<jupyter_code>from sklearn.pipeline import Pipeline
from sklearn.model_selection import RepeatedStratifiedKFold, cross_validate
from sklearn.preprocessing import StandardScaler, RobustScaler, MinMaxScaler

from sklearn.metrics import mean_squared_error
from sklearn.metrics import accuracy_score, make_scorer
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.linear_model import Lasso, Ridge

def display_scores(scores, metric):
  for m in metric:
    name = 'test_'+m.lower()
    print('='*30)
    print('Metric:', m)
    print('Mean:', scores[name].mean())
    print('Standard deviation:', scores[name].std())

def evaluate_model(model, x, y):
    """ Evaluate the model using Stratified KFold """
    cv = RepeatedStratifiedKFold(n_splits=10, n_repeats=5)
    scoring = {'mse': make_scorer(mean_squared_error),
               'r2': make_scorer(r2_score)}
    scores = cross_validate(model, x, y, cv=cv, scoring=scoring, n_jobs=-1, verbose=0)
    return scores


from functools import partial

metrics = ['MSE', 'R2']
display_scores = partial(display_scores, metric=metrics)<jupyter_output><empty_output><jupyter_text>Linear Regression<jupyter_code>lr_pipline = Pipeline(steps=[('sc', StandardScaler()),
                             ('lr', LinearRegression())])

lr_pipline.fit(X_train, y_train)
scores = evaluate_model(lr_pipline, X_train, y_train)

display_scores(scores)
accuracy = lr_pipline.score(X_test,y_test)
print('='*30)
print("Accuracy: {}".format(accuracy))<jupyter_output>==============================
Metric: MSE
Mean: 0.5711031313466379
Standard deviation: 0.03243457915391726
==============================
Metric: R2
Mean: 0.2677575301105744
Standard deviation: 0.040730353515205334
==============================
Accuracy: 0.2805987328049383<jupyter_text>Lasso<jupyter_code>lr_pipline = Pipeline(steps=[('sc', StandardScaler()),
                             ('lr', Lasso())])

lr_pipline.fit(X_train, y_train)
scores = evaluate_model(lr_pipline, X_train, y_train)

display_scores(scores)
accuracy = lr_pipline.score(X_test,y_test)
print('='*30)
print("Accuracy: {}".format(accuracy))<jupyter_output>==============================
Metric: MSE
Mean: 0.7799967896668153
Standard deviation: 0.008336560628281513
==============================
Metric: R2
Mean: -8.406080708021512e-05
Standard deviation: 5.4518320024484535e-05
==============================
Accuracy: -8.54334577953253e-05<jupyter_text>Ridge<jupyter_code>lr_pipline = Pipeline(steps=[('sc', StandardScaler()),
                             ('lr', Ridge())])

lr_pipline.fit(X_train, y_train)
scores = evaluate_model(lr_pipline, X_train, y_train)

display_scores(scores)
accuracy = lr_pipline.score(X_test,y_test)
print('='*30)
print("Accuracy: {}".format(accuracy))
def models():
  models = [('Linnear Regression', Pipeline(steps=[('sc', RobustScaler()),
                                                   ('lr', LinearRegression())])),
            ('Lasso', Pipeline(steps=[('sc', RobustScaler()),
                             ('lr', Lasso())])),
            ('Ridge', Pipeline(steps=[('sc', RobustScaler()),
                             ('lr', Ridge())]))
            ]
  for name, model in models:
    yield name, model

results = []
names = []

for name, model in models():
    print('-'*20)
    print(name)
    scores = evaluate_model(model, X_train, y_train)
    results.append(scores['test_mse'])
    names.append(name)
    display_scores(scores)
    print()
    
plt.boxplot(results, labels=names, showmeans=True)
plt.title('MSE Scores')
plt.xlabel('Model')
plt.ylabel('MSE Score')
plt.grid()
plt.show()<jupyter_output>--------------------
Linnear Regression
==============================
Metric: MSE
Mean: 0.5701888372249606
Standard deviation: 0.04003789217697872
==============================
Metric: R2
Mean: 0.26881910090417327
Standard deviation: 0.05224208390028372

--------------------
Lasso
==============================
Metric: MSE
Mean: 0.7799967896668152
Standard deviation: 0.008336560628281463
==============================
Metric: R2
Mean: -8.406080708021957e-05
Standard deviation: 5.451832002443679e-05

--------------------
Ridge
==============================
Metric: MSE
Mean: 0.5704520241788906
Standard deviation: 0.03663325376957441
==============================
Metric: R2
Mean: 0.26870687301768803
Standard deviation: 0.04429979211458056
{'fraction_non_alphanumeric': 0.1444236861343514, 'fraction_numerical': 0.04134513849402973, 'mean_word_length': 4.821276595744681, 'pattern_counts': {'":': 0, '<': 70, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '269901', 'license': 'MIT', 'n_tokens_mistral': 4316, 'n_tokens_neox': 3816, 'n_words': 1025, 'path': 'notebooks/04_wine_prediction_challenge.ipynb', 'repo_name': 'christianvadillo/saturdaysai', 'size': '12311'}