<jupyter_start><jupyter_text>Questions for these datasets:Q1: Is a certain type of wine (red or white) associated with higher quality?Q2: What level of acidity (pH value) receives the highest average rating?Acidity Levels:High: Lowest 25% of pH valuesModerately High: 25% – 50% of pH valuesMedium: 50% – 75% of pH valuesLow: 75% – max pH valueQ3: Do wines with higher alcoholic content receive better ratings?We shall consider two groups of wine samples:Low alcohol (samples with an alcohol content less than the median)High alcohol (samples with an alcohol content greater than or equal to the median)Q4: Do sweeter wines (more residual sugar) receive better ratings?Q5:How significant are other chemicals for wine quality rating?<jupyter_code>red_df = pd.read_csv('datasets/wine_quality/winequality-red.csv', sep=';')

white_df = pd.read_csv('datasets/wine_quality/winequality-white.csv', sep=';')
red_df.head()
white_df.head()
# check which columns have missing values, as well as the number of entries/features
red_df.info()
white_df.info()
# check for duplicates in the data
sum(red_df.duplicated())
# drop duplicates
red_df.drop_duplicates(inplace=True)
# confirm correction by rechecking for duplicates in the data
sum(red_df.duplicated())
sum(white_df.duplicated())
white_df.drop_duplicates(inplace=True)
sum(white_df.duplicated())
# just looking around
red_df.describe()
white_df.describe()
# color column to merge both datasets
red_df['color'] = 'red'
red_df
white_df['color'] = 'white'
white_df
wine_df = red_df.append(white_df, ignore_index=True, sort=True)
wine_df
# look around, some distributions
wine_df.hist(figsize=(8, 8));
pd.plotting.scatter_matrix(wine_df, figsize=(8, 8));<jupyter_output><empty_output><jupyter_text>Q1: Is a certain type of wine (red or white) associated with higher quality?<jupyter_code>colors = ['#800020', '#FFBF00']
color_means = wine_df.groupby('color')['quality'].mean()
color_means.plot(kind='bar', title='Average Wine Quality by Color', color=colors, alpha=.7)
plt.xlabel('Color', fontsize=18)
plt.ylabel('Quality', fontsize=18);
counts = wine_df.groupby(['quality', 'color']).count()
counts
#We’re missing a red wine value for a the 9 rating. Even though this number is a 0, we need it for our plot. Run the last two cells after running the cell below.

# get counts for each rating and color
color_counts = wine_df.groupby(['color', 'quality']).count()['pH']
color_counts

# get total counts for each color
color_totals = wine_df.groupby('color').count()['pH']
color_totals

# get proportions by dividing red rating counts by total # of red samples
red_proportions = color_counts['red'] / color_totals['red']
red_proportions

# get proportions by dividing white rating counts by total # of white samples
white_proportions = color_counts['white'] / color_totals['white']
white_proportions

red_proportions['9'] = 0
red_proportions

ind = np.arange(len(red_proportions)) # the x locations for the groups
width = 0.35 # the width of the bars

# plot bars
red_bars = plt.bar(ind, red_proportions, width, color=colors[0], alpha=.7, label='Red Wine')
white_bars = plt.bar(ind + width, white_proportions, width, color=colors[1], alpha=.7, label='White Wine')

# title and labels
plt.ylabel('Proportion')
plt.xlabel('Quality')
plt.title('Proportion by Wine Color and Quality')
locations = ind + width / 2 # xtick locations
labels = ['3', '4', '5', '6', '7', '8', '9'] # xtick labels
plt.xticks(locations, labels)

# legend
plt.legend();<jupyter_output><empty_output><jupyter_text>/Q1: Is a certain type of wine (red or white) associated with higher quality?Somewhat, as it seems  Q2: What level of acidity (pH value) receives the highest average rating?<jupyter_code>wine_df['pH'].describe() # View the min, 25%, 50%, 75%, max pH values
bin_edges = [2.72, 3.11, 3.21, 3.33, 4.01] # Fill in this list with five values you just found

bin_names = [ 'high', 'moderately high' , 'medium', 'low' ] # Name each acidity level category

# Creates acidity_levels column
wine_df['acidity levels'] = pd.cut(df['pH'], bin_edges, labels=bin_names)

# Checks for successful creation of this column
wine_df.head()

# Find the mean quality of each acidity level with groupby
wine_df.groupby('acidity levels').mean()['quality']<jupyter_output><empty_output><jupyter_text>/Q2: What level of acidity (pH value) receives the highest average rating?Medium-to-Low  Q3: Do wines with higher alcoholic content receive better ratings?<jupyter_code># get the median amount of alcohol content
wine_df['alcohol'].describe()
# select samples with alcohol content less than the median
low_alcohol = wine_df[wine_df['alcohol'] < 10.549241]

# select samples with alcohol content greater than or equal to the median
high_alcohol = wine_df[wine_df['alcohol'] >= 10.549241]

# ensure these queries included each sample exactly once
num_samples = wine_df.shape[0]
num_samples == low_alcohol['quality'].count() + high_alcohol['quality'].count() # should be True

# get mean quality rating for the low alcohol and high alcohol groups
bin_edges = [8,10.5,14.9]
bin_names = ['low', 'high']
wine_df['alcohol levels'] = pd.cut(wine_df['alcohol'], bin_edges, labels=bin_names)
wine_df.groupby('alcohol levels').mean()['quality']
# Use query to select each group and get its mean quality
median = wine_df['alcohol'].median()
low = wine_df['alcohol < {}'.format(median))
high = wine_df.query('alcohol >= {}'.format(median))

mean_quality_low = low['quality'].mean()
mean_quality_high = high['quality'].mean()

# Create a bar chart with proper labels
locations = [1, 2]
heights = [mean_quality_low, mean_quality_high]
labels = ['Low', 'High']
plt.bar(locations, heights, tick_label=labels)
plt.title('Average Quality Ratings by Alcohol Content')
plt.xlabel('Alcohol Content')
plt.ylabel('Average Quality Rating');<jupyter_output><empty_output><jupyter_text>/Q3: Do wines with higher alcoholic content receive better ratings?Kind of, but not drasticaly(on average)  Q4:Do sweeter wines receive higher ratings?<jupyter_code># Use query to select each group and get its mean quality
median = wine_df['residual sugar'].median()
lowsugar = wine_df.loc[wine_df['residual sugar'] < median]
highsugar = wine_df.loc[wine_df['residual sugar'] >= median]

num_samples = wine_df.shape[0]
num_samples == lowsugar['quality'].count() + highsugar['quality'].count() # should be True

mean_lowsugar = lowsugar['quality'].mean()
mean_highsugar = highsugar['quality'].mean()

# Create a bar chart with proper labels
locations = [1, 2]
heights = [mean_lowsugar, mean_highsugar]
labels = ['Low Sugar', 'High Sugar']
plt.bar(locations, heights, tick_label=labels)
plt.title ('Average Quality Ratings by Sugar Level')
plt.xlabel('Sugar Content')
plt.ylabel('Average Quality Rating');
mean_highsugar-mean_lowsugar<jupyter_output><empty_output><jupyter_text>/Q4:Do sweeter wines receive higher ratings?On average no  Q5:How significant are other chemicals for wine quality rating?<jupyter_code>red_df.groupby(['quality']).mean()
red_df.plot(x='quality', y='volatile acidity', kind='scatter');
red_df.plot(x='quality', y='alcohol', kind='scatter');
white_df.groupby(['quality']).mean()
white_df.plot(x='quality', y='volatile acidity', kind='scatter');
white_df.plot(x='quality', y='alcohol', kind='scatter');
#Some considerable variance of means appears to be observed only among these parameters
df = wine_df[['quality', 'citric acid', 'volatile acidity', 'chlorides', 'pH', 'alcohol', 'free sulfur dioxide', 'total sulfur dioxide']].copy() 
df.groupby(['quality']).mean()
#pd.plotting.scatter_matrix(df, figsize=(8, 8))
#df.info()

wine_df['linguistic quality estimate'] = '1mediocre'
wine_df.loc[wine_df['quality'] > 4, 'linguistic quality estimate'] = '2decent'
wine_df.loc[wine_df['quality'] > 7, 'linguistic quality estimate'] = '3good'

#ling_df = wine_df.groupby(['linguistic quality estimate'])
df_1 = wine_df[wine_df['linguistic quality estimate'] == '1mediocre']
df_2 = wine_df[wine_df['linguistic quality estimate'] == '2decent']
df_3 = wine_df[wine_df['linguistic quality estimate'] == '3good']

wine_df.groupby(['linguistic quality estimate']).mean()
#wine_df.plot(x='linguistic quality estimate', y='volatile acidity', kind='box');
#fig, ax = plt.subplots(figsize=(8, 6))
#ax.box(df_1['volatile acidity']);


plt.boxplot([df_1['volatile acidity'], df_2['volatile acidity'], df_3['volatile acidity']]);
plt.boxplot([df_1['chlorides'], df_2['chlorides'], df_3['chlorides']]);
plt.boxplot([df_1['citric acid'], df_2['citric acid'], df_3['citric acid']]);
plt.boxplot([df_1['free sulfur dioxide'], df_2['free sulfur dioxide'], df_3['free sulfur dioxide']]);
plt.boxplot([df_1['total sulfur dioxide'], df_2['total sulfur dioxide'], df_3['total sulfur dioxide']]);<jupyter_output><empty_output><jupyter_text>/Q5:How significant are other chemicals for wine quality rating?In general, the most significant chemicals for quality rating in these datasets appear to be volatile acidity, chlorides, citric acid, free sulfur dioxide and total sulfur dioxide.And, in general, the higher quality wines exhibit smaller content and smaller spread of the aforementioned characteristics  Conclusions:/Q1: Is a certain type of wine (red or white) associated with higher quality?Somewhat, as it seems/Q2: What level of acidity (pH value) receives the highest average rating?Medium-to-Low/Q3: Do wines with higher alcoholic content receive better ratings?Kind of, but not drasticaly(on average)/Q4:Do sweeter wines receive higher ratings?On average no/Q5:How significant are other chemicals for wine quality rating?In general, the most significant chemicals for quality rating in these datasets appear to be volatile acidity, chlorides, citric acid, free sulfur dioxide and total sulfur dioxide.And, in general, the higher quality wines exhibit smaller content and smaller spread of the aforementioned characteristicsSome final remarks:The distinctness of taste/flavour of wine is the result of complex combination of, sometimes unique, organic/non-organic chemical compounds, I don't think we can reliably estimate the relative significance of wine quality factors with these datasets, since they don't have features for such aromatic compounds. Some unfinished experiments:<jupyter_code>#wine_df.groupby().count()
#totals = wine_df.groupby(['linguistic quality estimate']).count()['volatile acidity']
#proportions = counts / totals
#proportion1 = 
fig, ax = plt.subplots(figsize=(8, 6))
#ax.plot(kind='bar', df_1['volatile acidity']/len(df_1.index), alpha=0.5, label='1mediocre')
#ax.plot(kind='bar', df_2['volatile acidity']/len(df_2.index), alpha=0.5, label='2decent')
#ax.plot(kind='bar', df_3['volatile acidity']/len(df_3.index), alpha=0.5, label='3good')
ax.hist(df_1['volatile acidity'], density=True, histtype='step', stacked=True);
ax.hist(df_2['volatile acidity'], density=True, histtype='step', stacked=True);
ax.hist(df_3['volatile acidity'], density=True, histtype='step', stacked=True);
#ax.scatter(df_2['volatile acidity'])
#ax.set_title('Dsitributions of linguistic ratings')
#ax.set_xlabel('volatile acidity')
#ax.set_ylabel('proportion')
#ax.legend(loc='upper right')
#plt.show()
wine_df.plot(x='quality', y='volatile acidity', kind='density', xlim=0);
wine_df.plot(x='quality', y='residual sugar', kind='density', xlim=0, ylim=0);
wine_df.groupby(pd.cut(wine_df['quality'], np.arange(0, 10, 2))).mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13260926688271876, 'fraction_numerical': 0.015590785670491372, 'mean_word_length': 6.901730103806228, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 34, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '665136', 'license': 'MIT', 'n_tokens_mistral': 3857, 'n_tokens_neox': 3556, 'n_words': 1114, 'path': 'End-to-End Data Analysis practice/Data Analysis of Wine Quality.ipynb', 'repo_name': 'ColdTeapot273K/data-stuff', 'size': '11417'}