<jupyter_start><jupyter_text>Homework 6, Part One: Lots and lots of questions about beer  Do your importing and your setup<jupyter_code>import pandas as pd

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Read in the file `craftcans.csv`, and look at the first first rows<jupyter_code>df = pd.read_csv("craftcans.csv")
df<jupyter_output><empty_output><jupyter_text>How many rows do you have in the data? What are the column types?<jupyter_code>df.shape
df.dtypes<jupyter_output><empty_output><jupyter_text>Checking out our alcohol  What are the top 10 producers of cans of beer?<jupyter_code>df.Brewery.value_counts().head(10)<jupyter_output><empty_output><jupyter_text>What is the most common ABV? (alcohol by volume)<jupyter_code>df.ABV.value_counts(ascending=False).head()<jupyter_output><empty_output><jupyter_text>Oh, weird, ABV isn't a number. Convert it to a number for me, please.It's going to take a few steps! First, let's just look at the ABV column by itself<jupyter_code>df.ABV<jupyter_output><empty_output><jupyter_text>Hm, `%` isn't part of  a number. Let's remove it.When you're confident you got it right, save the results back into the `ABV` column.- *Tip: In programming the easiest way to remove something is to *replacing it with nothing*.- *Tip: "nothing" might seem like `NaN` sinc we talked about it a lot in class, but in this case it isn't! It's just an empty string, like ""*- *Tip: `.replace` is used for replacing ENTIRE cells, while `.str.replace` is useful for replacing PARTS of cells (see my New York example)*<jupyter_code>df['ABV'] = df.ABV.str.replace('%', "")<jupyter_output><empty_output><jupyter_text>Now let's turn `ABV` into a numeric data typeSave the results back into the `ABV` column (again), and then check `df.dtypes` to make sure it worked.- *Tip: We used `.astype(int)` during class, but this has a decimal in it...*<jupyter_code>df['ABV'] = df.ABV.astype(float)
df.dtypes<jupyter_output><empty_output><jupyter_text>What's the ABV of the average beer look like? Show me in two different ways: one command to show the `median`/`mean`/etc, and secondly show me a chart<jupyter_code>df.groupby(by='Beer').ABV.median().head()
df.groupby(by='Beer').ABV.median().head().plot(kind='barh')<jupyter_output><empty_output><jupyter_text>We don't have ABV for all of the beers, how many are we missing them from?- *Tip: You can use `isnull()` or `notnull()` to see where a column is missing data.*- *Tip: You just want to count how many `True`s and `False`s there are.*- *Tip: It's a weird trick involving something we usually use to count things in a column*<jupyter_code>df.ABV.isnull().shape<jupyter_output><empty_output><jupyter_text>Looking at locationBrooklyn used to produce 80% of the country's beer! Let's see if it's still true.  What are the top 10 cities in the US for canned craft beer?<jupyter_code>df.Location.value_counts().head(10)<jupyter_output><empty_output><jupyter_text>List all of the beer from Brooklyn, NY<jupyter_code>df[df.Location == 'Brooklyn, NY']<jupyter_output><empty_output><jupyter_text>What brewery in Brooklyn puts out the most cans of beer?<jupyter_code>df.Brewery[df.Location == 'Brooklyn, NY'].value_counts()<jupyter_output><empty_output><jupyter_text>What are the five most popular styles of beer produced by Sixpoint?<jupyter_code>df[df.Brewery == 'Sixpoint Craft Ales'].Style.value_counts().head(5)<jupyter_output><empty_output><jupyter_text>List all of the breweries in New York state.- *Tip: We want to match *part* of the `Location` column, but not all of it.*- *Tip: Watch out for `NaN` values! You might be close, but you'll need to pass an extra parameter to make it work without an error.*<jupyter_code>df.Location.isna().sum()
df = df.dropna(subset=['Location'])
df[df.Location.str.contains(", NY")]<jupyter_output><empty_output><jupyter_text>Now *count* all of the breweries in New York state<jupyter_code>df.Brewery[df.Location.str.contains(", NY")].value_counts()<jupyter_output><empty_output><jupyter_text>Measuring International Bitterness Units Display all of the IPAsInclude American IPAs, Imperial IPAs, and anything else with "IPA in it."IPA stands for [India Pale Ale](https://www.bonappetit.com/story/ipa-beer-styles), and is probably the most popular kind of beer in the US for people who are drinking [craft beer](https://www.craftbeer.com/beer/what-is-craft-beer).<jupyter_code>df.Style.isna().sum()
df = df.dropna(subset=['Style'])
df[df.Style.str.contains(" IPA")]<jupyter_output><empty_output><jupyter_text>IPAs are usually pretty hoppy and bitter. IBU stands for [International Bitterness Unit](http://www.thebrewenthusiast.com/ibus/), and while a lot of places like to brag about having the most bitter beer (it's an American thing!), IBUs don't necessary *mean anything*.Let's look at how different beers have different IBU measurements.  Try to get the average IBU measurement across all beers<jupyter_code>#df.IBUs.median()<jupyter_output><empty_output><jupyter_text>Oh no, it doesn't work!It looks like some of those values *aren't numbers*. There are two ways to fix this:1. Do the `.replace` and `np.nan` thing we did in class. Then convert the column to a number. This is boring.2. When you're reading in your csv, there [is an option called `na_values`](http://pandas.pydata.org/pandas-docs/version/0.23/generated/pandas.read_csv.html). You can give it a list of **numbers or strings to count as `NaN`**. It's a lot easier than doing the `np.nan` thing, although you'll need to go add it up top and run all of your cells again.- *Tip: Make sure you're giving `na_values` a LIST, not just a string* Now try to get the average IBUs again<jupyter_code>import numpy as np

df['IBUs'] = df.IBUs.replace(["Does not apply"], np.nan)
df.IBUs.median()<jupyter_output><empty_output><jupyter_text>Draw the distribution of IBU measurements, but with *twenty* bins instead of the default of 10- *Tip: Every time I ask for a distribution, I'm looking for a histogram*- *Tip: Use the `?` to get all of the options for building a histogram*- *Tip: Make sure your `matplotlib` thing is set up right!*<jupyter_code>%matplotlib inline
df.IBUs.hist(bins=25)<jupyter_output><empty_output><jupyter_text>Hm, Interesting distribution. List all of the beers with IBUs above the 75th percentile- *Tip: There's a single that gives you the 25/50/75th percentile*- *Tip: You can just manually type the number when you list those beers*<jupyter_code>df.IBUs.isna().sum()
df = df.dropna(subset=['IBUs'])
df.IBUs.isna().sum()
df['IBUs'] = df.IBUs.astype(float)
a = df['IBUs']
df.IBUs.describe()
 df[df.IBUs >= 64.0]<jupyter_output><empty_output><jupyter_text>List all of the beers with IBUs below the 25th percentile<jupyter_code>df[df.IBUs <= 21.0]<jupyter_output><empty_output><jupyter_text>List the median IBUs of each type of beer. Graph it.Put the highest at the top, and the missing ones at the bottom.- Tip: Look at the options for `sort_values` to figure out the `NaN` thing. The `?` probably won't help you here.<jupyter_code>df.groupby(by='Style').IBUs.median().sort_values(ascending=True).plot(x='Style', y='IBUs', kind='barh')<jupyter_output><empty_output><jupyter_text>Hmmmm, it looks like they are generally different styles. What are the most common 5 styles of high-IBU beer vs. low-IBU beer?- *Tip: You'll want to think about it in three pieces - filtering to only find the specific beers beers, then finding out what the most common styles are, then getting the top 5.*- *Tip: You CANNOT do this in one command. It's going to be one command for the high and one for the low.*- *Tip: "High IBU" means higher than 75th percentile, "Low IBU" is under 25th percentile*<jupyter_code>df[df.IBUs >= 64.0].head(5).groupby(by='Beer').head()
 df[df.IBUs <= 21.0].head(5).groupby(by='Beer').head()<jupyter_output><empty_output><jupyter_text>Get the average IBU of "Witbier", "Hefeweizen" and "American Pale Wheat Ale" stylesI'm counting these as wheat beers. If you see any other wheat beer categories, feel free to include them. I want ONE measurement and ONE graph, not three separate ones. And 20 to 30 bins in the histogram, please.- *Tip: I hope that `isin` is in your toolbox*<jupyter_code>df[df.Style.isin(['Witbier', 'Hefeweizen', 'American Pale Wheat Ale'])].IBUs.astype(float).mean()<jupyter_output><empty_output><jupyter_text>Draw a histogram of the IBUs of those beers<jupyter_code>df[df.Style.isin(['Witbier', 'Hefeweizen', 'American Pale Wheat Ale'])].IBUs.hist()<jupyter_output><empty_output><jupyter_text>Get the average IBU of any style with "IPA" in it (also draw a histogram)<jupyter_code>df[df.Style.str.contains(" IPA")].IBUs.median()
df[df.Style.str.contains(" IPA")].IBUs.hist()<jupyter_output><empty_output><jupyter_text>Plot those two histograms on top of one anotherTo plot two plots on top of one another, you do two steps.1. First, you make a plot using `plot` or `hist`, and you save it into a variable called `ax`.2. You draw your second graph using `plot` or `hist`, and send `ax=ax` to it as a parameter.It would look something like this:```pythonax = df.plot(....)df.plot(ax=ax, ....)``` (...except totally different)<jupyter_code>ax = df[df.Style.isin(['Witbier', 'Hefeweizen', 'American Pale Wheat Ale'])].IBUs.hist()<jupyter_output><empty_output><jupyter_text>Compare the ABV of wheat beers vs. IPAs : their IBUs were really different, but how about their alcohol percentage?Wheat beers might include witbier, hefeweizen, American Pale Wheat Ale, and anything else you think is wheaty. IPAs probably have "IPA" in their name.<jupyter_code>df[df.Style.isin(['Witbier', 'Hefeweizen', 'American Pale Wheat Ale'])].mean()
df[df.Style.str.contains(" IPA")].mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11895949002673246, 'fraction_numerical': 0.005860579888957434, 'mean_word_length': 5.864502470007057, 'pattern_counts': {'":': 0, '<': 119, '<?xml version=': 0, '>': 119, 'https://': 2, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '29', 'dir': 'jupyter-structured-clean-dedup', 'id': '151090', 'license': 'MIT', 'n_tokens_mistral': 3376, 'n_tokens_neox': 3192, 'n_words': 1118, 'path': '06-homework/Dataset ONE - Beer cans.ipynb', 'repo_name': 'elenamlutz/foundations_homework', 'size': '9726'}