<jupyter_start><jupyter_text>Pandas Series Exercises Big Idea>Change is a process, not an event. - <NAME> (Design for How People Learn) Objectives By the end of the lesson and exercises, you will...- know how to create and read data into a pandas Series.- understand the vectorized functionality of the pandas Series.- be able to apply various methods and attributes to a pandas Series.- apply what you learned to solve previously seen problems in a different way.<jupyter_code>import numpy as np
import pandas as pd

import matplotlib.pyplot as plt

plt.rc('figure', figsize=(12, 6))<jupyter_output><empty_output><jupyter_text>1. Use pandas to create a Series from the following data:<jupyter_code>fruits_list = ["kiwi", "mango", "strawberry", "pineapple", "gala apple", "honeycrisp apple", "tomato", "watermelon", "honeydew", "kiwi", "kiwi", "kiwi", "mango", "blueberry", "blackberry", "gooseberry", "papaya"]<jupyter_output><empty_output><jupyter_text>___  a. Name the variable that holds the series fruits.<jupyter_code>fruits = pd.Series(fruits_list)<jupyter_output><empty_output><jupyter_text>___  b. Run `.describe()` method on the Series to see what describe returns for a series of strings.- `fruits` is a non-numerical Series, so we get back a Series with:     - the count of elements in the `fruits` Series.    - the count of the unique elements in the `fruits` Series.    - the most frequently occuring element in the `fruits` Series. (mode)    - the frequency of the most frequently occuring element.<jupyter_code>fruits.describe()<jupyter_output><empty_output><jupyter_text>___  c. Run the code necessary to produce only the unique fruit names.- The `.unique()` function returns the unique elements from a pandas Series in a NumPy array.<jupyter_code>fruits.unique()
print(f'Calling the `.unique()` method on a Series returns a {type(fruits.unique())}.')<jupyter_output>Calling the `.unique()` method on a Series returns a <class 'numpy.ndarray'>.<jupyter_text>-  What if I wanted to know the number of unique elements in my Series?<jupyter_code>fruits.nunique()<jupyter_output><empty_output><jupyter_text>___  d. Determine how many times each value occurs in the series.<jupyter_code>fruits.value_counts()
print(f'Calling the `.value_counts()` method on a Series returns a {type(fruits.value_counts())}.')<jupyter_output>Calling the `.value_counts()` method on a Series returns a <class 'pandas.core.series.Series'>.<jupyter_text>___  e. Determine the most frequently occurring fruit name from the series.- As always, there are several ways to acquire this information. I'm demonstrating a few here, but as is true in life, context is key. Your data needs will dictate the best method to use. Are you just exploring, do you need a single value, what data type do you want, etc.<jupyter_code>fruits.value_counts()
# If I know that there is only one row with the max value, this is cool.

fruits.value_counts().head(1)
# .idxmax() returns the index or row label with the max value. This returns only the first occurance of the max value.

fruits.value_counts().idxmax()
# What if there's more than one row with the max value?

fruits.value_counts().nlargest(n=1, keep='all')<jupyter_output><empty_output><jupyter_text>___  f. Determine the least frequently occurring fruit name from the series.<jupyter_code># I can look at my value counts.

fruits.value_counts()
# I have to set the n manually here.

fruits.value_counts().tail(11)
# Can I use a more programmatic approach?

fruits.value_counts().nsmallest(keep='all')<jupyter_output><empty_output><jupyter_text>___  g. Write the code to get the longest string from the fruits series.Again, I'll demonstrate a couple of ways to acquire this information. - If you want to learn more about using the python `.max()` function with a key, [here](https://www.programiz.com/python-programming/methods/built-in/max) is a cool tutorial.<jupyter_code>fruits
fruits.str.len()
fruits.str.len().max()
# Find the max length for the elements in the Series.

max_len = fruits.str.len().max()
# Create a boolean Series for element in the Series meeting my condition.

bool_fruits = fruits.str.len() == max_len
bool_fruits
# Use indexing with my boolean Series.

fruits[bool_fruits]<jupyter_output><empty_output><jupyter_text>**There is however this little gem.**<jupyter_code># This returns the string value.

max(fruits, key=len)<jupyter_output><empty_output><jupyter_text>___  h. Find the fruit(s) with 5 or more letters in the name.<jupyter_code># Create the boolean Series and pass it as a selector to my indexing operator.

fruits.str.len() > 4
fruits[fruits.str.len() > 4]<jupyter_output><empty_output><jupyter_text>___  i. Capitalize all the fruit strings in the series.- The `.str` attribute gives me access to vectorized string functions or methods, so I can perform the same manipulation on every element in my Series. *Remember, unless I reassign this Series, my original Series has not been changed.*<jupyter_code>fruits.str.capitalize()
# My original is unchanged.

fruits<jupyter_output><empty_output><jupyter_text>___  j. Count the letter "a" in all the fruits (use string vectorization).- The `.count()` string method will count the occurrence of a string or regex pattern in each string of a Series and return a new Series.<jupyter_code>fruits.str.count('a')<jupyter_output><empty_output><jupyter_text>___  k. Output the number of vowels in each and every fruit.- I can write a function to count the vowels in a single string and apply that function to every value in the Series.- I can pass a regex pattern to the `.count()` string method that looks for any of the characters in the square brackets in each string value in my Series. *We will do A LOT more with regular expressions in the future, but I like this solution because it uses the vectorized functionality of my string method with my pandas Series.*>`[..]` matches any single character in the square brackets and `[^..]` matches any single character not in the square brackets.<jupyter_code>vowels = list('aeiou')
vowels
def count_vowels(fruit):
    return len([let for let in fruit.lower() if let in vowels])
fruits.apply(count_vowels)<jupyter_output><empty_output><jupyter_text>___<jupyter_code>vowel_counts = fruits.str.lower().str.count(r'[aeiou]')
vowel_counts
# Want to see the name and vowel_counts? Here's one way to handle that...the index is pretty cool!

fruity_vowels = pd.DataFrame({'fruits': fruits, 'vowel_count': vowel_counts})
fruity_vowels<jupyter_output><empty_output><jupyter_text>___  l. Use the `.apply()` method and a lambda function to find the fruit(s) containing two or more "o" letters in the name.- Here I'm showing two steps, so you can see what's going on step by step.    1. I apply a lambda function that returns a boolean Series with values of `True` if the value in my `fruits` series meets the condition of having two or more 'o' letters and a `False` if the value in my `fruits` series does not meet the condition.        2. I want the actual values from my `fruits` Series that evaluate to `True`, so I'm going to pass my boolean Series as a selector to the indexing operator `[]` and get a subset of my original Series.<jupyter_code># Read this lambda function as, 'Return to me the fruit if the fruit meets my condition.'
# By passing this to the `apply()` function, it applies my lambda function to each value in my series `fruits`.

fruits.apply(lambda fruit: fruit.count('o') > 1)
# Read this boolean indexing as, 'Return to me the value in the `fruits` Series WHERE my value is True.'

fruits[fruits.apply(lambda fruit: fruit.count('o') > 1)]
# I can also solve this using a vectorized string method.

fruits.str.count('o')
# Return to me the value in the `fruits` Series WHERE my value is True.

fruits[fruits.str.count('o') >= 2]<jupyter_output><empty_output><jupyter_text>___  m. Write the code to get only the fruits containing "berry" in the name.- I'm going to use the same pattern I used above by creating a boolean Series for fruits that contain the string `berry` and using that boolean Series as a selector for a subset of data from my `fruits` Series.<jupyter_code>fruits.str.contains('berry')
fruits[fruits.str.contains('berry')]<jupyter_output><empty_output><jupyter_text>___  n. Write the code to get only the fruits containing "apple" in the name.- This is the same as above but with a different string.<jupyter_code>fruits[fruits.str.contains('apple')]<jupyter_output><empty_output><jupyter_text>___  o. Which fruit has the highest amount of vowels?- I can find the max value of vowel counts and create a bool Series as a filter for my fruits Series to return any and all values that have the highest number of vowels.<jupyter_code># Get the count of vowels like I did in a previous problem.

vowel_counts = fruits.str.count(r'[aeiou]')
vowel_counts
# Find the max count of vowels

max_vowels = (fruits.str.count(r'[aeiou]')).max()
max_vowels
# Create a boolean Series comparing the count of vowels in my original Series to my max count

bool_series = fruits.str.count(r'[aeiou]') == max_vowels
bool_series
# Use indexing to return rows that meet my condition.

fruits[bool_series]<jupyter_output><empty_output><jupyter_text>2. Use pandas to create a Series from the following data:<jupyter_code>nums_list = ['$796,459.41', '$278.60', '$482,571.67', '$4,503,915.98', '$2,121,418.3', '$1,260,813.3', '$87,231.01', '$1,509,175.45', '$4,138,548.00', '$2,848,913.80', '$594,715.39', '$4,789,988.17', '$4,513,644.5', '$3,191,059.97', '$1,758,712.24', '$4,338,283.54', '$4,738,303.38', '$2,791,759.67', '$769,681.94', '$452,650.23']
numbers = pd.Series(nums_list)
numbers<jupyter_output><empty_output><jupyter_text>___  a. What is the data type of the series?- I can see above that the dtype is `object`, but I'll also use the `.dtype` attribute for demonstation.<jupyter_code>numbers.dtype<jupyter_output><empty_output><jupyter_text>___  b. Use series operations to convert the series to a numeric data type.- This is the fun part of getting a Series with dollar signs and commas in it! Before I can just convert these values from objects to floats, I have to remove those symbols.<jupyter_code># Gotta love method chaining! Check the dtype at the bottom of the Series.

numbers.str.replace('$', '').str.replace(',', '').astype('float')
# Assign to a variable.

float_series = numbers.str.replace('$', '').str.replace(',', '').astype('float')
float_series
print(f'float_series contains {float_series.dtype} values.')<jupyter_output>float_series contains float64 values.<jupyter_text>___  c. What is the maximum value?<jupyter_code>float_series.max()<jupyter_output><empty_output><jupyter_text>**The minimum?**<jupyter_code>float_series.min()<jupyter_output><empty_output><jupyter_text>___  d. Bin the data into 4 equally sized intervals and show how many values fall into each bin.- I can do this with `.value_counts()` if I just want to bin values and get a frequency count for each bin.- If I want to customize bin labels and/or edges, I can use `pd.cut()` and chain `.value_counts()` to get a count.<jupyter_code># I can do this with value_counts().

float_series.value_counts(bins=4)
# Using `.cut()` to convert continuous values into categories. Now I have to group them.

pd.cut(float_series, 4)
# I group the observations above by bin/category and get a count of values in each bin/category.

pd.cut(float_series, 4).value_counts().sort_index()<jupyter_output><empty_output><jupyter_text>___  e. Plot a histogram of the data. Be sure to include a title and axis labels.<jupyter_code>float_series.value_counts(bins=4).sort_index(ascending=False).plot(kind='barh', 
                                                                   color='thistle',  
                                                                   width=1)

plt.title('4 Bins to Rule Them All')
plt.xlabel('Count')
plt.ylabel('US $')
plt.show()<jupyter_output><empty_output><jupyter_text>3. Use pandas to create a Series from the following exam scores:<jupyter_code>scores_list = [60, 86, 75, 62, 93, 71, 60, 83, 95, 78, 65, 72, 69, 81, 96, 80, 85, 92, 82, 78]
scores = pd.Series(scores_list)
# Validate that scores is a Series object.

type(scores)
# I have 20 values in my Series

scores.size<jupyter_output><empty_output><jupyter_text>___  a. What is the minimum exam score? The max, mean, median?<jupyter_code># I can see all the descriptive statistics about the numerical Series using the `.describe()` method.

scores.describe()<jupyter_output><empty_output><jupyter_text>**I can one-at-a-time it.**- Using the functions offers me parameters for more flexibility if I need it.<jupyter_code>scores.min()
scores.max()
scores.mean()
scores.median()<jupyter_output><empty_output><jupyter_text>___  b. Plot a histogram of the scores.<jupyter_code>scores.plot.hist(color='thistle')

plt.title('Distribution of Scores')
plt.xlabel('Scores')
plt.show()<jupyter_output><empty_output><jupyter_text>___  c. Convert each of the numbers above into a letter grade. For example, 86 should be a 'B' and 95 should be an 'A'.- A = 90-100- B = 80-89- C = 75-79- D = 70-74- F = < 70<jupyter_code># Define bin edges.
bin_edges = [0, 70, 75, 80, 90, 101]

# Create a list of bin labels; you should have one less than bin edges.
bin_labels = ['F', 'D', 'C', 'B', 'A']

# Use the .cut() function to create 5 bins as defined and labeled and create Series of value_counts sorted by index value.
pd.cut(scores, bins=bin_edges, labels=bin_labels).value_counts().sort_index()
# Plot and title the code above

pd.cut(scores, 
       bins=bin_edges, 
       labels=bin_labels,
       include_lowest=True).value_counts().sort_index().plot.barh(color='thistle',
                                                                  ec='black',
                                                                  width=.8)

plt.title('Letter Grades')
plt.xlabel('Number of Students')
plt.ylabel('Letter Grade')

plt.show()<jupyter_output><empty_output><jupyter_text>___  d. Write the code necessary to implement a curve. >ie. The score closest to 100 should be converted to a 100, and the same number of points added to that score should be given to every other score as well.<jupyter_code># Find the amount to curve my grades.

curve = 100 - scores.max()
curve
# Create curved_scores series.

curved_scores = scores + curve
curved_scores
# Define bin edges.
bin_edges = [0, 70, 75, 80, 90, 101]

# Create a list of bin labels; you should have one less than bin edges.
bin_labels = ['F', 'D', 'C', 'B', 'A']

# Check out counts
pd.cut(curved_scores, bins=bin_edges, labels=bin_labels).value_counts().sort_index()
# Create and plot bins
pd.cut(curved_scores, 
       bins=bin_edges, 
       labels=bin_labels,
       include_lowest=True).value_counts().sort_index().plot.barh(color='plum',
                                                                  ec='black',
                                                                  width=.8)

plt.title('Curved Letter Grades')
plt.xlabel('Number of Students')
plt.ylabel('Letter Grade')

plt.show()<jupyter_output><empty_output><jupyter_text>___<jupyter_code># Define bin edges.
bin_edges = [0, 70, 75, 80, 90, 101]

# Create a list of bin labels; you should have one less than bin edges.
bin_labels = ['F', 'D', 'C', 'B', 'A']

##################### first plot #################################

plt.subplot(2,1,1)

# code to plot letter grades

pd.cut(scores, 
       bins=bin_edges, 
       labels=bin_labels,
       include_lowest=True).value_counts().sort_index().plot.barh(color='thistle',
                                                                  ec='black',
                                                                  width=.8)

# Titles and labels for plot 1

plt.title('Letter Grades')
plt.xlabel('Number of Students')
plt.ylabel('Letter Grade')
plt.xlim(0, 8)

plt.show()

##################### second plot ################################

plt.subplot(2,1,2)

# code to plot curved letter grades

pd.cut(curved_scores, 
       bins=bin_edges, 
       labels=bin_labels).value_counts().sort_index().plot.barh(color='plum',
                                                                ec='black',
                                                                width=.8)

# Titles and labels for plot 2

plt.title('Curved Letter Grades')
plt.xlabel('Number of Students')
plt.ylabel('Letter Grade')

plt.show()<jupyter_output><empty_output><jupyter_text>4. Use pandas to create a Series from the following string:<jupyter_code>string = '<KEY>'
type(string)
# Add a space between each letter in the string, so I can split on that space.

' '.join(string)
# Create and peek at list of letters.

letters_list = ' '.join(string).split(' ')
letters_list[:5]
# Create and peek at Series of letters

letters = pd.Series(letters_list)
letters.head()
type(letters)<jupyter_output><empty_output><jupyter_text>a. What is the most frequently occuring letter? Least frequently occuring?<jupyter_code># I can sort my value counts and use a .head() and .tail().

letters.value_counts().sort_values(ascending=False)
# I can do this more programatically.

letters.value_counts().nlargest(n=1, keep='all')
letters.value_counts().nsmallest(n=1, keep='all')<jupyter_output><empty_output><jupyter_text>___  b. How many vowels are in the list?<jupyter_code># Count the vowels.

letters.str.lower().str.count(r'[aeiou]')
# Sum all the vowels in the Series.

letters.str.lower().str.count(r'[aeiou]').sum()<jupyter_output><empty_output><jupyter_text>___  c. How many consonants are in the list?<jupyter_code># Identify and total the consonants in the Series

letters.str.lower().str.count(r'[^aeiou]').sum()<jupyter_output><empty_output><jupyter_text>___  d. Create a series that has all of the same letters, but uppercased.<jupyter_code>letters.str.upper()<jupyter_output><empty_output><jupyter_text>___  e. Create a bar plot of the frequencies of the 6 most frequently occuring letters.<jupyter_code>letters.value_counts().head(7)
letters.value_counts().head(7).plot(kind='barh', 
                                    color='thistle',
                                    ec='black',
                                    width=.8)

plt.title('Top 6 Letters')

# I wanted the most frequent letter at the top, so I chose to invert the y-axis.
plt.gca().invert_yaxis()

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11391578718311392, 'fraction_numerical': 0.017789141551517788, 'mean_word_length': 4.147857742929151, 'pattern_counts': {'":': 0, '<': 163, '<?xml version=': 0, '>': 170, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 31, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '40', 'dir': 'jupyter-structured-clean-dedup', 'id': '240838', 'license': 'MIT', 'n_tokens_mistral': 6145, 'n_tokens_neox': 5720, 'n_words': 2045, 'path': 'easley-data-science-libraries/pandas_series_walkthrough.ipynb', 'repo_name': 'o0amandagomez0o/numpy-pandas-visualization-exercises', 'size': '18584'}