<jupyter_start><jupyter_text>Optimize Custom Aggregation FunctionIn this challenge, your goal is to find the fastest solution to the problem while only using the Pandas library. The ChallengeThe `college_pop` dataset contains the name, state and population of all higher-ed institutions in the US and its territories. For each state, find the percentage of the total state population made up by the 5 largest colleges of that state.  Import pandas and build DataFrame from csv file<jupyter_code>import pandas as pd
college = pd.read_csv('https://raw.githubusercontent.com/DunderData/Pandas-Challenges/master/data/college_pop.csv')
college.head()<jupyter_output><empty_output><jupyter_text>Basic info for the DataFrame<jupyter_code>college.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 7535 entries, 0 to 7534
Data columns (total 3 columns):
name     7535 non-null object
state    7535 non-null object
pop      6874 non-null float64
dtypes: float64(1), object(2)
memory usage: 176.7+ KB<jupyter_text>Basic summary stats<jupyter_code>college.describe()<jupyter_output><empty_output><jupyter_text>Remove null values for the 'pop' column<jupyter_code>college = college.dropna()<jupyter_output><empty_output><jupyter_text>Updated info for DataFrame<jupyter_code>college.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 6874 entries, 0 to 7163
Data columns (total 3 columns):
name     6874 non-null object
state    6874 non-null object
pop      6874 non-null float64
dtypes: float64(1), object(2)
memory usage: 214.8+ KB<jupyter_text>Data Wrangling<jupyter_code># I referenced "Python for Data Analysis" - Ch 10, pg 304
def top(df, n=5, column='pop'):
    return df.sort_values(by=column, ascending=False)[:n]
st_top_5 = college.groupby('state').apply(top)
st_top_5.rename(columns={'state': 'ST'}, inplace=True)
st_top_5_total = st_top_5.groupby('ST').sum()
decimal = st_top_5_total / college.groupby('state').sum()<jupyter_output><empty_output><jupyter_text>My solution:<jupyter_code>percentages = decimal.apply(lambda x: x * 100)
percentages.rename(columns={'pop': 'pct_of_total_st_college_pop_largest_5'}, inplace=True)
percentages<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10466696873584051, 'fraction_numerical': 0.03715450838241957, 'mean_word_length': 6.942446043165468, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '443866', 'license': 'MIT', 'n_tokens_mistral': 801, 'n_tokens_neox': 742, 'n_words': 193, 'path': 'dunderdata/challenges/my_progress/Challenge 001 - Optimize Custom Aggregation Function.ipynb', 'repo_name': 'djrgit/coursework', 'size': '2207'}