<jupyter_start><jupyter_text>10.2 Data Aggregation ![Screenshot_2020-07-22%20Python%20for%20Data%20Analysis%20-%20Python%20for%20Data%20Analysis,%202nd%20Edition%20pdf.png](attachment:Screenshot_2020-07-22%20Python%20for%20Data%20Analysis%20-%20Python%20for%20Data%20Analysis,%202nd%20Edition%20pdf.png)<jupyter_code>import pandas as pd
import numpy as np

df = pd.DataFrame({'key1' : ['a', 'a', 'b', 'b', 'a'],
                   'key2' : ['one', 'two', 'one', 'two', 'one'],
                   'data1' : np.random.randn(5),
                   'data2' : np.random.randn(5)})
df
grouped = df.groupby('key1')<jupyter_output><empty_output><jupyter_text>quantile computes sample quantiles of a Series or a DataFrame’s columns.<jupyter_code>grouped['data1'].quantile(0.9)<jupyter_output><empty_output><jupyter_text>To use your own aggregation functions, pass any function that aggregates an array tothe aggregate or agg method<jupyter_code>def peak_to_peak(arr):
    return arr.max() - arr.min()
grouped.agg(peak_to_peak)
grouped.describe()<jupyter_output><empty_output><jupyter_text>Column-Wise and Multiple Function Application<jupyter_code>tips = pd.read_csv(r'D:\tips.csv')
tips['tip_pct'] = tips['tip'] / tips['total_bill']
tips.head()<jupyter_output><empty_output><jupyter_text>aggregating a Series or all of the columns of a DataFrame is amatter of using aggregate with the desired function or calling a method like mean orstd. However, you may want to aggregate using a different function depending on thecolumn,  or  multiple  functions  at  once.<jupyter_code>grouped = tips.groupby(['day', 'smoker'])
grouped_pct = grouped['tip_pct']
grouped_pct.agg('mean')
grouped_pct.agg(['mean', 'std', peak_to_peak])<jupyter_output><empty_output><jupyter_text>You  don’t  need  to  accept  the  names  that  GroupBy  gives  to  the  columns;  notably,lambda  functions  have  the  name  '',  which  makes  them  hard  to  identify(you  can  see  for  yourself  by  looking  at  a  function’s  __name__  attribute).  Thus,  if  youpass a list of (name, function) tuples, the first element of each tuple will be used asthe  DataFrame  column  names<jupyter_code>grouped_pct.agg([('foo', 'mean'), ('bar', np.std)])
functions = ['count', 'mean', 'max']
result = grouped[['tip_pct', 'total_bill']].agg(functions)
result<jupyter_output><empty_output><jupyter_text>As  you  can  see,  the  resulting  DataFrame  has  hierarchical  columns,  the  same  as  youwould  get  aggregating  each  column  separately  and  using  concat  to  glue  the  resultstogether using the column names as the keys argument<jupyter_code>result['tip_pct']
ftuples = [('Durchschnitt', 'mean'), ('Abweichung', np.var)]
grouped[['tip_pct', 'total_bill']].agg(ftuples)<jupyter_output><empty_output><jupyter_text>Now,  suppose  you  wanted  to  apply  potentially  different  functions  to  one  or  more  ofthe columns. To do this, pass a dict to agg that contains a mapping of column namesto any of the function specifications listed so far<jupyter_code>grouped.agg({'tip' : np.max, 'size' : 'sum'})
grouped.agg({'tip_pct' : ['min', 'max', 'mean', 'std'],
             'size' : 'sum'})<jupyter_output><empty_output><jupyter_text>Returning Aggregated Data Without Row Indexes In  all  of  the  examples  up  until  now,  the  aggregated  data  comes  back  with  an  index,potentially  hierarchical,  composed  from  the  unique  group  key  combinations.  Sincethis  isn’t  always  desirable,  you  can  disable  this  behavior  in  most  cases  by  passingas_index=False to groupby<jupyter_code>tips.groupby(['day', 'smoker'], as_index=False).mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11851851851851852, 'fraction_numerical': 0.02167352537722908, 'mean_word_length': 5.190152801358234, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '637048', 'license': 'MIT', 'n_tokens_mistral': 1335, 'n_tokens_neox': 1276, 'n_words': 326, 'path': 'Python-For-Data-Analysis/Chapter 10 Data Aggregation/10.2 Data Aggregation.ipynb', 'repo_name': 'insomniac-klutz/Data-Science-Referentica', 'size': '3645'}