<jupyter_start><jupyter_text>Groupby_agg : Shortcut for assigning a groupby-transform to a new column.  Background This notebook serves to show how to use the `groupby_agg` method from pyjanitor's general functions submodule.The `groupby_agg` method allows us to add the result of an aggregation from a grouping, as a new column, back to the dataframe.Currently in pandas, to append a column back to a dataframe, you do it in three steps:1. Groupby a column or columns2. Apply the `transform` method with an aggregate function on the grouping, and finally3. Assign the result of the transform to a new column in the dataframe.In pseudo-code, this might look something like:```pythondf = df.assign(    new_column_name=df.groupby(...)[...].transform(...))```The `groupby_agg` method allows you to achieve the same result in a single function call and with sensible arguments. The example below illustrates the use of this function.<jupyter_code># load modules
import pandas as pd
import numpy as np
import janitor<jupyter_output><empty_output><jupyter_text>Examples  Basic example We start off with a simple example.Given a `df` as defined below, we wish to use `groupby_agg` to find the average price for each item, and join the results back to the original dataframe.<jupyter_code>df = pd.DataFrame(
    {
        "item": ["shoe", "shoe", "bag", "shoe", "bag"],
        "MRP": [220, 450, 320, 200, 305],
        "number_sold": [100, 40, 56, 38, 25],
    }
)
df<jupyter_output><empty_output><jupyter_text>Note that the output of `groupby_agg` contains the same number of rows as the input dataframe, i.e., the operation here is a groupby + transform.Here, `by` is the name(s) of the column(s) being grouped over. `agg` is the aggregate function (e.g. sum, mean, count...), which is beinng applied to the data in the column specified by `agg_column_name`.Finally, `new_column_name` is the name of the newly-added column containing the transformed values.<jupyter_code>df = df.groupby_agg(
    by="item",
    agg="mean",
    agg_column_name="MRP",
    new_column_name="Avg_MRP",
)
df<jupyter_output><empty_output><jupyter_text>Specifying multiple columns to group over The basic example shown above specified a single column in `by` to group over.Grouping over multiple columns is also supported in general, since `groupby_agg` is just using the standard pandas `DataFrame.groupby` method under the hood.An example is shown below:<jupyter_code>df = pd.DataFrame(
    {
        "date": pd.date_range("2021-01-12", periods=5, freq="W"),
        "item": ["sneaker", "boots", "sneaker", "bag", "bag"],
        "MRP": [230, 450, 300, 200, 305],
    }
)
df
df = df.groupby_agg(
    by=["item", df["date"].dt.month],
    agg="mean",
    agg_column_name="MRP",
    new_column_name="Avg_MRP_by_item_month",
)
df<jupyter_output><empty_output><jupyter_text>The `dropna` parameter If the column(s) being grouped over (`by`) contains null values, you can include the null values as its own individual group, by passing `False` to `dropna`. Otherwise, the default behaviour is to `dropna=True`, in which case, the corresponding transformed values (in `new_column_name`) will be left as NaN.This feature was introduced in Pandas 1.1.You may read more about this parameter in the [Pandas user guide](https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.htmlid2).<jupyter_code>df = pd.DataFrame(
    {
        "name": ("black", "black", "black", "red", "red"),
        "type": ("chair", "chair", "sofa", "sofa", "plate"),
        "num": (4, 5, 12, 4, 3),
        "nulls": (1, 1, np.nan, np.nan, 3),
    }
)
df<jupyter_output><empty_output><jupyter_text>Let's get the value counts of the values in the `nulls` column.Compare the two outputs from the following cell when `dropna` is set to True and False respectively:<jupyter_code>print("With dropna=True (default)")
filtered_df = df.groupby_agg(
    by=["nulls"],
    agg="size",
    agg_column_name="type",
    new_column_name="counter",
    dropna=True,
)
display(filtered_df)

print("With dropna=False")
filtered_df = df.groupby_agg(
    by=["nulls"],
    agg="size",
    agg_column_name="type",
    new_column_name="counter",
    dropna=False,
)
display(filtered_df)<jupyter_output><empty_output><jupyter_text>Method chaining The `groupby_agg` method can be extended for different purposes. One of these is groupwise filtering, where only groups that meet a condition are retained.Let's explore this with an example, reusing one of the small dataframe from before:<jupyter_code>df = pd.DataFrame(
    {
        "name": ("black", "black", "black", "red", "red"),
        "type": ("chair", "chair", "sofa", "sofa", "plate"),
        "num": (4, 5, 12, 4, 3),
        "nulls": (1, 1, np.nan, np.nan, 3),
    }
)

filtered_df = df.groupby_agg(
    by=["name", "type"],
    agg="size",
    agg_column_name="type",
    new_column_name="counter",
).query("counter > 1")
filtered_df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1230800323362975, 'fraction_numerical': 0.015157639450282943, 'mean_word_length': 4.505005561735262, 'pattern_counts': {'":': 14, '<': 29, '<?xml version=': 0, '>': 30, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '529588', 'license': 'MIT', 'n_tokens_mistral': 1674, 'n_tokens_neox': 1572, 'n_words': 554, 'path': 'examples/notebooks/groupby_agg.ipynb', 'repo_name': 'aliavni/pyjanitor', 'size': '4948'}