Q:

Pandas - get latest 'n' columns value of a grouped data

I am new pandas and I have the following data of job scheduler program:
| Job Name   | Region        | Status | Timestamp            |
| some_job_1 | some_region_1 | DONE   | 2018-10-02T03:46:25Z |
| some_job_1 | some_region_2 | ERROR  | 2018-10-02T03:44:25Z |
| some_job_2 | some_region_1 | DONE   | 2018-10-01T03:46:25Z |
| some_job_1 | some_region_2 | ERROR  | 2018-11-02T03:44:25Z |

Now, I want the top 5 failing jobs in a time range and it's last 'n' execution status. Which should look something like this:
| Job Name   | Region        | DONE | ERROR | Last 5 runs                  |
| some_job_1 | some_region_1 | 3    | 12    | ERROR DONE ERROR ERROR ERROR |
| some_job_1 | some_region_2 | 2    | 9     | ERROR DONE ERROR ERROR ERROR |
| some_job_2 | some_region_1 | 2    | 8     | ERROR DONE ERROR ERROR ERROR |
| some_job_2 | some_region_2 | 5    | 7     | ERROR DONE ERROR ERROR ERROR |
| some_job_3 | some_region_2 | 5    | 7     | ERROR DONE ERROR ERROR ERROR |

And I have gotten as far as this:
| Job Name   | Region        | DONE | ERROR | 
| some_job_1 | some_region_1 | 3    | 12    | 
| some_job_1 | some_region_2 | 2    | 9     | 
| some_job_2 | some_region_1 | 2    | 8     | 
| some_job_2 | some_region_2 | 5    | 7     | 
| some_job_3 | some_region_2 | 5    | 7     | 

using:
data.groupby(['Job Name', 'Region']).Status.value_counts().unstack().fillna(0).sort_values('ERROR', ascending=False).head(5)

I tried using last() but with no success. Appreciate if someone can help me out!

A:

You can define the aggregations in a separate agg function and apply them to the groupby object.
def aggs(data):
    values = dict(data['Status'].value_counts())
    values['Last'] = list(data[['Status', 'Timestamp']].sort_values('Timestamp', ascending=False)['Status'].head(5))
    return pd.Series(values, index=['DONE', 'ERROR', 'Last'])

data.groupby(['Job Name', 'Region']).apply(aggs).fillna(0)

                            DONE    ERROR   Last
Job Name    Region          
some_job_1  some_region_1   1.0     0.0    [DONE]
            some_region_2   0.0     2.0    [ERROR, ERROR]
some_job_2  some_region_1   1.0     0.0    [DONE]

