df0 = pd.merge(df[df.fold == 1], aux2, on='group').set_index('id')
df = df.set_index('id')
df.loc[df.fold == 1, 'group_average'] = df0.loc[:, ('group_average_y')]
df = df.reset_index()
Векторизация вычислений в pandas