ETS-DACP has the highest loss for open domain Pile loss. However, we did not observe any degradation of performance on open domain tasks with ETS-DACP. Surprisingly, there is a tight correlation between losses of ETS-DACP-ent and ETS-DACP-ppl, while ETS-DACP-ppl performs consistently and considerably worse than ETS-DACP-ent on our tasks. These observations suggest that there is no good correlation between actual our task performance and loss curves.