These results clearly show that _not all data is equal for continual pre-training_. In fact, all the data used in efficient DACP methods (10%) is a subset of the data in DACP. Since DACP's (100%) performance is lower than ETS-DACP/ETA-DACP-ent, adding more data on top of highly similar or high entropy data actually hurts the performance. The difference in results between hard and soft sampling adds more evidence to this observation. While there is variability across tasks, on an average, adding examples from outside the top decile of metrics hurts the performance with the notable exception of ETS-DACP-com which is a combination of all three metrics. Hence, we should carefully curate the data for any domain continual pre-training.