Implicit domain aggregation.A remaining limitation is that we still need to acquire the components of validation data \(s_{1\dots K}\) in advance. This can be inconvenient if the validation set is collected separately from the training ones. For instance, the validation data may come from real-world user queries that cover unknown compositions of various domains. To remove the constraint on validation components, we assume that we can decompose the validation data into \(K\) implicit domains whose losses are predictable with Eqn. 7, and we treat their proportions in the validation data \(s_{1\dots K}\) as learnable parameters. This leads to the final form of our data mixing laws6. With this perspective, we fit a data mixing law with the overall losses end to end.