Identify which shard data belongs to
{'login': 'borisdayma', 'id': 715491, 'node_id': 'MDQ6VXNlcjcxNTQ5MQ==', 'avatar_url': 'https://avatars.githubusercontent.com/u/715491?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/borisdayma', 'html_url': 'https://github.com/borisdayma', 'followers_url': 'https://api.github.com/users/borisdayma/followers', 'following_url': 'https://api.github.com/users/borisdayma/following{/other_user}', 'gists_url': 'https://api.github.com/users/borisdayma/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/borisdayma/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/borisdayma/subscriptions', 'organizations_url': 'https://api.github.com/users/borisdayma/orgs', 'repos_url': 'https://api.github.com/users/borisdayma/repos', 'events_url': 'https://api.github.com/users/borisdayma/events{/privacy}', 'received_events_url': 'https://api.github.com/users/borisdayma/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892871, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODcx', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/enhancement', 'name': 'enhancement', 'color': 'a2eeef', 'default': True, 'description': 'New feature or request'}]
[]
Independently of this I think it raises the need to allow multiprocessing during streaming so that we get samples from multiple shards in one batch.
**Is your feature request related to a problem? Please describe.**

I'm training on a large dataset made of multiple sub-datasets.
During training I can observe some jumps in loss which may correspond to different shards.

![image](https://user-images.githubusercontent.com/715491/136668758-521263aa-a9b2-4ad2-8d22-060b6bf86a1c.png)

My suspicion is that either:
* some of the sub-datasets are harder for the model than others
* some of the sub-datasets are not formatted properly

I'd like to identify which shards correspond to those jumps.

**Describe the solution you'd like**

It would be nice to have a key associated to each data sample or data batch containing details on where the data comes from (shard idx + item idx within the shard).
This should be supported both in local and streaming mode.

**Describe alternatives you've considered**
A fix would be for me to add myself details (shard id, sample id) as part of each data sample.
The inconvenient is that it requires users to process/reupload every dataset when they need this feature.
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3048/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}