combination of shuffle and filter results in a bug
{'login': 'rabeehk', 'id': 6278280, 'node_id': 'MDQ6VXNlcjYyNzgyODA=', 'avatar_url': 'https://avatars.githubusercontent.com/u/6278280?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/rabeehk', 'html_url': 'https://github.com/rabeehk', 'followers_url': 'https://api.github.com/users/rabeehk/followers', 'following_url': 'https://api.github.com/users/rabeehk/following{/other_user}', 'gists_url': 'https://api.github.com/users/rabeehk/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/rabeehk/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/rabeehk/subscriptions', 'organizations_url': 'https://api.github.com/users/rabeehk/orgs', 'repos_url': 'https://api.github.com/users/rabeehk/repos', 'events_url': 'https://api.github.com/users/rabeehk/events{/privacy}', 'received_events_url': 'https://api.github.com/users/rabeehk/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892857, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODU3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/bug', 'name': 'bug', 'color': 'd73a4a', 'default': True, 'description': "Something isn't working"}]
[]
I cannot reproduce this on master and pyarrow==4.0.1.
 Hi ! There was a regression in `datasets` 1.12 that introduced this bug. It has been fixed in #3019 in 1.13

Can you try to update `datasets` and try again ? Thanks a lot, fixes with 1.13
## Describe the bug
Hi,
I would like to shuffle a dataset, then filter it based on each existing label. however, the combination of `filter`, `shuffle` seems to results in a bug. In the minimal example below, as you see in the filtered results, the filtered labels are not unique, meaning filter has not worked. Any suggestions as a temporary fix is appreciated @lhoestq.

Thanks.
 Best regards
Rabeeh 

## Steps to reproduce the bug
```python
import numpy as np
import datasets 

datasets = datasets.load_dataset('super_glue', 'rte', script_version="master")
shuffled_data = datasets["train"].shuffle(seed=42)
for label in range(2):
    print("label ", label)
    data = shuffled_data.filter(lambda example: int(example['label']) == label)
    print("length ", len(data), np.unique(data['label']))
```

## Expected results
Filtering per label, should only return the data with that specific label.

## Actual results
As you can see, filtered data per label, has still two labels of [0, 1]
```
label  0
length  1249 [0 1]
label  1
length  1241 [0 1]
```
## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 1.12.1 
- Platform: linux 
- Python version: 3.7.11 
- PyArrow version: 5.0.0 

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3190/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}