load_dataset("timit_asr") gives back duplicates of just one sample text
{'login': 'ekeleshian', 'id': 33647474, 'node_id': 'MDQ6VXNlcjMzNjQ3NDc0', 'avatar_url': 'https://avatars.githubusercontent.com/u/33647474?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/ekeleshian', 'html_url': 'https://github.com/ekeleshian', 'followers_url': 'https://api.github.com/users/ekeleshian/followers', 'following_url': 'https://api.github.com/users/ekeleshian/following{/other_user}', 'gists_url': 'https://api.github.com/users/ekeleshian/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/ekeleshian/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/ekeleshian/subscriptions', 'organizations_url': 'https://api.github.com/users/ekeleshian/orgs', 'repos_url': 'https://api.github.com/users/ekeleshian/repos', 'events_url': 'https://api.github.com/users/ekeleshian/events{/privacy}', 'received_events_url': 'https://api.github.com/users/ekeleshian/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892857, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODU3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/bug', 'name': 'bug', 'color': 'd73a4a', 'default': True, 'description': "Something isn't working"}]
[]
['Upgrading datasets to version 1.6 fixes the issue'
 'This bug was fixed in #1995. Upgrading the `datasets` should work! '
 'Thanks @ekeleshian for having reported.\r\n\r\nI am closing this issue once that you updated `datasets`. Feel free to reopen it if the problem persists.']
## Describe the bug
When you look up on key ["train"] and then ['text'], you get back a list  with just one sentence duplicated 4620 times. Namely, the sentence "Would such an act of refusal be useful?". Similarly when you look up ['test'] and then ['text'], the list is one sentence repeated "The bungalow was pleasantly situated near the shore." 1680 times. 

I tried to work around the issue by downgrading to datasets version 1.3.0, inspired by [this post](https://www.gitmemory.com/issue/huggingface/datasets/2052/798904836) and removing the entire huggingface directory from ~/.cache, but I still get the same issue.  

## Steps to reproduce the bug
```python
from datasets import load_dataset
timit = load_dataset("timit_asr")
print(timit['train']['text'])
print(timit['test']['text'])
```

## Expected Result
Rows of diverse text, like how it is shown in the [wav2vec2.0 tutorial](https://colab.research.google.com/github/patrickvonplaten/notebooks/blob/master/Fine_tuning_Wav2Vec2_for_English_ASR.ipynb)
<img width="485" alt="Screen Shot 2021-05-05 at 9 09 57 AM" src="https://user-images.githubusercontent.com/33647474/117146094-d9b77f00-ad81-11eb-8306-f281850c127a.png">


## Actual results
Rows of repeated text.
<img width="319" alt="Screen Shot 2021-05-05 at 9 11 53 AM" src="https://user-images.githubusercontent.com/33647474/117146231-f8b61100-ad81-11eb-834a-fc10410b0c9c.png">


## Versions
- Datasets: 1.3.0
- Python: 3.9.1
- Platform: macOS-11.2.1-x86_64-i386-64bit}

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/2323/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}