News_commentary Dataset Translation Pairs are of Incorrect Language Specified Pairs
{'login': 'anassalamah', 'id': 8571003, 'node_id': 'MDQ6VXNlcjg1NzEwMDM=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8571003?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/anassalamah', 'html_url': 'https://github.com/anassalamah', 'followers_url': 'https://api.github.com/users/anassalamah/followers', 'following_url': 'https://api.github.com/users/anassalamah/following{/other_user}', 'gists_url': 'https://api.github.com/users/anassalamah/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/anassalamah/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/anassalamah/subscriptions', 'organizations_url': 'https://api.github.com/users/anassalamah/orgs', 'repos_url': 'https://api.github.com/users/anassalamah/repos', 'events_url': 'https://api.github.com/users/anassalamah/events{/privacy}', 'received_events_url': 'https://api.github.com/users/anassalamah/received_events', 'type': 'User', 'site_admin': False}
[]
[]
Hi @anassalamah,

Could you please try with this:
```python
train_ds = load_dataset("news_commentary", lang1="ar", lang2="en", split='train[:98%]')
val_ds = load_dataset("news_commentary", lang1="ar", lang2="en", split='train[98%:]')
``` Hello @albertvillanova, 

Thanks for the suggestion. I didn't know you could do that. however, it didn't resolve the issue

![image](https://user-images.githubusercontent.com/8571003/114169966-ec819400-993a-11eb-8a67-930f9a9b2290.png)

I used load_dataset to load the news_commentary dataset for "ar-en" translation pairs but found translations from Arabic to Hindi.  

```
train_ds = load_dataset("news_commentary", "ar-en", split='train[:98%]')
val_ds = load_dataset("news_commentary", "ar-en", split='train[98%:]')

# filtering out examples that are not ar-en translations but ar-hi
val_ds = val_ds.filter(lambda example, indice: indice not in chain(range(1312,1327) ,range(1384,1399), range(1030,1042)), with_indices=True)
```

* I'm fairly new to using datasets so I might be doing something wrong
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/2190/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}