TydiQA dataset is mixed and is not split per language 
{'login': 'dorost1234', 'id': 79165106, 'node_id': 'MDQ6VXNlcjc5MTY1MTA2', 'avatar_url': 'https://avatars.githubusercontent.com/u/79165106?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/dorost1234', 'html_url': 'https://github.com/dorost1234', 'followers_url': 'https://api.github.com/users/dorost1234/followers', 'following_url': 'https://api.github.com/users/dorost1234/following{/other_user}', 'gists_url': 'https://api.github.com/users/dorost1234/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/dorost1234/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/dorost1234/subscriptions', 'organizations_url': 'https://api.github.com/users/dorost1234/orgs', 'repos_url': 'https://api.github.com/users/dorost1234/repos', 'events_url': 'https://api.github.com/users/dorost1234/events{/privacy}', 'received_events_url': 'https://api.github.com/users/dorost1234/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['You can filter the languages this way:\r\n```python\r\ntydiqa_en = tydiqa_dataset.filter(lambda x: x["language"] == "english")\r\n```\r\n\r\nOtherwise maybe we can have one configuration per language ?\r\nWhat do you think of this for example ?\r\n\r\n```python\r\nload_dataset("tydiqa", "primary_task.en")\r\n```'
 'Hi\nthank you very much for the great response, this will be really wonderful\nto have one configuration per language, as one need the dataset in majority\nof case per language for cross-lingual evaluations.\nThis becomes also then more close to TFDS format, which is separated per\nlanguage https://www.tensorflow.org/datasets/catalog/tydi_qa which will be\nreally awesome to have.\nthanks\n\nOn Mon, Mar 29, 2021 at 6:17 PM Quentin Lhoest ***@***.***>\nwrote:\n\n> You can filter the languages this way:\n>\n> tydiqa_en = tydiqa_dataset.filter(lambda x: x["language"] == "english")\n>\n> Otherwise maybe we can have one configuration per language ?\n> What do you think of this for example ?\n>\n> load_dataset("tydiqa", "primary_task.en")\n>\n> —\n> You are receiving this because you authored the thread.\n> Reply to this email directly, view it on GitHub\n> <https://github.com/huggingface/datasets/issues/2132#issuecomment-809516799>,\n> or unsubscribe\n> <https://github.com/notifications/unsubscribe-auth/AS37NMXPW2PWSQ2RHG73O7TTGCY4LANCNFSM4Z7ER7IA>\n> .\n>\n'
 '@lhoestq  I greatly appreciate any updates on this. thanks a lot']
Hi @lhoestq 
Currently TydiQA is mixed and user can only access the whole training set of all languages:
https://www.tensorflow.org/datasets/catalog/tydi_qa

for using this dataset, one need to train/evaluate in each separate language, and having them mixed, makes it hard to use this dataset. This is much convenient for user to have  them split and I appreciate your help on this. 

Meanwhile, till hopefully this is split per language, I greatly appreciate telling me how I can preprocess and get data per language. thanks a lot 
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/2132/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}