Added PragmEval benchmark
{'login': 'sileod', 'id': 9168444, 'node_id': 'MDQ6VXNlcjkxNjg0NDQ=', 'avatar_url': 'https://avatars.githubusercontent.com/u/9168444?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/sileod', 'html_url': 'https://github.com/sileod', 'followers_url': 'https://api.github.com/users/sileod/followers', 'following_url': 'https://api.github.com/users/sileod/following{/other_user}', 'gists_url': 'https://api.github.com/users/sileod/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/sileod/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/sileod/subscriptions', 'organizations_url': 'https://api.github.com/users/sileod/orgs', 'repos_url': 'https://api.github.com/users/sileod/repos', 'events_url': 'https://api.github.com/users/sileod/events{/privacy}', 'received_events_url': 'https://api.github.com/users/sileod/received_events', 'type': 'User', 'site_admin': False}
[]
[]
["> Really cool ! Thanks for adding this one :)\r\n> Good job at adding all those citations for each task\r\n> \r\n> Looks like the dummy data test doesn't pass. Maybe some files are missing in the dummy_data.zip files ?\r\n> The error reports `pragmeval/verifiability/train.tsv` to be missing\r\n> \r\n> Also could you add the tags part of the dataset card (the rest is optional) ?\r\n> See more info here : https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md#tag-the-dataset-and-write-the-dataset-card\r\n\r\nIn the prior commits I generated dataset_infos and the dummy files myself\r\nNow they are generated with the cli, and the tests now seem to be passing better\r\nI will look into the tag\r\n"
 'Looks like you did a good job with dummy data in the first place !\r\nThe downside of automatically generated dummy data is that the files are heavier (here 40KB per file).\r\nIf you could replace the generated dummy files with the one you created yourself it would be awesome, since the one you did yourself are way lighter (around 1KB per file). Using small files make `git clone` run faster so we encourage to use small dummy_data files.'
 'could you rebase from master ? it should fix the CI'
 '> could you rebase from master ? it should fix the CI\r\n\r\nI think it is due to the file structure of the dummy data that causes test failure. The automatically generated dummy data pass the tests'
 'Indeed the error reports that `pragmeval/verifiability/train.tsv` is missing for the verifiability dummy_data.zip file.\r\nTo fix that you should add the missing data files in each dummy_data.zip file.\r\nTo test that your dummy data work you can run\r\n```\r\nRUN_SLOW=1 pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_all_configs_\r\n```\r\nif some file is missing it should tell you which one'
 "Also it looks like you haven't rebased from master yet, even though you did a `rebase` commit. \r\n\r\nrebasing should fix the other CI fails"
 "It's ok if we have `RemoteDatasetTest ` errors, they're fixed on master"
 'merging since the CI is fixed on master'
 "Hey @sileod! Super nice to see you participating ;)\r\n\r\nDid you officially joined the sprint by posting on [the forum thread](https://discuss.huggingface.co/t/open-to-the-community-one-week-team-effort-to-reach-v2-0-of-hf-datasets-library/2176) and joining our slack?\r\n\r\nI can't seem to find you there! Should I add you directly with your gmail address?"
 'Hi @sileod 👋 ']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/955', 'html_url': 'https://github.com/huggingface/datasets/pull/955', 'diff_url': 'https://github.com/huggingface/datasets/pull/955.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/955.patch', 'merged_at': datetime.datetime(2020, 12, 3, 9, 36, 47)}
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/955/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}