Add long answer candidates to natural questions dataset
{'login': 'seirasto', 'id': 4257308, 'node_id': 'MDQ6VXNlcjQyNTczMDg=', 'avatar_url': 'https://avatars.githubusercontent.com/u/4257308?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/seirasto', 'html_url': 'https://github.com/seirasto', 'followers_url': 'https://api.github.com/users/seirasto/followers', 'following_url': 'https://api.github.com/users/seirasto/following{/other_user}', 'gists_url': 'https://api.github.com/users/seirasto/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/seirasto/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/seirasto/subscriptions', 'organizations_url': 'https://api.github.com/users/seirasto/orgs', 'repos_url': 'https://api.github.com/users/seirasto/repos', 'events_url': 'https://api.github.com/users/seirasto/events{/privacy}', 'received_events_url': 'https://api.github.com/users/seirasto/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['_The documentation is not available anymore as the PR was closed or merged._'
 'Once we have added `long_answer_candidates` maybe it would be worth to also add the missing `candidate_index` (inside `long_answer`). What do you think, @seirasto ?'
 'Also note the "Data Fields" section in the README is missing the `long_answer` field.\r\n\r\nMoreover, there is no instance example in "Data Instances" section.'
 'We could either make these fixes in this PR or in a subsequent PR.'
 "@albertvillanova I've added the missing fields and updated the README to include a data instance and some other things. "
 "Great! I've made the updates to align the README. Please let me know if I missed anything."
 'As there were many minor little fixes, I thought it would be easier to fix them directly.'
 'I think the loading script is OK now. If it is also validated by another datasets maintainer, I could run the generation of the pre-processed data and then merge this PR into master (once all the tests are green).\r\n\r\nCC: @lhoestq '
 'It looks good to me, thanks @seirasto !'
 'I have merged the master branch, so that we include all the fixes on Apache Beam + Google Dataflow.'
 'Pre-processing is running!\r\n\r\nAlready finished for "dev" config:\r\n```python\r\nIn [2]: ds = load_dataset("datasets/natural_questions", "dev")\r\n\r\nIn [3]: ds\r\nOut[3]: \r\nDatasetDict({\r\n    validation: Dataset({\r\n        features: [\'id\', \'document\', \'question\', \'long_answer_candidates\', \'annotations\'],\r\n        num_rows: 7830\r\n    })\r\n})\r\n```'
 'There is an issue while running the preprocessing for the "default" (train+dev) config. Train data files are larger than than dev ones and workers run out of memory.\r\n\r\nI\'m opening a separate issue to handle this problem: #4525'
 '@seirasto is proposing uploading their preprocessed data files to our Datasets bucket.\r\n\r\nI think @lhoestq can give a more informed answer about authentication requirements.'
 "Now that the data fiels are uploaded, can you merge the `main` branch into yours to re-trigger the CI @seirasto please ? :) Then I think we can merge if it's good for you @albertvillanova "
 'Merge is done! I think someone needs to approve the CI to run :) '
 'Can you run `make style` to fix the code formatting required by the CI please ?'
 "Thanks @albertvillanova! I've committed all your suggestions."
 "The CI is green. I'm merging this PR."]
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/4368', 'html_url': 'https://github.com/huggingface/datasets/pull/4368', 'diff_url': 'https://github.com/huggingface/datasets/pull/4368.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/4368.patch', 'merged_at': datetime.datetime(2022, 7, 26, 20, 18, 42)}
This is a modification of the Natural Questions dataset to include missing information specifically related to long answer candidates. (See here: https://github.com/google-research-datasets/natural-questions#long-answer-candidates). This information is important to ensure consistent comparison with prior work. It does not disturb the rest of the format . @lhoestq @albertvillanova 
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/4368/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}