wikiann dataset is missing columns 
{'login': 'dorost1234', 'id': 79165106, 'node_id': 'MDQ6VXNlcjc5MTY1MTA2', 'avatar_url': 'https://avatars.githubusercontent.com/u/79165106?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/dorost1234', 'html_url': 'https://github.com/dorost1234', 'followers_url': 'https://api.github.com/users/dorost1234/followers', 'following_url': 'https://api.github.com/users/dorost1234/following{/other_user}', 'gists_url': 'https://api.github.com/users/dorost1234/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/dorost1234/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/dorost1234/subscriptions', 'organizations_url': 'https://api.github.com/users/dorost1234/orgs', 'repos_url': 'https://api.github.com/users/dorost1234/repos', 'events_url': 'https://api.github.com/users/dorost1234/events{/privacy}', 'received_events_url': 'https://api.github.com/users/dorost1234/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892877, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODc3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/good%20first%20issue', 'name': 'good first issue', 'color': '7057ff', 'default': True, 'description': 'Good for newcomers'}]
[]
['Here please find TFDS format of this dataset: https://www.tensorflow.org/datasets/catalog/wikiann\r\nwhere there is a span column, this is really necessary to be able to use the data, and I appreciate your help @lhoestq '
 'Hi !\r\nApparently you can get the spans from the NER tags using `tags_to_spans` defined here:\r\n\r\nhttps://github.com/tensorflow/datasets/blob/c7096bd38e86ed240b8b2c11ecab9893715a7d55/tensorflow_datasets/text/wikiann/wikiann.py#L81-L126\r\n\r\nIt would be nice to include the `spans` field in this dataset as in TFDS. This could be a good first issue for new contributors !\r\n\r\nThe objective is to use `tags_to_spans` in the `_generate_examples` method [here](https://github.com/huggingface/nlp/blob/c98e4b8f23e3770c401c6d9326e243e1ffd599ec/datasets/wikiann/wikiann.py#L292-L316) to create he `spans` for each example.'
 'Hi @lhoestq \r\nthank you very much for the help, it would be very nice to have it included, here is the full code, one need to also convert tags to string first:\r\n\r\n```\r\nimport datasets \r\nfrom datasets import load_dataset\r\n\r\ndef tags_to_spans(tags):\r\n  """Convert tags to spans."""\r\n  spans = set()\r\n  span_start = 0\r\n  span_end = 0\r\n  active_conll_tag = None\r\n  for index, string_tag in enumerate(tags):\r\n    # Actual BIO tag.\r\n    bio_tag = string_tag[0]\r\n    assert bio_tag in ["B", "I", "O"], "Invalid Tag"\r\n    conll_tag = string_tag[2:]\r\n    if bio_tag == "O":\r\n      # The span has ended.\r\n      if active_conll_tag:\r\n        spans.add((active_conll_tag, (span_start, span_end)))\r\n      active_conll_tag = None\r\n      # We don\'t care about tags we are\r\n      # told to ignore, so we do nothing.\r\n      continue\r\n    elif bio_tag == "B":\r\n      # We are entering a new span; reset indices and active tag to new span.\r\n      if active_conll_tag:\r\n        spans.add((active_conll_tag, (span_start, span_end)))\r\n      active_conll_tag = conll_tag\r\n      span_start = index\r\n      span_end = index\r\n    elif bio_tag == "I" and conll_tag == active_conll_tag:\r\n      # We\'re inside a span.\r\n      span_end += 1\r\n    else:\r\n      # This is the case the bio label is an "I", but either:\r\n      # 1) the span hasn\'t started - i.e. an ill formed span.\r\n      # 2) We have IOB1 tagging scheme.\r\n      # We\'ll process the previous span if it exists, but also include this\r\n      # span. This is important, because otherwise, a model may get a perfect\r\n      # F1 score whilst still including false positive ill-formed spans.\r\n      if active_conll_tag:\r\n        spans.add((active_conll_tag, (span_start, span_end)))\r\n      active_conll_tag = conll_tag\r\n      span_start = index\r\n      span_end = index\r\n  # Last token might have been a part of a valid span.\r\n  if active_conll_tag:\r\n    spans.add((active_conll_tag, (span_start, span_end)))\r\n  # Return sorted list of spans\r\n  return sorted(list(spans), key=lambda x: x[1][0])\r\n\r\ndataset = load_dataset(\'wikiann\', \'en\', split="train")\r\nner_tags = {\r\n   0:"O",\r\n   1:"B-PER",\r\n   2:"I-PER",\r\n   3:"B-ORG",\r\n   4:"I-ORG",\r\n   5:"B-LOC",\r\n   6:"I-LOC"\r\n}\r\n\r\ndef get_spans(tokens, tags):\r\n  """Convert tags to textspans."""\r\n  spans = tags_to_spans(tags)\r\n  text_spans = [\r\n      x[0] + ": " + " ".join([tokens[i]\r\n                              for i in range(x[1][0], x[1][1] + 1)])\r\n      for x in spans\r\n  ]\r\n  if not text_spans:\r\n    text_spans = ["None"]\r\n  return text_spans\r\n\r\n\r\nfor i, d in enumerate(dataset):\r\n   tokens = d[\'tokens\']\r\n   tags = d[\'ner_tags\']\r\n   tags = [ner_tags[i] for i in tags]\r\n   spans = get_spans(tokens, tags)\r\n   print("spans ", spans)\r\n   print(d)\r\n   if i > 10:\r\n     break; \r\n```\r\nI am not sure how to contribute to the repository and how things work, could you let me know how one can access the datasets to be able to contribute to the repository? Maybe I could do it then\r\nthanks \r\n'
 'Cool ! Let me give you some context:\r\n\r\n#### Contribution guide\r\n\r\nYou can find the contribution guide here:\r\n\r\nhttps://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md\r\n\r\nIt explains how to set up your dev environment in a few steps.\r\n\r\n#### Dataset loading\r\n\r\nEach Dataset is defined by a Table that have many rows (one row = one example) and columns (one column = one feature).\r\nTo change how a dataset is constructed, you have to modify its dataset script that you can find here:\r\n\r\nhttps://github.com/huggingface/datasets/blob/master/datasets/wikiann/wikiann.py\r\n\r\nIt includes everything needed to load the WikiANN dataset.\r\nYou can load locally a modified version of `wikiann.py` with `load_dataset("path/to/wikiann.py")`.\r\n\r\n#### Define a new column\r\n\r\nEach column has a name and a type. You can see how the features of WikiANN are defined here:\r\n\r\nhttps://github.com/huggingface/datasets/blob/c98e4b8f23e3770c401c6d9326e243e1ffd599ec/datasets/wikiann/wikiann.py#L245-L263\r\n\r\nIdeally we would have one additional feature "spans":\r\n```python\r\n        "spans": datasets.Sequence(datasets.Value("string")),\r\n```\r\n\r\n#### Compute the content of each row\r\n\r\nTo build the WikiANN rows, the _generate_examples method from [here](https://github.com/huggingface/nlp/blob/c98e4b8f23e3770c401c6d9326e243e1ffd599ec/datasets/wikiann/wikiann.py#L292-L316) is used. This function `yield` one python dictionary for each example:\r\n```python\r\nyield guid_index, {"tokens": tokens, "ner_tags": ner_tags, "langs": langs}\r\n```\r\n\r\nThe objective would be to return instead something like\r\n```python\r\nspans = spans = get_spans(tokens, tags)\r\nyield guid_index, {"tokens": tokens, "ner_tags": ner_tags, "langs": langs, "spans": spans}\r\n```\r\n\r\nLet me know if you have questions !'
 'The PR was merged. Issue should be closed.\r\n\r\nCC: @lhoestq ']
Hi
Wikiann dataset needs to have "spans" columns, which is necessary to be able to use this dataset, but this column is missing from huggingface datasets, could you please have a look? thank you @lhoestq 
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/2130/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}