[Feature request] Add a feature to dataset
{'login': 'sarahwie', 'id': 8027676, 'node_id': 'MDQ6VXNlcjgwMjc2NzY=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8027676?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/sarahwie', 'html_url': 'https://github.com/sarahwie', 'followers_url': 'https://api.github.com/users/sarahwie/followers', 'following_url': 'https://api.github.com/users/sarahwie/following{/other_user}', 'gists_url': 'https://api.github.com/users/sarahwie/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/sarahwie/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/sarahwie/subscriptions', 'organizations_url': 'https://api.github.com/users/sarahwie/orgs', 'repos_url': 'https://api.github.com/users/sarahwie/repos', 'events_url': 'https://api.github.com/users/sarahwie/events{/privacy}', 'received_events_url': 'https://api.github.com/users/sarahwie/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['Do you have an example of what you would like to do? (you can just add a field in the output of the unction you give to map and this will add this field in the output table)'
 "Given another source of data loaded in, I want to pre-add it to the dataset so that it aligns with the indices of the arrow dataset prior to performing map.\r\n\r\nE.g. \r\n```\r\nnew_info = list of length dataset['train']\r\n\r\ndataset['train'] = dataset['train'].map(lambda x: some_function(x, new_info[index of x]))\r\n\r\ndef some_function(x, new_info_x):\r\n    # adds new_info[index of x] as a field to x\r\n    x['new_info'] = new_info_x\r\n    return x\r\n```\r\nI was thinking to instead create a new field in the arrow dataset so that instance x contains all the necessary information when map function is applied (since I don't have index information to pass to map function)."
 "This is what I have so far: \r\n\r\n```\r\nimport pyarrow as pa\r\nfrom nlp.arrow_dataset import Dataset\r\n\r\naug_dataset = dataset['train'][:]\r\naug_dataset['new_info'] = new_info\r\n\r\n#reformat as arrow-table\r\nschema = dataset['train'].schema\r\n\r\n# this line doesn't work:\r\nschema.append(pa.field('new_info', pa.int32()))\r\n\r\ntable = pa.Table.from_pydict(\r\n    aug_dataset,\r\n    schema=schema\r\n)\r\ndataset['train'] = Dataset(table) \r\n```"
 "Maybe you can use `with_indices`?\r\n\r\n```python\r\nnew_info = list of length dataset['train']\r\n\r\ndef some_function(indice, x):\r\n    # adds new_info[index of x] as a field to x\r\n    x['new_info'] = new_info_x[indice]\r\n    return x\r\n\r\ndataset['train'] = dataset['train'].map(some_function, with_indices=True)\r\n```"
 'Oh great. That should work. I missed that in the documentation- thanks :) ']
Is there a straightforward way to add a field to the arrow_dataset, prior to performing map?
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/256/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}