Adding a new column to the dataset after set_format was called
{'login': 'villmow', 'id': 2743060, 'node_id': 'MDQ6VXNlcjI3NDMwNjA=', 'avatar_url': 'https://avatars.githubusercontent.com/u/2743060?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/villmow', 'html_url': 'https://github.com/villmow', 'followers_url': 'https://api.github.com/users/villmow/followers', 'following_url': 'https://api.github.com/users/villmow/following{/other_user}', 'gists_url': 'https://api.github.com/users/villmow/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/villmow/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/villmow/subscriptions', 'organizations_url': 'https://api.github.com/users/villmow/orgs', 'repos_url': 'https://api.github.com/users/villmow/repos', 'events_url': 'https://api.github.com/users/villmow/events{/privacy}', 'received_events_url': 'https://api.github.com/users/villmow/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['Hi ! Indeed if you add a column to a formatted dataset, then the new dataset gets a new formatting in which:\r\n```\r\nnew formatted columns = (all columns - previously unformatted columns)\r\n```\r\nTherefore the new column is going to be formatted using the `torch` formatting.\r\n\r\nIf you want your new column to be unformatted you can re-run this line:\r\n```python\r\ndata.set_format("torch", columns=["some_integer_column1", "some_integer_column2"], output_all_columns=True)\r\n```'
 'Hi, thanks that solved my problem. Maybe mention that in the documentation. '
 'Ok cool :) \r\nAlso I just did a PR to mention this behavior in the documentation'
 'Closed by #1888']
Hi, 

thanks for the nice library. I'm in the process of creating a custom dataset, which has a mix of tensors and lists of strings. I stumbled upon an error and want to know if its a problem on my side. 

I load some lists of strings and integers, then call `data.set_format("torch", columns=["some_integer_column1", "some_integer_column2"], output_all_columns=True)`. This converts the integer columns into tensors, but keeps the lists of strings as they are. I then call `map` to add a new column to my dataset, which is a **list of strings**. Once I iterate through my dataset, I get an error that the new column can't be converted into a tensor (which is probably caused by `set_format`). 

Below some pseudo code:
```python
    def augment_func(sample: Dict) -> Dict:
        # do something
        return {
         "some_integer_column1" : augmented_data["some_integer_column1"],  # <-- tensor
         "some_integer_column2" : augmented_data["some_integer_column2"],  # <-- tensor
         "NEW_COLUMN": targets,  # <-- list of strings
        }


    data = datasets.load_dataset(__file__, data_dir="...", split="train")
    data.set_format("torch", columns=["some_integer_column1", "some_integer_column2"], output_all_columns=True)

    augmented_dataset = data.map(augment_func, batched=False)
    
    for sample in augmented_dataset:
        print(sample)  # fails

```

and the exception:
```python
Traceback (most recent call last):
  File "dataset.py", line 487, in <module>
    main()
  File "dataset.py", line 471, in main
    for sample in augmented_dataset:
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 697, in __iter__
    yield self._getitem(
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1069, in _getitem
    outputs = self._convert_outputs(
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 890, in _convert_outputs
    v = map_nested(command, v, **map_nested_kwargs)
  File "lib/python3.8/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    return function(data_struct)
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 850, in command
    return [map_nested(command, i, **map_nested_kwargs) for i in x]
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 850, in <listcomp>
    return [map_nested(command, i, **map_nested_kwargs) for i in x]
  File "lib/python3.8/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    return function(data_struct)
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 850, in command
    return [map_nested(command, i, **map_nested_kwargs) for i in x]
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 850, in <listcomp>
    return [map_nested(command, i, **map_nested_kwargs) for i in x]
  File "lib/python3.8/site-packages/datasets/utils/py_utils.py", line 225, in map_nested
    return function(data_struct)
  File "lib/python3.8/site-packages/datasets/arrow_dataset.py", line 851, in command
    return torch.tensor(x, **format_kwargs)
TypeError: new(): invalid data type 'str'
```

Thanks!

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/1872/reactions', 'total_count': 1, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 1}