Dataset.map() turns tensors into lists?
{'login': 'tombosc', 'id': 5270804, 'node_id': 'MDQ6VXNlcjUyNzA4MDQ=', 'avatar_url': 'https://avatars.githubusercontent.com/u/5270804?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/tombosc', 'html_url': 'https://github.com/tombosc', 'followers_url': 'https://api.github.com/users/tombosc/followers', 'following_url': 'https://api.github.com/users/tombosc/following{/other_user}', 'gists_url': 'https://api.github.com/users/tombosc/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/tombosc/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/tombosc/subscriptions', 'organizations_url': 'https://api.github.com/users/tombosc/orgs', 'repos_url': 'https://api.github.com/users/tombosc/repos', 'events_url': 'https://api.github.com/users/tombosc/events{/privacy}', 'received_events_url': 'https://api.github.com/users/tombosc/received_events', 'type': 'User', 'site_admin': False}
[]
[]
A solution is to have the tokenizer return a list instead of a tensor, and then use `dataset_tok.set_format(type = 'torch')` to convert that list into a tensor. Still not sure if bug. It is expected behavior, you should set the format to `"torch"` as you mentioned to get pytorch tensors back.
By default datasets returns pure python objects.
I apply `Dataset.map()` to a function that returns a dict of torch tensors (like a tokenizer from the repo transformers). However, in the mapped dataset, these tensors have turned to lists!

```import datasets
import torch  
from datasets import load_dataset                                                                                                                 
print("version datasets", datasets.__version__)

dataset = load_dataset("snli", split='train[0:50]')  

def tokenizer_fn(example):
    # actually uses a tokenizer which does something like:
    return {'input_ids': torch.tensor([[0, 1, 2]])}

print("First item in dataset:\n", dataset[0])
tokenized = tokenizer_fn(dataset[0])
print("Tokenized hyp:\n", tokenized)
dataset_tok = dataset.map(tokenizer_fn, batched=False,
        remove_columns=['label', 'premise', 'hypothesis'])
print("Tokenized using map:\n", dataset_tok[0])
print(type(tokenized['input_ids']), type(dataset_tok[0]['input_ids']))
dataset_tok = dataset.map(tokenizer_fn, batched=False,
                          remove_columns=['label', 'premise', 'hypothesis'])
print("Tokenized using map:\n", dataset_tok[0])
print(type(tokenized['input_ids']), type(dataset_tok[0]['input_ids']))
```

The output is:

```
version datasets 1.1.3
Reusing dataset snli (/home/tom/.cache/huggingface/datasets/snli/plain_text/1.0.0/bb1102591c6230bd78813e229d5dd4c7fbf4fc478cec28f298761eb69e5b537c)
First item in dataset:
 {'premise': 'A person on a horse jumps over a broken down airplane.', 'hypothesis': 'A person is training his horse for a competition.', 'label': 1}
Tokenized hyp:
 {'input_ids': tensor([[0, 1, 2]])}
Loading cached processed dataset at /home/tom/.cache/huggingface/datasets/snli/plain_text/1.0.0/bb1102591c6230bd78813e229d5dd4c7fbf4fc478cec28f298761eb69e5b537c/cache-fe38f449fe9ac46f.arrow
Tokenized using map:
 {'input_ids': [[0, 1, 2]]}
<class 'torch.Tensor'> <class 'list'>
```

Or am I doing something wrong?

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/1046/reactions', 'total_count': 3, '+1': 3, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}