using map on loaded Tokenizer 10x - 100x slower than default Tokenizer?
Thanks for the experiments @johncookds and @wumpusman ! 

> Hi, which version of datasets has datasets.set_caching_enabled(False)?

Currently you have to install `datasets` from source to have this feature, but this will be available in the next release in a few days.

> I'm trying to figure out why the overhead of map would increase the time by double (figured it would be a fixed increase in time)? Though maybe this is expected behavior.

Could you also try with double the number of characters ? This should let us have an idea of the fixed cost (hashing) and the dynamic cost (actual tokenization, grows with the size of the input)

> @lhoestq, do you by chance know how I can redirect this issue to Tokenizer?

Feel free to post an issue on the `transformers` repo. Also I'm sure there should be related issues so you can also look for someone with the same concerns on the `transformers` repo.
This could total relate to me misunderstanding particular call functions, but I added words to a GPT2Tokenizer, and saved it to disk (note I'm only showing snippets but I can share more) and the map function ran much slower: 

````
def save_tokenizer(original_tokenizer,text,path="simpledata/tokenizer"):
    words_unique = set(text.split(" "))
    for i in words_unique:
        original_tokenizer.add_tokens(i)
    original_tokenizer.save_pretrained(path)

tokenizer2 = GPT2Tokenizer.from_pretrained(os.path.join(experiment_path,experiment_name,"tokenizer_squad"))

train_set_baby=Dataset.from_dict({"text":[train_set["text"][0][0:50]]})
````

I then applied the dataset map function on a fairly small set of text:

```
%%time
train_set_baby = train_set_baby.map(lambda d:tokenizer2(d["text"]),batched=True)

```


The run time for train_set_baby.map was 6 seconds, and the batch itself was 2.6 seconds

**100% 1/1 [00:02<00:00, 2.60s/ba] CPU times: user 5.96 s, sys: 36 ms, total: 5.99 s Wall time: 5.99 s**

In comparison using (even after adding additional tokens): 
`
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")`

```
%%time
train_set_baby = train_set_baby.map(lambda d:tokenizer2(d["text"]),batched=True)

```
The time is 
**100% 1/1 [00:00<00:00, 34.09ba/s] CPU times: user 68.1 ms, sys: 16 µs, total: 68.1 ms Wall time: 62.9 ms**

It seems this might relate to the tokenizer save or load function, however, the issue appears to come up when I apply the loaded tokenizer  to the map function. 

I should also add that playing around with the amount of words I add to the tokenizer before I save it to disk and load it into memory  appears to impact the time it takes to run the map function. 



using map on loaded Tokenizer 10x - 100x slower than default Tokenizer? 
 This could total relate to me misunderstanding particular call functions, but I added words to a GPT2Tokenizer, and saved it to disk (note I'm only showing snippets but I can share more) and the map function ran much slower: 

````
def save_tokenizer(original_tokenizer,text,path="simpledata/tokenizer"):
    words_unique = set(text.split(" "))
    for i in words_unique:
        original_tokenizer.add_tokens(i)
    original_tokenizer.save_pretrained(path)

tokenizer2 = GPT2Tokenizer.from_pretrained(os.path.join(experiment_path,experiment_name,"tokenizer_squad"))

train_set_baby=Dataset.from_dict({"text":[train_set["text"][0][0:50]]})
````

I then applied the dataset map function on a fairly small set of text:

```
%%time
train_set_baby = train_set_baby.map(lambda d:tokenizer2(d["text"]),batched=True)

```


The run time for train_set_baby.map was 6 seconds, and the batch itself was 2.6 seconds

**100% 1/1 [00:02<00:00, 2.60s/ba] CPU times: user 5.96 s, sys: 36 ms, total: 5.99 s Wall time: 5.99 s**

In comparison using (even after adding additional tokens): 
`
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")`

```
%%time
train_set_baby = train_set_baby.map(lambda d:tokenizer2(d["text"]),batched=True)

```
The time is 
**100% 1/1 [00:00<00:00, 34.09ba/s] CPU times: user 68.1 ms, sys: 16 µs, total: 68.1 ms Wall time: 62.9 ms**

It seems this might relate to the tokenizer save or load function, however, the issue appears to come up when I apply the loaded tokenizer  to the map function. 

I should also add that playing around with the amount of words I add to the tokenizer before I save it to disk and load it into memory  appears to impact the time it takes to run the map function. 


 
 Thanks for the experiments @johncookds and @wumpusman ! 

> Hi, which version of datasets has datasets.set_caching_enabled(False)?

Currently you have to install `datasets` from source to have this feature, but this will be available in the next release in a few days.

> I'm trying to figure out why the overhead of map would increase the time by double (figured it would be a fixed increase in time)? Though maybe this is expected behavior.

Could you also try with double the number of characters ? This should let us have an idea of the fixed cost (hashing) and the dynamic cost (actual tokenization, grows with the size of the input)

> @lhoestq, do you by chance know how I can redirect this issue to Tokenizer?

Feel free to post an issue on the `transformers` repo. Also I'm sure there should be related issues so you can also look for someone with the same concerns on the `transformers` repo.