`datasets.map` multi processing much slower than single processing 
{'login': 'hwijeen', 'id': 29157715, 'node_id': 'MDQ6VXNlcjI5MTU3NzE1', 'avatar_url': 'https://avatars.githubusercontent.com/u/29157715?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/hwijeen', 'html_url': 'https://github.com/hwijeen', 'followers_url': 'https://api.github.com/users/hwijeen/followers', 'following_url': 'https://api.github.com/users/hwijeen/following{/other_user}', 'gists_url': 'https://api.github.com/users/hwijeen/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/hwijeen/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/hwijeen/subscriptions', 'organizations_url': 'https://api.github.com/users/hwijeen/orgs', 'repos_url': 'https://api.github.com/users/hwijeen/repos', 'events_url': 'https://api.github.com/users/hwijeen/events{/privacy}', 'received_events_url': 'https://api.github.com/users/hwijeen/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892857, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODU3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/bug', 'name': 'bug', 'color': 'd73a4a', 'default': True, 'description': "Something isn't working"}]
[]
['Hi @hwijeen, you might want to look at issues #1796 and #1949. I think it could be something related to the I/O operations being performed.'
 'I see that many people are experiencing the same issue. Is this problem considered an "official" bug that is worth a closer look? @lhoestq'
 "Yes this is an official bug. On my side I haven't managed to reproduce it but @theo-m has. We'll investigate this !"
 'Thank you for the reply! I would be happy to follow the discussions related to the issue.\r\nIf you do not mind, could you also give a little more explanation on my p.s.2? I am having a hard time figuring out why the single processing `map` uses all of my cores.\r\n@lhoestq @theo-m '
 'Regarding your ps2: It depends what function you pass to `map`.\r\nFor example, fast tokenizers from `transformers` in Rust tokenize texts and parallelize the tokenization over all the cores.'
 'I am still experiencing this issue with datasets 1.9.0..\r\nHas there been a further investigation? \r\n<img width="442" alt="image" src="https://user-images.githubusercontent.com/29157715/126143387-8b5ddca2-a896-4e18-abf7-4fbf62a48b41.png">\r\n']
Hi, thank you for the great library.

I've been using datasets to pretrain language models, and it often involves datasets as large as ~70G.
My data preparation step is roughly two steps: `load_dataset` which splits corpora into a table of sentences, and `map` converts a sentence into a list of integers, using a tokenizer.

I noticed that `map` function with `num_proc=mp.cpu_count() //2` takes more than 20 hours to finish the job where as `num_proc=1` gets the job done in about 5 hours. The machine I used has 40 cores, with 126G of RAM. There were no other jobs when `map` function was running.

What could be the reason? I would be happy to provide information necessary to spot the reason.

p.s. I was experiencing the imbalance issue mentioned in [here](https://github.com/huggingface/datasets/issues/610#issuecomment-705177036) when I was using multi processing.
p.s.2 When I run `map` with `num_proc=1`, I see one tqdm bar but all the cores are working. When `num_proc=20`, only 20 cores work. 
![Screen Shot 2021-03-05 at 11 04 59](https://user-images.githubusercontent.com/29157715/110056895-ef6cf000-7da2-11eb-8307-6698e9fb1ad4.png)

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/1992/reactions', 'total_count': 1, '+1': 1, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}