Caching processed dataset at wrong folder
{'login': 'mrm8488', 'id': 3653789, 'node_id': 'MDQ6VXNlcjM2NTM3ODk=', 'avatar_url': 'https://avatars.githubusercontent.com/u/3653789?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/mrm8488', 'html_url': 'https://github.com/mrm8488', 'followers_url': 'https://api.github.com/users/mrm8488/followers', 'following_url': 'https://api.github.com/users/mrm8488/following{/other_user}', 'gists_url': 'https://api.github.com/users/mrm8488/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/mrm8488/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/mrm8488/subscriptions', 'organizations_url': 'https://api.github.com/users/mrm8488/orgs', 'repos_url': 'https://api.github.com/users/mrm8488/repos', 'events_url': 'https://api.github.com/users/mrm8488/events{/privacy}', 'received_events_url': 'https://api.github.com/users/mrm8488/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892857, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODU3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/bug', 'name': 'bug', 'color': 'd73a4a', 'default': True, 'description': "Something isn't working"}]
[]
['Thanks for reporting !\r\nIt uses a temporary file to write the data.\r\nHowever it looks like the temporary file is not placed in the right directory during the processing'
 'Well actually I just tested and the temporary file is placed in the same directory, so it should work as expected.\r\nWhich version of `datasets` are you using ?'
 '`datasets-1.0.1`\r\nHere you can reproduce it here:\r\nhttps://colab.research.google.com/drive/1O0KcepTFsmpkBbrbLLMq42iwTKmQh8d5?usp=sharing\r\n'
 'It looks like a pyarrow issue with google colab.\r\nFor some reason this code increases the disk usage of google colab while it actually writes into google drive:\r\n\r\n```python\r\nimport pyarrow as pa\r\n\r\nstream = pa.OSFile("/content/drive/My Drive/path/to/file.arrow", "wb")\r\nwriter = pa.RecordBatchStreamWriter(stream, schema=pa.schema({"text": pa.string()}))\r\nwriter.write_table(pa.Table.from_pydict({"text": ["a"*511 + "\\n"] * ((1 << 30) // 512)}))  # 1GiB\r\nwriter.close()\r\nstream.close()\r\n```\r\n\r\nMoreover if I `rm` the file on google drive, it frees disk space on google colab.'
 "It looks like replacing `pa.OSFile` by `open` fixes it, I'm going to open a PR"
 'Ok. Thank you so much!'
 "Actually I did more tests it doesn't >.<\r\nI'll let you know if I find a way to fix that"
 'Actually I also have the issue when writing a regular text file\r\n\r\n```python\r\nf = open("/content/drive/My Drive/path/to/file", "w")\r\nf.write(("a"*511 + "\\n") * ((1 << 30) // 512))  # 1GiB\r\nf.close()\r\n```\r\n\r\nIs that supposed to happen ?'
 "The code you wrote should write a 1GB file in the Google Drive folder. Doesn't it? "
 'Yes it does, but the disk usage of google colab also increases by 1GB'
 'I could check it and as you say as I write to te Drive disk the colab disk also increases...'
 'To reproduce it: \r\n```bash\r\n!df -h | grep sda1\r\n```\r\n```python\r\nf = open("/content/drive/My Drive/test_to_remove.txt", "w")\r\nf.write(("a"*511 + "\\n") * ((1 << 30) // 512))  # 1GiB\r\nf.write(("a"*511 + "\\n") * ((1 << 30) // 512))  # 1GiB\r\nf.close()\r\n```\r\n```bash\r\n!ls -lh /content/drive/My\\ Drive/test_to_remove.txt\r\n\r\n!df -h | grep sda1\r\n\r\n!rm -rf /content/drive/My\\ Drive/test_to_remove.txt\r\n\r\n```\r\n[Colab](https://colab.research.google.com/drive/1D0UiweCYQwwWZ65EEhuqqbaDDbhJYXfm?usp=sharing)\r\n\r\n\r\n'
 'Apparently, Colab uses a local cache of the data files read/written from Google Drive. See:\r\n- https://github.com/googlecolab/colabtools/issues/2087#issuecomment-860818457\r\n- https://github.com/googlecolab/colabtools/issues/1915#issuecomment-804234540\r\n- https://github.com/googlecolab/colabtools/issues/2147#issuecomment-885052636']
Hi guys, I run this on my Colab (PRO):

```python
from datasets import load_dataset
dataset = load_dataset('text', data_files='/content/corpus.txt', cache_dir='/content/drive/My Drive', split='train')

def encode(examples):
  return tokenizer(examples['text'], truncation=True, padding='max_length')

dataset = dataset.map(encode, batched=True)
```
The file is about 4 GB, so I cannot process it on the Colab HD because there is no enough space. So I decided to mount my Google Drive fs and do it on it.
The dataset is cached in the right place but by processing it (applying `encode` function) seems to use a different folder because Colab HD starts to grow and it crashes when it should be done in the Drive fs.

What gets me crazy, it prints it is processing/encoding the dataset in the right folder:
```
Testing the mapped function outputs
Testing finished, running the mapping function on the dataset
Caching processed dataset at /content/drive/My Drive/text/default-ad3e69d6242ee916/0.0.0/7e13bc0fa76783d4ef197f079dc8acfe54c3efda980f2c9adfab046ede2f0ff7/cache-b16341780a59747d.arrow
```
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/643/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}