Support remote cache_dir
{'login': 'albertvillanova', 'id': 8515462, 'node_id': 'MDQ6VXNlcjg1MTU0NjI=', 'avatar_url': 'https://avatars.githubusercontent.com/u/8515462?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/albertvillanova', 'html_url': 'https://github.com/albertvillanova', 'followers_url': 'https://api.github.com/users/albertvillanova/followers', 'following_url': 'https://api.github.com/users/albertvillanova/following{/other_user}', 'gists_url': 'https://api.github.com/users/albertvillanova/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/albertvillanova/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/albertvillanova/subscriptions', 'organizations_url': 'https://api.github.com/users/albertvillanova/orgs', 'repos_url': 'https://api.github.com/users/albertvillanova/repos', 'events_url': 'https://api.github.com/users/albertvillanova/events{/privacy}', 'received_events_url': 'https://api.github.com/users/albertvillanova/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['_The documentation is not available anymore as the PR was closed or merged._'
 '@lhoestq thanks for your review.\r\n\r\nPlease note that `xjoin` cannot be used in this context, as it always returns a POSIX path string and this is not suitable on Windows machines.'
 "<s>`xjoin` returns windows paths (not posix) on windows, since it just extends`os.path.join` </s>\r\n\r\nActually you are right.\r\n\r\nhttps://github.com/huggingface/datasets/blob/08ec04ccb59630a3029b2ecd8a14d327bddd0c4a/src/datasets/utils/streaming_download_manager.py#L104-L105\r\n\r\nThough this is not an issue because posix paths (as returned by Path().as_posix()) work on windows. That's why we can replace `os.path.join` with `xjoin` in streaming mode. They look like `c:/Program Files/` or something (can't confirm right now, I don't have a windows with me)"
 'Until now, we have always replaced "/" in paths with `os.path.join` (`os.sep`,...) in order to support Windows paths (that contain r"\\\\").\r\n\r\nNow, you suggest ignoring this and work with POSIX strings (with "/").\r\n\r\nAs an example, when passing `cache_dir=r"C:\\Users\\Username\\.mycache"`:\r\n- Until now, it results in `self._cache_downloaded_dir = r"C:\\Users\\Username\\.mycache\\downloads"`\r\n- If we use `xjoin`, it will give `self._cache_downloaded_dir = "C:/Users/Username/.mycache/downloads"`\r\n\r\nYou say this is OK and we don\'t care if we work with POSIX strings on Windows machines.\r\n\r\nI\'m incorporating your suggested changes then...'
 'Also note that using `xjoin`, if we pass `cache_dir="C:\\\\Users\\\\Username\\\\.mycache"`, we get:\r\n- `self._cache_dir_root = "C:\\\\Users\\\\Username\\\\.mycache"`\r\n- `self._cache_downloaded_dir = "C:/Users/Username/.mycache/downloads"`'
 'It looks like it broke the CI on windows :/ maybe this was not a good idea, sorry']
{'url': 'https://api.github.com/repos/huggingface/datasets/pulls/4347', 'html_url': 'https://github.com/huggingface/datasets/pull/4347', 'diff_url': 'https://github.com/huggingface/datasets/pull/4347.diff', 'patch_url': 'https://github.com/huggingface/datasets/pull/4347.patch', 'merged_at': datetime.datetime(2022, 5, 25, 16, 27, 3)}
This PR implements complete support for remote `cache_dir`. Before, the support was just partial.

This is useful to create datasets using Apache Beam (parallel data processing) builder with `cache_dir` in a remote bucket, e.g., for Wikipedia dataset.
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/4347/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}