[Bug] FileLock dependency incompatible with filesystem
{'login': 'jarednielsen', 'id': 4564897, 'node_id': 'MDQ6VXNlcjQ1NjQ4OTc=', 'avatar_url': 'https://avatars.githubusercontent.com/u/4564897?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/jarednielsen', 'html_url': 'https://github.com/jarednielsen', 'followers_url': 'https://api.github.com/users/jarednielsen/followers', 'following_url': 'https://api.github.com/users/jarednielsen/following{/other_user}', 'gists_url': 'https://api.github.com/users/jarednielsen/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/jarednielsen/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/jarednielsen/subscriptions', 'organizations_url': 'https://api.github.com/users/jarednielsen/orgs', 'repos_url': 'https://api.github.com/users/jarednielsen/repos', 'events_url': 'https://api.github.com/users/jarednielsen/events{/privacy}', 'received_events_url': 'https://api.github.com/users/jarednielsen/received_events', 'type': 'User', 'site_admin': False}
[]
[]
['Hi, can you give details on your environment/os/packages versions/etc?'
 'Environment is Ubuntu 18.04, Python 3.7.5, nlp==0.3.0, filelock=3.0.12.\r\n\r\nThe external volume is Amazon FSx for Lustre, and it by default creates files with limited permissions. My working theory is that FileLock creates a lockfile that isn\'t writable, and thus there\'s no way to acquire it by removing the .lock file. But Python is able to create new files and write to them outside of the FileLock package.\r\n\r\nWhen I attempt to use FileLock within a Docker container by writing to `/root/.cache/hello.txt`, it succeeds. So there\'s some permissions issue. But it\'s not a Docker configuration issue; I\'ve replicated it without Docker.\r\n```bash\r\necho "hello world" >> hello.txt\r\nls -l\r\n\r\n-rw-rw-r-- 1 ubuntu ubuntu 10 Jun 30 19:52 hello.txt\r\n```'
 'Looks like the `flock` syscall does not work on Lustre filesystems by default: https://github.com/benediktschmitt/py-filelock/issues/67.\r\n\r\nI added the `-o flock` option when mounting the filesystem, as [described here](https://docs.aws.amazon.com/fsx/latest/LustreGuide/getting-started-step2.html), which fixed the issue.'
 'Awesome, thanks a lot for sharing your fix!'
 "I'm wondering if this can be revisited. In some managed environments the same person using HF cannot change the file-system mount flags, (and the organization may be unwilling to change these flags due to other concerns) but can ensure that there won't be concurrent writes, for example because HF is offline and the models/datasets were downloaded earlier. \r\n\r\nThe real fix would be to FileLock itself, which does not seem very active and seems to not deal with failed system flock calls , which would be one way to fix this, as they mention in the issue below also raised by @jarednielsen \r\n\r\nhttps://github.com/tox-dev/py-filelock/issues/67"]
I'm downloading a dataset successfully with
`load_dataset("wikitext", "wikitext-2-raw-v1")`

But when I attempt to cache it on an external volume, it hangs indefinitely:
`load_dataset("wikitext", "wikitext-2-raw-v1", cache_dir="/fsx") # /fsx is an external volume mount`

The filesystem when hanging looks like this:
```bash
/fsx
----downloads
       ----94be...73.lock
----wikitext
       ----wikitext-2-raw
             ----wikitext-2-raw-1.0.0.incomplete
```

It appears that on this filesystem, the FileLock object is forever stuck in its "acquire" stage. I have verified that the issue lies specifically with the `filelock` dependency:
```python
open("/fsx/hello.txt").write("hello") # succeeds

from filelock import FileLock
with FileLock("/fsx/hello.lock"):
    open("/fsx/hello.txt").write("hello") # hangs indefinitely
```

Has anyone else run into this issue? I'd raise it directly on the FileLock repo, but that project appears abandoned with the last update over a year ago. Or if there's a solution that would remove the FileLock dependency from the project, I would appreciate that.
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/329/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}