ImageFolder add option to ignore some folders like '.ipynb_checkpoints'
{'login': 'cceyda', 'id': 15624271, 'node_id': 'MDQ6VXNlcjE1NjI0Mjcx', 'avatar_url': 'https://avatars.githubusercontent.com/u/15624271?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/cceyda', 'html_url': 'https://github.com/cceyda', 'followers_url': 'https://api.github.com/users/cceyda/followers', 'following_url': 'https://api.github.com/users/cceyda/following{/other_user}', 'gists_url': 'https://api.github.com/users/cceyda/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/cceyda/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/cceyda/subscriptions', 'organizations_url': 'https://api.github.com/users/cceyda/orgs', 'repos_url': 'https://api.github.com/users/cceyda/repos', 'events_url': 'https://api.github.com/users/cceyda/events{/privacy}', 'received_events_url': 'https://api.github.com/users/cceyda/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892871, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODcx', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/enhancement', 'name': 'enhancement', 'color': 'a2eeef', 'default': True, 'description': 'New feature or request'}]
[]
Maybe it would be nice to ignore private dirs like this one (ones starting with `.`) by default. 

CC @mariosasko  Maybe we can add a `ignore_hidden_files` flag to the builder configs of our packaged loaders (to be consistent across all of them), wdyt @lhoestq @albertvillanova?  I think they should always ignore them actually ! Not sure if adding a flag would be helpful @lhoestq But what if the user explicitly requests those files via regex?

`glob.glob` ignores hidden files (files starting with ".") by default unless they are explicitly requested, but fsspec's `glob` doesn't follow this behavior, which is probably a bug, so maybe we can raise an issue or open a PR in their repo? > @lhoestq But what if the user explicitly requests those files via regex?

Usually hidden files are meant to be ignored. If they are data files, they must be placed outside a hidden directory in the first place right ? I think it's more sensible to explain this than adding a flag.

> glob.glob ignores hidden files (files starting with ".") by default unless they are explicitly requested, but fsspec's glob doesn't follow this behavior, which is probably a bug, so maybe we can raise an issue or open a PR in their repo?

After globbing using `fsspec`, we already ignore files that start with a `.` in `_resolve_single_pattern_locally` and `_resolve_single_pattern_in_dataset_repository`, I guess we can just account for parent directories as well ?

We could open an issue on `fsspec` but I think they won't change this since it's an important breaking change for them.
**Is your feature request related to a problem? Please describe.**
I sometimes like to peek at the dataset images from jupyterlab. thus '.ipynb_checkpoints' folder appears where my dataset is and (just realized) leads to accidental duplicate image additions. I think this is an easy enough thing to miss especially if the dataset is very large.

**Describe the solution you'd like**
maybe have an option `ignore` or something .gitignore style
`dataset = load_dataset("imagefolder", data_dir="./data/original", ignore="regex?")`

**Describe alternatives you've considered**
Could filter out manually


{'url': 'https://api.github.com/repos/huggingface/datasets/issues/4115/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}