<commit_before># Yet Another S3-backed File System: yas3fs

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3.

**This is a personal project. No relation whatsoever exists between this project and my employer.**

* It allows to mount an S3 bucket (or a part of it) as a local folder.
* For maximum speed all data read from S3 is cached locally on the node.
* Access to file content is provided during the download from S3 using buffers.
* SNS notifications are used to update other nodes that something has changed on S3 and they need to invalidate their cache.
* Notifications can be listened using HTTP or SQS endpoints.
* With buffering enabled (the default) files can be accessed during the download from S3.
* If the cache grows to its maximum size the least accessed files are removed.
* AWS credentials can be passed using AWS\_ACCESS\_KEY\_ID and AWS\_SECRET\_ACCESS\_KEY environmental variables.
* In an EC2 instance a IAM role can be used to give access to S3/SNS/SQS resources.

On EC2 the command line doesn't need any information on the actual server and can easily be used within an Auto Scaling group.
<commit_msg>First Public Release<commit_after># Yet Another S3-backed File System: yas3fs

YAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3.

**This is a personal project. No relation whatsoever exists between this project and my employer.**

* It allows to mount an S3 bucket (or a part of it, if you specify a path) as a local folder.
* For maximum speed all data read from S3 is cached in memory locally on the node.
* SNS notifications are used to update other nodes that something has changed on S3 and they need to invalidate their cache.
* Notifications can be listened using HTTP or SQS endpoints.
* With buffering enabled (the default) files can be accessed during the download from S3.
* If the cache grows to its maximum size, the least accessed files are removed.
* AWS credentials can be passed using AWS\_ACCESS\_KEY\_ID and AWS\_SECRET\_ACCESS\_KEY environmental variables.
* In an EC2 instance a IAM role can be used to give access to S3/SNS/SQS resources.

On EC2 the command line doesn't need any information on the actual server and can easily be used within an Auto Scaling group.

{'fraction_non_alphanumeric': 0.04304778303917348, 'fraction_numerical': 0.01076194575979337, 'mean_word_length': 4.222471910112359, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6213447', 'n_tokens_mistral': 611, 'n_tokens_neox': 590, 'n_words': 374}