<commit_before># GCSFuse Performance Benchmarks

FIO is used to perform load tests on GCSFuse. Below tables shows
performance metrics of GCSFuse for different workloads for the given 
test setup:

## Test setup:

* Infra: GCP VM
* VM Type: n2-standard-48
* VM Bandwidth: 32Gbps
* VM location: us-central1-c
* GCS Bucket location: us-central1-c
* Framework: FIO
* Test runtime: 60sec

## Reads
### Sequential reads

| File Size | Bandwidth in GiB/sec |
|-----------|----------------------|
| 100KB     | 0.077                |
| 1MB       | 0.836                |
| 2MB       | 1.56                 |
| 5MB       | 3.55                 |
| 10MB      | 4.9                  |
| 50MB      | 6.39                 |
| 100MB     | 6.02                 |
| 200MB     | 6.17                 |
| 1GB       | 5.78                 |
| 5GB       | 6.23                 |

### Random reads

| File Size | Bandwidth in GiB/sec |
|-----------|----------------------|
| 100KB     | 0.079                |
| 1MB       | 0.812                |
| 2MB       | 1.23                 |
| 5MB       | 1.02                 |
| 10MB      | 0.9                  |
| 50MB      | 0.84                 |
| 100MB     | 0.82                 |
| 200MB     | 0.69                 |
| 1GB       | 0.52                 |
| 5GB       | 0.43                 |


### Recommendation for reads
GCSFuse performs well for sequential reads and recommendation is to use GCSFuse
for doing sequential reads on small files.<commit_msg>updated the metrics for us-west1<commit_after># GCSFuse Performance Benchmarks

FIO is used to perform load tests on GCSFuse. Below tables shows performance metrics of GCSFuse for different workloads for the given test setup:

## Test setup:

* Infra: GCP VM
* VM Type: n2-standard-48
* VM Bandwidth: 32Gbps
* OS: debian-11
* VM location: us-west1-c
* GCS Bucket location: us-west1
* Framework: FIO

### FIO Spec
* Test runtime: 60sec
* Thread count: 40
* Block Size: 1MB

### GCSFuse command
```
gcsfuse  --max-conns-per-host=100 --implicit-dirs --stat-cache-ttl=60s 
--type-cache-ttl=60s --disable-http2 <bucket-name> ~/gcs
```
## Reads
### Sequential reads

| File Size | Bandwidth in GiB/sec | IOPS | Avg Latency (msec) |
|-----------|----------------------|------|--------------------|
| 256KB     | 0.47                 | 1875 | 1350               |
| 1MB       | 1.56                 | 1554 | 1623               |
| 2MB       | 2.27                 | 2279 | 1110               |
| 5MB       | 3.75                 | 3796 | 670                |
| 10MB      | 4.02                 | 4071 | 625                |
| 50MB      | 4.37                 | 4433 | 574                |
| 100MB     | 4.33                 | 4390 | 579.94             |
| 200MB     | 4.31                 | 4371 | 582.89             |
| 1GB       | 4.32                 | 4383 | 580.67             |

### Random reads

| File Size | Bandwidth in GiB/sec | IOPS  | Avg Latency (msec) |
|-----------|----------------------|-------|--------------------|
| 256KB     | 0.45                 | 1780  | 1418               |
| 1MB       | 1.46                 | 1456  | 1728               |
| 2MB       | 2.00                 | 2010  | 1256               |
| 5MB       | 1.64                 | 1641  | 1539               |
| 10MB      | 1.44                 | 1435  | 1761               |
| 50MB      | 1.31                 | 1301  | 1934               |
| 100MB     | 1.07                 | 1056  | 2365               |
| 200MB     | 1.06                 | 1044  | 2405               |
| 1GB       | 0.91                 | 889   | 2809               |


### Recommendation for reads
GCSFuse performs well for sequential reads and recommendation is to use GCSFuse
for doing sequential reads on file sizes > 10MB and < 1GB.
{'fraction_non_alphanumeric': 0.14455917394757745, 'fraction_numerical': 0.09637278263171829, 'mean_word_length': 1.0127863612147043, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3053405', 'n_tokens_mistral': 1443, 'n_tokens_neox': 1106, 'n_words': 322}