<commit_before># Performance Tuning

## Performance Comparison

All tests are performed using v1.5 alpha, in the same initial conditions on Intel i7 CPU with 16GB RAM.

It's worth noting that parquet-dotnet has never been optimised for performance yet.

### Case 1

**File:** customer.impala.parquet

**Size:** 25Mb

**Rows**: 150'000

||Read|Write (uncompressed)|Write (gzip)|Write (snappy)|
|-|-|-|
|fastparquet|273 ms|266 ms|5'633 ms|307 ms|
|parquet-dotnet|354 ms|351 ms|1'317 ms|351 ms|
|Scala (Spark)|-|-|-|-|


![Perf00](img/perf00.png)

In this test Apache Spark cannot read the file due to an internal bug.

As you can see, Parquet.Net loses to fastparquet slightly in everything except for writing gzip compressed parquet files.<commit_msg>table<commit_after># Performance Tuning

## Performance Comparison

All tests are performed using v1.5 alpha, in the same initial conditions on Intel i7 CPU with 16GB RAM.

It's worth noting that parquet-dotnet has never been optimised for performance yet.

### Case 1

**File:** customer.impala.parquet

**Size:** 25Mb

**Rows**: 150'000

||Read|Write (uncompressed)|Write (gzip)|Write (snappy)|
|-|-|-|-|-|
|fastparquet|273 ms|266 ms|5'633 ms|307 ms|
|parquet-dotnet|354 ms|351 ms|1'317 ms|351 ms|
|Scala (Spark)|-|-|-|-|


![Perf00](img/perf00.png)

In this test Apache Spark cannot read the file due to an internal bug.

As you can see, Parquet.Net loses to fastparquet slightly in everything except for writing gzip compressed parquet files.
{'fraction_non_alphanumeric': 0.12299465240641712, 'fraction_numerical': 0.058823529411764705, 'mean_word_length': 4.870588235294117, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1829744', 'n_tokens_mistral': 564, 'n_tokens_neox': 479, 'n_words': 187}