<commit_before>---
layout: page
title: Release notes
---
#Release notes
## 2.0
### 2.0.1
1. Performance improvements while parsing delimited (CSV) sources. Performance in normal scenario should be improved 
by at least 50%. Both CPU and memory impact has been significantly improved.
1. Default cell cache size while parsing is now 1 (instead of 10). 
1. Changed behaviour when parsing quoted delimited sources. When a start quote is found but no end quote within 8k of data, the parser now
tries again to parse the source but considers that particular cell as not being quoted.
<commit_msg>Added release notes about performance improvements and configurable max line length.<commit_after>---
layout: page
title: Release notes
---
#Release notes
## 2.0
### 2.0.1
1. Performance improvements while parsing delimited (CSV) sources and fixed width sources where lines are separated. 
Performance in normal scenario should be improved 
by at least 50%. Both CPU and memory impact has been significantly improved.
1. Default cell cache size while parsing is now 1 (instead of 10). 
1. Changed behaviour when parsing quoted delimited sources. When a start quote is found but no end quote within 8k of data, the parser now
tries again to parse the source but considers that particular cell as not being quoted.
1. It is now possible to configure maximum line length while parsing. 

{'fraction_non_alphanumeric': 0.05309090909090909, 'fraction_numerical': 0.02109090909090909, 'mean_word_length': 4.212121212121212, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '730086', 'n_tokens_mistral': 356, 'n_tokens_neox': 324, 'n_words': 212}