<commit_before># Changelog

## 1.4.0 (upcoming)

* Pending changelog

## 1.3.0 (April 07, 2017)

* Sparta dispatcher added with Marathon API
* Sparta driver added in Marathon execution mode 
* Diver and Plugins jars endpoints
* Executions properties and unified spark submit arguments and configurations
* Policy view with important policy data
* Settings step in wizard
* Support user, groups and permissions with Oauth2 library
* Delete checkpoint option
* Sparta config refactor
* Docker files refactor
* Bugfix: fragments duplication<commit_msg>Changelog 1.4 (#1843)<commit_after># Changelog

## 1.4.0 (upcoming)

* Update to Spark 2.1
* Integration with HDFS securized
* Integration with Kafka securized
* Integration with Zookeeper securized
* Save Kafka offsets in Kafka with Spark 2.1 API
* Save transformations with writer
* Save Raw Data step
* Execution information with Spark properties
* Updated SSO with GoSec
* Added Postgres options

## 1.3.0 (April 07, 2017)

* Sparta dispatcher added with Marathon API
* Sparta driver added in Marathon execution mode 
* Diver and Plugins jars endpoints
* Executions properties and unified spark submit arguments and configurations
* Policy view with important policy data
* Settings step in wizard
* Support user, groups and permissions with Oauth2 library
* Delete checkpoint option
* Sparta config refactor
* Docker files refactor
* Bugfix: fragments duplication
{'fraction_non_alphanumeric': 0.05434015525758645, 'fraction_numerical': 0.025405786873676783, 'mean_word_length': 5.008474576271187, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1294576', 'n_tokens_mistral': 420, 'n_tokens_neox': 365, 'n_words': 178}