<commit_before># Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- Added support for document encoding. 

### Fixed

- Fixed issue parsing apostrophe control symbol, where a conditional was using the wrong symbol name.
- Fixed issue parsing other tokens, where they caused a variable-not-defined exception to be thrown during parsing.

## [0.5.0] - 2021-11-04

### Added

- Added support for PHP 7.4 and 8.0.
- Added continuous integration with [CircleCI](https://circleci.com/gh/jstewmc/rtf).
- Added code coverage analysis with [CodeCov](https://codecov.io/gh/jstewmc/rtf).
- Added coding standards with [slevomat/coding-standard](https://github.com/slevomat/coding-standard).
- Added vulnerability checking with [roave/security-advisories](https://github.com/Roave/SecurityAdvisories).

### Changed

- Modernized the library (i.e., add type hints, refactor big methods, move to services, add guard clauses, etc).
- Updated test suite from PHPUnit version 4 to version 9.
- Extended `jstewmc/stream` with custom methods to make lexing easier.

### Fixed

- Fixed [#1](https://github.com/jstewmc/rtf/issues/1), where the classnames in this library's PHPDoc comments were incorrect and prevented IDE code completion.
- Fixed [#3](https://github.com/jstewmc/rtf/issues/3), where installation instructions were missing from this library's README.

<commit_msg>Cut version 0.5.1<commit_after># Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.5.1] - 2021-11-06

### Added

- Added support for document encoding. If a document defines a character set (i.e., `\pc`, `\pca`, `\mac`) or it defines the `\ansi` character set and a code page (e.g., `\ansicpg1252`), this library will use that character encoding to render non-ASCII characters.

### Fixed

- Fixed issue parsing apostrophe control symbol, where a conditional was using the wrong symbol name.
- Fixed issue parsing other tokens, where they caused a variable-not-defined exception to be thrown during parsing.
- Fixed [#2](https://github.com/jstewmc/rtf/issues/2), where the `Apostrophe` control symbol did not work correctly.

## [0.5.0] - 2021-11-04

### Added

- Added support for PHP 7.4 and 8.0.
- Added continuous integration with [CircleCI](https://circleci.com/gh/jstewmc/rtf).
- Added code coverage analysis with [CodeCov](https://codecov.io/gh/jstewmc/rtf).
- Added coding standards with [slevomat/coding-standard](https://github.com/slevomat/coding-standard).
- Added vulnerability checking with [roave/security-advisories](https://github.com/Roave/SecurityAdvisories).

### Changed

- Modernized the library (i.e., add type hints, refactor big methods, move to services, add guard clauses, etc).
- Updated test suite from PHPUnit version 4 to version 9.
- Extended `jstewmc/stream` with custom methods to make lexing easier.

### Fixed

- Fixed [#1](https://github.com/jstewmc/rtf/issues/1), where the classnames in this library's PHPDoc comments were incorrect and prevented IDE code completion.
- Fixed [#3](https://github.com/jstewmc/rtf/issues/3), where installation instructions were missing from this library's README.


{'fraction_non_alphanumeric': 0.11918850380388842, 'fraction_numerical': 0.020850943927867006, 'mean_word_length': 4.228276877761414, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 17, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3335291', 'n_tokens_mistral': 1167, 'n_tokens_neox': 1062, 'n_words': 380}