<commit_before>Wire Format abstraction library
===

# Purpose

Chronicle Wire combines a number of concerns in a consistent manner.

- Application configuration.
- Data serialization
- Accessing off heap memory in a thread safe manner.
- High performance data exchange via binary formats.

## Why are these concerns conflated?
Often you want to use these interchangeably.

- to send the configuration of a server to a client or visa-versa.
- to store the configuration of a data store in it's header.
- in configuration be able to create any object or component.
- save a configuration after you have changed it.
- to be able to share data in memory between processes in a thread safe manner.

## Design

Chronicle Wire supports a separation of describing what data you want to store and retrieve
   and how it should be rendered/parsed.
   Wire handles a variety of formatting options for a wide range of formats.

A key aim of Wire is to support schema changes.  It should make reasonable 
    attempts to handle
* optional fields
* fields in a different order.
* fields the consumer doesn't expect. Optionally parsing them or ignoring them.
* more or less data than expected (in field-less formats) 
* reading a different type to the one written
* updating fixed length fields, atomically where possible via a "bound" data structure..
<commit_msg>Add documentation of how Wire can be used for configuration.<commit_after>Wire Format abstraction library
===

# Purpose

Chronicle Wire combines a number of concerns in a consistent manner.

- Application configuration. (Using YAML)
- Data serialization (YAML, binary YAML, JSON, Raw binary data)
- Accessing off heap memory in a thread safe manner. (Bind to shared off heap memory)
- High performance data exchange via binary formats. (Only include as much meta data as you need)

## Why are these concerns conflated?
Often you want to use these interchangeably.

- configuration includes aliased type information.  This supports easy extension through adding new classes/versions and cross platform through type aliasing.
- by supporting types, a configuration file can bootstrap itself. You control how the configuration file is decoded. [engine.yaml](https://github.com/OpenHFT/Chronicle-Engine/blob/master/demo/src/main/resources/engine.yaml)
- to send the configuration of a server to a client or visa-versa.
- to store the configuration of a data store in it's header.
- in configuration be able to create any object or component.
- save a configuration after you have changed it.
- to be able to share data in memory between processes in a thread safe manner.

## Design

Chronicle Wire supports a separation of describing what data you want to store and retrieve
   and how it should be rendered/parsed.
   Wire handles a variety of formatting options for a wide range of formats.

A key aim of Wire is to support schema changes.  It should make reasonable 
    attempts to handle
* optional fields
* fields in a different order.
* fields the consumer doesn't expect. Optionally parsing them or ignoring them.
* more or less data than expected (in field-less formats) 
* reading a different type to the one written
* updating fixed length fields, atomically where possible via a "bound" data structure..

{'fraction_non_alphanumeric': 0.04595588235294118, 'fraction_numerical': 0.0, 'mean_word_length': 4.077760497667185, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1130586', 'n_tokens_mistral': 790, 'n_tokens_neox': 764, 'n_words': 476}