<commit_before># Data Migration

1. [Overview](#overview)
2. [How it works](#how-it-works)
2. [Data migration in the browser](data-migration-in-the-browser)
3. [Data migration on the server](#data-migration-on-the-server)

## Overview

Colonel Kurtz 2.0.0 marked the first change to the structure of its
output. To handle old data, a migration tool has been added.

```
Upgrading Colonel Kurtz from version 0.0.0 to 2.0.0
✓ 2.0.0
```

For future releases, all data is tagged with the version of the editor
with which it was saved. You can view this in the output under the
`system` key:

```json
{
  "system" : { "version" : "2.0.0" },
  "blocks" : []
}
```

## How it works

Colonel Kurtz checks the [semver](http://semver.org/) for the provided
data. If no version is provided, it assumes the data is from release
before versioning was added. **Although semver is in place, ideally
changes the data structure will occur in a major release**.

## Data migration in the browser

**Data will automatically migrate to the latest version on
startup**.

## Data migration on the server

Colonel Kurtz also includes a binary that will perform the same
operations as the browser behavior.

This binary requires [node.js](nodejs.org).

```shell
$(npm bin)/colonel_kurtz oldData
```

`$(npm bin)` returns the project localized bin directory for node
packages. By default, this is under `./node_modules/.bin`. The binary
will update the JSON and provide STDOUT with the modernized
structure.<commit_msg>[update] Migration docs<commit_after># Data Migration

1. [Overview](#overview)
2. [How it works](#how-it-works)
2. [Data migration in the browser](data-migration-in-the-browser)
3. [Data migration on the server](#data-migration-on-the-server)

## Overview

Colonel Kurtz 2.0.0 marked the first change to the structure of its
output. To handle old data, a migration tool has been added.

For future releases, all data is tagged with the version of the editor
with which it was saved. You can view this in the output under the
`system` key:

```json
{
  "system" : { "version" : "2.0.0" },
  "blocks" : []
}
```

## How it works

Colonel Kurtz checks the [semver](http://semver.org/) for the provided
data. If no version is provided, it assumes the data is from release
before versioning was added. **Although semver is in place, ideally
changes the data structure will occur in a major release**.

## Data migration in the browser

Data will automatically migrate to the latest version on startup. When
Colonel Kurtz starts, it will pass its seed data through the migration
plugin.

```javascript
let instance = new ColonelKurtz({
  el   : document.querySelector('#app'),
  seed : document.querySelector('#truth').value
})

instance.start() // runs through all plugins, including migration
```

## Data migration on the server

Colonel Kurtz also includes a binary that will perform the same
operations as the browser behavior.

This binary requires [node.js](nodejs.org).

```shell
$(npm bin)/colonel_kurtz oldData

Upgrading Colonel Kurtz from version 0.0.0 to 2.0.0
✓ 2.0.0
```

`$(npm bin)` returns the project localized bin directory for node
packages. By default, this is under `./node_modules/.bin`. The binary
will update the JSON and provide STDOUT with the modernized
structure.
{'fraction_non_alphanumeric': 0.09963436928702012, 'fraction_numerical': 0.01157830591102986, 'mean_word_length': 4.227707006369426, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6492601', 'n_tokens_mistral': 988, 'n_tokens_neox': 932, 'n_words': 445}