<commit_before># Migration Guide (Released)
All migrations of webKnossos are documented in this file.
See `MIGRATIONS.unreleased.md` for the changes which are not yet part of an official release.

This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
User-facing changes are documented in the [changelog](CHANGELOG.released.md).

## [22.09.0](https://github.com/scalableminds/webknossos/releases/tag/22.09.0) - 2022-08-25
[Commits](https://github.com/scalableminds/webknossos/compare/22.08.0...22.09.0)

- webKnossos requires node 16 now. [#6350](https://github.com/scalableminds/webknossos/pull/6350)
- Removed the foreign datastore feature. If you have any foreign datastores registered in your webKnossos, running this upgrade may result in undefined behavior. [#6392](https://github.com/scalableminds/webknossos/pull/6392)

### Postgres Evolutions:
- [084-annotation-contributors.sql](conf/evolutions/084-annotation-contributors.sql)
- [085-add-annotations-publicationforeignkey](conf/evolutions/085-add-annotations-publicationforeignkey.sql)
- [086-drop-foreign-datastores.sql](conf/evolutions/086-drop-foreign-datastores.sql.sql)
- [087-zarr-private-links](conf/evolutions/087-zarr-private-links.sql)


## [22.08.0](https://github.com/scalableminds/webknossos/releases/tag/22.08.0) - 2022-07-20
[Commits](https://github.com/scalableminds/webknossos/compare/22.07.0...22.08.0)

 - Postgres evolution 83 (see below) introduces unique and url-safe constraints for annotation layer names. If the database contains entries violating those new constraints, they need to be fixed manually, otherwise the evolution will abort:
    - change null names to the front-end-side defaults:
        ```
        update webknossos.annotation_layers set name = 'Volume' where name is null and typ = 'Volume'
        update webknossos.annotation_layers set name = 'Skeleton' where name is null and typ = 'Skeleton'
        ```

    - find annotations with multiple layers, make unique manually
        ```
        select _annotation, name from webknossos.annotation_layers where _annotation in (select s._annotation from
        (select _annotation, count(_annotation) from webknossos.annotation_layers where typ = 'Volume' group by _annotation order by count(_annotation) desc limit 1000) as s
        where count > 1) and typ = 'Volume' order by _annotation
        ```

   - find layers with interesting names, manually remove spaces and special characters
        ```
        select * from webknossos.annotation_layers where not name ~* '^[A-Za-z0-9\-_\.]+$'
        ```

### Postgres Evolutions:

- [083-unique-layer-names.sql](conf/evolutions/083-unique-layer-names.sql) Note: Note that this evolution introduces constraints which may not be met by existing data. See above for manual steps


## [22.07.0](https://github.com/scalableminds/webknossos/releases/tag/22.07.0) - 2022-06-28<commit_msg>Migrations: fix sql links (#6435)<commit_after># Migration Guide (Released)
All migrations of webKnossos are documented in this file.
See `MIGRATIONS.unreleased.md` for the changes which are not yet part of an official release.

This project adheres to [Calendar Versioning](http://calver.org/) `0Y.0M.MICRO`.
User-facing changes are documented in the [changelog](CHANGELOG.released.md).

## [22.09.0](https://github.com/scalableminds/webknossos/releases/tag/22.09.0) - 2022-08-25
[Commits](https://github.com/scalableminds/webknossos/compare/22.08.0...22.09.0)

- webKnossos requires node 16 now. [#6350](https://github.com/scalableminds/webknossos/pull/6350)
- Removed the foreign datastore feature. If you have any foreign datastores registered in your webKnossos, running this upgrade may result in undefined behavior. [#6392](https://github.com/scalableminds/webknossos/pull/6392)

### Postgres Evolutions:
- [084-annotation-contributors.sql](conf/evolutions/084-annotation-contributors.sql)
- [085-annotation-publication.sql](conf/evolutions/085-annotation-publication.sql)
- [086-drop-foreign-datastores.sql](conf/evolutions/086-drop-foreign-datastores.sql)
- [087-zarr-private-links.sql](conf/evolutions/087-zarr-private-links.sql)


## [22.08.0](https://github.com/scalableminds/webknossos/releases/tag/22.08.0) - 2022-07-20
[Commits](https://github.com/scalableminds/webknossos/compare/22.07.0...22.08.0)

 - Postgres evolution 83 (see below) introduces unique and url-safe constraints for annotation layer names. If the database contains entries violating those new constraints, they need to be fixed manually, otherwise the evolution will abort:
    - change null names to the front-end-side defaults:
        ```
        update webknossos.annotation_layers set name = 'Volume' where name is null and typ = 'Volume'
        update webknossos.annotation_layers set name = 'Skeleton' where name is null and typ = 'Skeleton'
        ```

    - find annotations with multiple layers, make unique manually
        ```
        select _annotation, name from webknossos.annotation_layers where _annotation in (select s._annotation from
        (select _annotation, count(_annotation) from webknossos.annotation_layers where typ = 'Volume' group by _annotation order by count(_annotation) desc limit 1000) as s
        where count > 1) and typ = 'Volume' order by _annotation
        ```

   - find layers with interesting names, manually remove spaces and special characters
        ```
        select * from webknossos.annotation_layers where not name ~* '^[A-Za-z0-9\-_\.]+$'
        ```

### Postgres Evolutions:

- [083-unique-layer-names.sql](conf/evolutions/083-unique-layer-names.sql) Note: Note that this evolution introduces constraints which may not be met by existing data. See above for manual steps


## [22.07.0](https://github.com/scalableminds/webknossos/releases/tag/22.07.0) - 2022-06-28
{'fraction_non_alphanumeric': 0.11658653846153846, 'fraction_numerical': 0.04635989010989011, 'mean_word_length': 4.065217391304348, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 14, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2320697', 'n_tokens_mistral': 1964, 'n_tokens_neox': 1743, 'n_words': 479}