<commit_before># Migration Guide : CzechIdM 9.7.x to CzechIdM 10.0.x

## 💡 Introduction

This guide describes the various things that are needed when migrating from CzechIdM version 9.7.x to version 10.
In version 10 were upgraded major devstack dependencies (see list bellow).

The goals of this version:
- Fix known issues with newer versions of currently used third party libraries (e.g. ModelMapper). More than 100 vulnerabilities are removed.
- To be up to date. Some third party libraries cannot be used with our old devstack.
- Remove obsolete deprecated classes and methods.

Why is migration good for you:
- Frontend localization can be changed without refresh is needed.
- Script textarea contains suggestion box (thx to newer library).
- Frontend performance was increased.
- Logger level can be changed by application configuration without restart is needed.

# 🌗 Backend
In this chapter will be describe migration for the backend part of IdM.

> Note for **administator**:  is needed to read [Before ugrade](#before-upgrade) and [Configuration section](#configuration-properties).

> Note for **module developer**: is needed to read [Update custom module guide](#update-custom-module) and related conceptual and breaking changes.

> Note for **product developer:** is needed to read it all :).

### 🚀 Upgraded libraries

- Spring Boot ``1.3.8.RELEASE`` => ``2.1.7.RELEASE``
  - Spring ``4.2.8.RELEASE`` => ``5.1.9.RELEASE``
  - Spring Security ``4.0.4.RELEASE`` => ``5.1.6.RELEASE``
  - Spring Data ``1.9.5.RELEASE`` => ``2.1.10.RELEASE``
  - Hibernate ``4.3.11.Final`` => ``5.3.10.Final``
  - Spring Data Rest removed at all
- Activiti ``5.22.0`` => ``6.0.0``
- Groovy ``2.4.7`` => ``2.5.8``
 - Groovy Sandbox ``1.11`` => ``1.19``
- ModelMapper ``0.7.8`` => ``2.3.5``
- Guava ``18.0`` => ``28.1-jre``
- Swagger ``2.7.0`` => ``2.9.2``
- Forest index ``0.3.0`` => ``1.1.1``
- ... *other minor and third party libraries*.

> Note for developer: We are using openjdk and Tomcat 9 for development.

#### Removed libraries

- **Spring Data Rest** - we are using Spring rest controllers.
- **Joda** time - and related libraries for json and dababase support.
- Spring Boot **Websocket** - websocket support removed.
- **org.codehaus.jackson** - we are usign **com.fasterxml.jackson** only.

## Before upgrade

- Delete all persisted events. Can be used:
  - task ``DeleteExecutedEventTaskExecutor``
  - or button on event agenda (available for super admin users).
- Resolve all workflow task (complete or cancel). New workflow definitions version will be deployed automatically. Old workflow tasks will be alive, but its saver to resolve them before new version is deployed. Historic tasks and processes will be preserved untouched and their detail will be available.

## Database migration

When the CzechIdM version 10 is started for the first time, it will do an automatic update (as with any 9 version) to the schema by provided Flyway change scripts.

> Note for developer: Default database name configured for the all ``dev`` profiles was renamed to ``bcv_idm_10`` to prevent update (by Flyway) old database on the background - **old database can be used for LTS version 9.7.x development**, so clone database is needed.

## Configuration properties

Based on upgraded libraries we have to add and remove configuration properties (mainly related to connection pool and hibernate).

### Added properties

- ``spring.jpa.properties.hibernate.jdbc.time_zone=UTC`` - datetime will be persisted in UTC in database.
- ``spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true`` - disable warning during the boot, when Hibernate tries to retrieve some meta information from the database.
- ``spring.datasource.hikari.maximumPoolSize=25`` - enlarge pool size by default. This property should be revised for each project.
- ``spring.jpa.properties.hibernate.session_factory.interceptor=eu.bcvsolutions.idm.core.model.repository.listener.AuditableInterceptor`` - replaced deprecated Hiberante property.
- ``spring.jpa.hibernate.use-new-id-generator-mappings=false`` - Spring boot 2 changed default to ``true``, but we are using ``IDENTITY`` identifier generators for mssql database.


### Removed properties

- ``spring.jpa.properties.jadira.usertype.autoRegisterUserTypes`` - we are using java time now, so configuration for Joda time is not needed.
- ``spring.jpa.properties.hibernate.ejb.interceptor`` - replaced by new property above.

## Conceptual changes

- Spring repository queries, which updates audited entity (``@Modifying`` annotation) is prohibited, because audit is skipped this way. Use entity service (find, then update). All product provided repository methods are removed in version 10 (see changelog).

## Breaking changes

- **Joda time library was removed** - all entities, dtos and services use java time now ⇒ api was changed, all places which used joda time have to be refactored (included workflow and groovy scripts). The related issue is with [serialized dtos](#serialized-dtos) in workflow properties and operation result.
- **Activiti** 6 registered new ``formService`` bean usable in workflow definition ⇒ IdM eav service is not available under ``formService`` name any more. New bean alias ``idmFormService`` was created and has to be used in workflows.
- **Hibernate** removed data type ``org.hibernate.type.StringClobType`` - all entities has to be refactored, type ``org.hibernate.type.TextType`` has to be used.
- **Envers** changed api for extending criteria (check ``AuditCriterion`` usage).
- **Mockito** changes behavior for ``any(Class)`` checker - doesn't support ``null`` parameter value now. This is used just in unit tests. Test can be compiled but doesn't work.
- **Spring** data repository api changes:
  - e.g. ``findOne`` renamed to ``findOneById`` and returns ``Optional<E>``,
  - the concept is changed the same way for all methods.

## Update custom module

Due to breaking changes above, custom module requires some refactoring, before it's compatible with CzechIdM version 10. Some refactoring can be done with replaces, but some places has to be changed manually.

### Replaces

> Case sensitive find is expected.

- ``org.hibernate.type.StringClobType`` ⇒ ``org.hibernate.type.TextType``<commit_msg>Note about supported PostgreSQL versions<commit_after># Migration Guide : CzechIdM 9.7.x to CzechIdM 10.0.x

## 💡 Introduction

This guide describes the various things that are needed when migrating from CzechIdM version 9.7.x to version 10.
In version 10 were upgraded major devstack dependencies (see list bellow).

The goals of this version:
- Fix known issues with newer versions of currently used third party libraries (e.g. ModelMapper). More than 100 vulnerabilities are removed.
- To be up to date. Some third party libraries cannot be used with our old devstack.
- Remove obsolete deprecated classes and methods.

Why is migration good for you:
- Frontend localization can be changed without refresh is needed.
- Script textarea contains suggestion box (thx to newer library).
- Frontend performance was increased.
- Logger level can be changed by application configuration without restart is needed.

# 🌗 Backend
In this chapter will be describe migration for the backend part of IdM.

> Note for **administator**:  is needed to read [Before ugrade](#before-upgrade) and [Configuration section](#configuration-properties).

> Note for **module developer**: is needed to read [Update custom module guide](#update-custom-module) and related conceptual and breaking changes.

> Note for **product developer:** is needed to read it all :).

### 🚀 Upgraded libraries

- Spring Boot ``1.3.8.RELEASE`` => ``2.1.7.RELEASE``
  - Spring ``4.2.8.RELEASE`` => ``5.1.9.RELEASE``
  - Spring Security ``4.0.4.RELEASE`` => ``5.1.6.RELEASE``
  - Spring Data ``1.9.5.RELEASE`` => ``2.1.10.RELEASE``
  - Hibernate ``4.3.11.Final`` => ``5.3.10.Final``
  - Spring Data Rest removed at all
- Flyway ``4.0`` => ``5.2.4``
- Activiti ``5.22.0`` => ``6.0.0``
- Groovy ``2.4.7`` => ``2.5.8``
 - Groovy Sandbox ``1.11`` => ``1.19``
- ModelMapper ``0.7.8`` => ``2.3.5``
- Guava ``18.0`` => ``28.1-jre``
- Swagger ``2.7.0`` => ``2.9.2``
- Forest index ``0.3.0`` => ``1.1.1``
- ... *other minor and third party libraries*.

> Note for developer: We are using openjdk and Tomcat 9 for development.

#### Removed libraries

- **Spring Data Rest** - we are using Spring rest controllers.
- **Joda** time - and related libraries for json and dababase support.
- Spring Boot **Websocket** - websocket support removed.
- **org.codehaus.jackson** - we are usign **com.fasterxml.jackson** only.

## Before upgrade

- Delete all persisted events. Can be used:
  - task ``DeleteExecutedEventTaskExecutor``
  - or button on event agenda (available for super admin users).
- Resolve all workflow task (complete or cancel). New workflow definitions version will be deployed automatically. Old workflow tasks will be alive, but its saver to resolve them before new version is deployed. Historic tasks and processes will be preserved untouched and their detail will be available.

## Database migration

When the CzechIdM version 10 is started for the first time, it will do an automatic update (as with any 9 version) to the schema by provided Flyway change scripts.

Updated Flyway version requires **PostgresSQL database version >= 9.4** ([supported versions](https://flywaydb.org/documentation/database/postgresql)).

> Note for developer: Default database name configured for the all ``dev`` profiles was renamed to ``bcv_idm_10`` to prevent update (by Flyway) old database on the background - **old database can be used for LTS version 9.7.x development**, so clone database is needed.

## Configuration properties

Based on upgraded libraries we have to add and remove configuration properties (mainly related to connection pool and hibernate).

### Added properties

- ``spring.jpa.properties.hibernate.jdbc.time_zone=UTC`` - datetime will be persisted in UTC in database.
- ``spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation=true`` - disable warning during the boot, when Hibernate tries to retrieve some meta information from the database.
- ``spring.datasource.hikari.maximumPoolSize=25`` - enlarge pool size by default. This property should be revised for each project.
- ``spring.jpa.properties.hibernate.session_factory.interceptor=eu.bcvsolutions.idm.core.model.repository.listener.AuditableInterceptor`` - replaced deprecated Hiberante property.
- ``spring.jpa.hibernate.use-new-id-generator-mappings=false`` - Spring boot 2 changed default to ``true``, but we are using ``IDENTITY`` identifier generators for mssql database.


### Removed properties

- ``spring.jpa.properties.jadira.usertype.autoRegisterUserTypes`` - we are using java time now, so configuration for Joda time is not needed.
- ``spring.jpa.properties.hibernate.ejb.interceptor`` - replaced by new property above.

## Conceptual changes

- Spring repository queries, which updates audited entity (``@Modifying`` annotation) is prohibited, because audit is skipped this way. Use entity service (find, then update). All product provided repository methods are removed in version 10 (see changelog).

## Breaking changes

- **Joda time library was removed** - all entities, dtos and services use java time now ⇒ api was changed, all places which used joda time have to be refactored (included workflow and groovy scripts). The related issue is with [serialized dtos](#serialized-dtos) in workflow properties and operation result.
- **Activiti** 6 registered new ``formService`` bean usable in workflow definition ⇒ IdM eav service is not available under ``formService`` name any more. New bean alias ``idmFormService`` was created and has to be used in workflows.
- **Hibernate** removed data type ``org.hibernate.type.StringClobType`` - all entities has to be refactored, type ``org.hibernate.type.TextType`` has to be used.
- **Flyway** removed [support](https://flywaydb.org/documentation/database/postgresql) for old PostgreSgl databases lower than version 9.4.
- **Envers** changed api for extending criteria (check ``AuditCriterion`` usage).
- **Mockito** changes behavior for ``any(Class)`` checker - doesn't support ``null`` parameter value now. This is used just in unit tests. Test can be compiled but doesn't work.
- **Spring** data repository api changes:
  - e.g. ``findOne`` renamed to ``findOneById`` and returns ``Optional<E>``,
  - the concept is changed the same way for all methods.

## Update custom module

Due to breaking changes above, custom module requires some refactoring, before it's compatible with CzechIdM version 10. Some refactoring can be done with replaces, but some places has to be changed manually.

### Replaces

> Case sensitive find is expected.

- ``org.hibernate.type.StringClobType`` ⇒ ``org.hibernate.type.TextType``
{'fraction_non_alphanumeric': 0.10787963393826586, 'fraction_numerical': 0.01713975492477121, 'mean_word_length': 4.397655922980326, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6905175', 'n_tokens_mistral': 3695, 'n_tokens_neox': 3445, 'n_words': 1569}