<commit_before>==================
Unreleased Changes
==================

This file collects *unreleased* changes only.

For release notes, see:

  https://crate.io/docs/reference/en/latest/release_notes/index.html

For developers: changes should be recorded here (using RST syntax) as you are
developing CrateDB. When a new release is being cut, changes will be moved to
the appropriate section of the docs.

Breaking Changes
================

- The columns ``number_of_shards``, ``number_of_replicas``, and
  ``self_referencing_column_name`` in ``information_schema.tables`` changed to
  return ``NULL`` for non-sharded tables.

- Adapted queries in the Admin UI to be compatible with CrateDB 3.0+

- For HTTP authentication, support was dropped for the ``X-User`` header, used
  to provide a username, which has been deprecated in ``2.3.0.`` in favour of
  the standard HTTP ``Authorization`` header.

- Dropped support for tables that have been created with CrateDB prior to
  version 2.0. Tables which require upgrading are indicated in the cluster
  checks, including visually shown in the Admin UI, if running the latest 2.2
  or 2.3 release. The upgrade of tables needs to happen before updating CrateDB
  to this version. This can be done by exporting the data with ``COPY TO`` and
  importing it into a new table with ``COPY FROM``.  Alternatively you can use
  ``INSERT`` with query.

- Data paths as defined in ``path.data`` must not contain the cluster name as a
  folder. Data paths which are not compatible with this version are indicated
  in the node checks, including visually shown in the Admin UI, if running the
  latest 2.2 or 2.3 release.

- The ``bootstrap.seccomp`` setting that controls system call filters has been
  replaced in favour of ``bootstrap.system_call_filter``.

- The ``error_trace`` GET parameter of the HTTP endpoint only allows ``true``
  and ``false`` in lower case. Other values are not allowed any more and will
  result in a parsing exception.

- Store level throttling settings (``indices.store.throttle.*``) have been
  removed.

- The gateway recovery table setting ``recovery.initial_shards`` has been
  removed. Nodes will recover their unassigned local primary shards
  immediatelly after restart.

- The `_node` column on `sys.shards` and `sys.operations` has been renamed to
  `node`, is now visible by default and has been trimmed to only include
  `node['id']` and `node['name']`. In order to get all information a join query
  can be used with `sys.nodes`.

Changes
=======

- Changed the primary key constraints of the information schema tables
  ``table_constraints``, ``referential_constraints``, ``table_partitions``,
  ``key_column_usage``, ``columns``, and ``tables`` to be SQL compliant.

- Fixed path of logfile name in rotation pattern in ``log4j2.properties``. It
  now writes into the correct logging directory instead of the parent
  directory.

- Added support for views which can be created using the new ``CREATE VIEW``
  statement and dropped using the ``DROP VIEW`` statement. Views are listed in
  ``information_schema.views`` and they show up in
  ``information_schema.tables`` as well as ``information_schema.columns``

- Added support for ``INSERT INTO ... ON CONFLICT DO NOTHING``. The statement
  ignores insert values which would cause duplicate keys.

- Arrays can now contain mixed types if they're safely convertible. JSON
  libraries tend to encode values like ``[ 0.0, 1.2]`` as ``[ 0, 1.2 ]``, this
  caused an error because of the strict type match we enforced before.

- DEPRECATED: The ``ON DUPLICATE KEY UPDATE`` clause has been deprecated in
  favor of the ``ON CONFLICT DO UPDATE SET`` clause.

- Added support for ON CONFLICT clause in insert statements.
  ``INSERT INTO ... ON CONFLICT (pk_col) DO UPDATE SET col = val`` is identical
  to ``INSERT INTO ... ON DUPLICATE KEY UPDATE col = val``.
  The special EXCLUDED table can be used to refer to the insert values:
  ``INSERT INTO ... ON CONFLICT (pk_col) DO UPDATE SET col = EXCLUDED.col``

- Added support for ``DEALLOCATE`` statement which is used by certain Postgres
  Wire Protocol clients (e.g. libpq) to deallocate a prepared statement and
  release its resources.

- Implemented ``constraint_schema`` and ``table_schema`` in
  ``information_schema.key_column_usage`` correctly and documented the full
  table schema.

- Implemented the ``Block Hash Join`` algorithm which is now used for
  ``Equi-Joins``.

- Added new `sys.health` system information table to expose the health of all
  tables and table partitions.

- The first argument (``field``) of the ``EXTRACT`` function has been limited
  to string literals and identifiers, as it was documented.

- Added note about the newly available `JMX HTTP Exporter` to the monitoring
  documentation section.

- Enable logging of JVM garbage collection times that help to debug memory
  pressure and garbage collection issues. GC log files are stored separately to
  the standard CrateDB logs and the files are log-rotated.

- CrateDB will now by default create a heap dump in case of a crash caused by
  an out of memory error. This makes it necessary to account for the additional
  disk space requirements.

- Added 'epoch' argument to the extract function which returns the number of
  seconds since Jan 1, 1970. For example:
  ``extract(epoch from '1970-01-01T00:00:01')`` returns ``1.0`` seconds.

- Add max_token_length parameter to whitespace tokenizer.

- Statistics for jobs and operations are enabled by default. If you don't need
  any statistics, please set ``stats.enabled`` to ``false``.

- ``ALTER TABLE <name> OPEN`` will now wait for all shards to become active
  before returning to be consistent with the behaviour of other statements.

- CrateDB is now based on Elasticsearch 6.1.3 and Lucene 7.1.0.

- Added a new ``bengali`` language analyzer and a ``bengali_normalization``
  token filter.

- Added new ``cluster.routing.allocation.disk.watermark.flood_stage`` setting,
  that controls at which disk usage indices should become read-only to prevent
  running out of disk space. There is also a new node check that indicates
  whether the threshold is exceeded.

- Added new tokenizers ``simple_pattern`` and ``simple_pattern_split`` which
  allow to tokenize text for the fulltext index by a regular expression pattern.

- Changed ``BEGIN`` and ``SET SESSION`` to no longer require ``DQL``
  permissions on the ``CLUSTER`` level.

- Implemented a `Ready` node status JMX metric expressing if the node is ready
  for processing SQL statements.

Fixes
=====

- Fixed an issue that would cause a NullPointerException to be thrown when 
  executing an ``EXPLAIN`` statement for a ``CROSS JOIN``.

- Do not allow renaming a table when connected to a read-only node.

- Fixed an issue that would cause a ClassCastException to be thrown instead of
  BatchUpdateException with a descriptive message, when using batched prepared
  statements with ``SELECT`` queries.
<commit_msg>Add CHANGES entry for view privilege class<commit_after>==================
Unreleased Changes
==================

This file collects *unreleased* changes only.

For release notes, see:

  https://crate.io/docs/reference/en/latest/release_notes/index.html

For developers: changes should be recorded here (using RST syntax) as you are
developing CrateDB. When a new release is being cut, changes will be moved to
the appropriate section of the docs.

Breaking Changes
================

- The columns ``number_of_shards``, ``number_of_replicas``, and
  ``self_referencing_column_name`` in ``information_schema.tables`` changed to
  return ``NULL`` for non-sharded tables.

- Adapted queries in the Admin UI to be compatible with CrateDB 3.0+

- For HTTP authentication, support was dropped for the ``X-User`` header, used
  to provide a username, which has been deprecated in ``2.3.0.`` in favour of
  the standard HTTP ``Authorization`` header.

- Dropped support for tables that have been created with CrateDB prior to
  version 2.0. Tables which require upgrading are indicated in the cluster
  checks, including visually shown in the Admin UI, if running the latest 2.2
  or 2.3 release. The upgrade of tables needs to happen before updating CrateDB
  to this version. This can be done by exporting the data with ``COPY TO`` and
  importing it into a new table with ``COPY FROM``.  Alternatively you can use
  ``INSERT`` with query.

- Data paths as defined in ``path.data`` must not contain the cluster name as a
  folder. Data paths which are not compatible with this version are indicated
  in the node checks, including visually shown in the Admin UI, if running the
  latest 2.2 or 2.3 release.

- The ``bootstrap.seccomp`` setting that controls system call filters has been
  replaced in favour of ``bootstrap.system_call_filter``.

- The ``error_trace`` GET parameter of the HTTP endpoint only allows ``true``
  and ``false`` in lower case. Other values are not allowed any more and will
  result in a parsing exception.

- Store level throttling settings (``indices.store.throttle.*``) have been
  removed.

- The gateway recovery table setting ``recovery.initial_shards`` has been
  removed. Nodes will recover their unassigned local primary shards
  immediatelly after restart.

- The `_node` column on `sys.shards` and `sys.operations` has been renamed to
  `node`, is now visible by default and has been trimmed to only include
  `node['id']` and `node['name']`. In order to get all information a join query
  can be used with `sys.nodes`.

Changes
=======

- Introduced a new VIEW privilege class which is used when granting/denying
  access on views.

- Changed the primary key constraints of the information schema tables
  ``table_constraints``, ``referential_constraints``, ``table_partitions``,
  ``key_column_usage``, ``columns``, and ``tables`` to be SQL compliant.

- Fixed path of logfile name in rotation pattern in ``log4j2.properties``. It
  now writes into the correct logging directory instead of the parent
  directory.

- Added support for views which can be created using the new ``CREATE VIEW``
  statement and dropped using the ``DROP VIEW`` statement. Views are listed in
  ``information_schema.views`` and they show up in
  ``information_schema.tables`` as well as ``information_schema.columns``

- Added support for ``INSERT INTO ... ON CONFLICT DO NOTHING``. The statement
  ignores insert values which would cause duplicate keys.

- Arrays can now contain mixed types if they're safely convertible. JSON
  libraries tend to encode values like ``[ 0.0, 1.2]`` as ``[ 0, 1.2 ]``, this
  caused an error because of the strict type match we enforced before.

- DEPRECATED: The ``ON DUPLICATE KEY UPDATE`` clause has been deprecated in
  favor of the ``ON CONFLICT DO UPDATE SET`` clause.

- Added support for ON CONFLICT clause in insert statements.
  ``INSERT INTO ... ON CONFLICT (pk_col) DO UPDATE SET col = val`` is identical
  to ``INSERT INTO ... ON DUPLICATE KEY UPDATE col = val``.
  The special EXCLUDED table can be used to refer to the insert values:
  ``INSERT INTO ... ON CONFLICT (pk_col) DO UPDATE SET col = EXCLUDED.col``

- Added support for ``DEALLOCATE`` statement which is used by certain Postgres
  Wire Protocol clients (e.g. libpq) to deallocate a prepared statement and
  release its resources.

- Implemented ``constraint_schema`` and ``table_schema`` in
  ``information_schema.key_column_usage`` correctly and documented the full
  table schema.

- Implemented the ``Block Hash Join`` algorithm which is now used for
  ``Equi-Joins``.

- Added new `sys.health` system information table to expose the health of all
  tables and table partitions.

- The first argument (``field``) of the ``EXTRACT`` function has been limited
  to string literals and identifiers, as it was documented.

- Added note about the newly available `JMX HTTP Exporter` to the monitoring
  documentation section.

- Enable logging of JVM garbage collection times that help to debug memory
  pressure and garbage collection issues. GC log files are stored separately to
  the standard CrateDB logs and the files are log-rotated.

- CrateDB will now by default create a heap dump in case of a crash caused by
  an out of memory error. This makes it necessary to account for the additional
  disk space requirements.

- Added 'epoch' argument to the extract function which returns the number of
  seconds since Jan 1, 1970. For example:
  ``extract(epoch from '1970-01-01T00:00:01')`` returns ``1.0`` seconds.

- Add max_token_length parameter to whitespace tokenizer.

- Statistics for jobs and operations are enabled by default. If you don't need
  any statistics, please set ``stats.enabled`` to ``false``.

- ``ALTER TABLE <name> OPEN`` will now wait for all shards to become active
  before returning to be consistent with the behaviour of other statements.

- CrateDB is now based on Elasticsearch 6.1.3 and Lucene 7.1.0.

- Added a new ``bengali`` language analyzer and a ``bengali_normalization``
  token filter.

- Added new ``cluster.routing.allocation.disk.watermark.flood_stage`` setting,
  that controls at which disk usage indices should become read-only to prevent
  running out of disk space. There is also a new node check that indicates
  whether the threshold is exceeded.

- Added new tokenizers ``simple_pattern`` and ``simple_pattern_split`` which
  allow to tokenize text for the fulltext index by a regular expression pattern.

- Changed ``BEGIN`` and ``SET SESSION`` to no longer require ``DQL``
  permissions on the ``CLUSTER`` level.

- Implemented a `Ready` node status JMX metric expressing if the node is ready
  for processing SQL statements.

Fixes
=====

- Fixed an issue that would cause a NullPointerException to be thrown when 
  executing an ``EXPLAIN`` statement for a ``CROSS JOIN``.

- Do not allow renaming a table when connected to a read-only node.

- Fixed an issue that would cause a ClassCastException to be thrown instead of
  BatchUpdateException with a descriptive message, when using batched prepared
  statements with ``SELECT`` queries.

{'fraction_non_alphanumeric': 0.08309130835163278, 'fraction_numerical': 0.007225331161011547, 'mean_word_length': 4.156318480642805, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3540660', 'n_tokens_mistral': 3908, 'n_tokens_neox': 3739, 'n_words': 1940}