<commit_before>// Category: debezium-using
// Type: assembly
// ModuleID: debezium-logging
[id="debezium-logging"]
= {prodname} logging

:linkattrs:
:icons: font
:source-highlighter: highlight.js
:toc:
:toc-placement: macro

toc::[]

{prodname} has extensive logging built into its connectors,
and you can change the logging configuration to control which of these log statements appear in the logs
and where those logs are sent.
{prodname} (as well as Kafka, Kafka Connect, and Zookeeper) use the https://logging.apache.org/log4j/1.2/[Log4j] logging framework for Java.

By default, the connectors produce a fair amount of useful information when they start up,
but then produce very few logs when the connector is keeping up with the source databases.
This is often sufficient when the connector is operating normally,
but may not be enough when the connector is behaving unexpectedly.
In such cases, you can change the logging level so that the connector generates much more verbose log messages describing what the connector is doing and what it is not doing.

// Type: concept
// ModuleID: debezium-logging-concepts
// Title: {prodname} logging concepts
[id="logging-concepts"]
== Logging concepts

Before configuring logging, you should understand what Log4J _loggers_, _log levels_, and _appenders_ are.

[discrete]
=== Loggers

Each log message produced by the application is sent to a specific _logger_
(for example, `io.debezium.connector.mysql`).
Loggers are arranged in hierarchies.
For example, the `io.debezium.connector.mysql` logger is the child of the `io.debezium.connector` logger,
which is the child of the `io.debezium` logger.
At the top of the hierarchy,
the _root logger_ defines the default logger configuration for all of the loggers beneath it.

[discrete]
=== Log levels

Every log message produced by the application also has a specific _log level_:

1. `ERROR` - errors, exceptions, and other significant problems
2. `WARN` - _potential_ problems and issues
3. `INFO` - status and general activity (usually low-volume)
4. `DEBUG` - more detailed activity that would be useful in diagnosing unexpected behavior
5. `TRACE` - very verbose and detailed activity (usually very high-volume)

[discrete]
=== Appenders

An _appender_ is essentially a destination where log messages are written.
Each appender controls the format of its log messages,
giving you even more control over what the log messages look like.

To configure logging, you specify the desired level for each logger and the appender(s) where those log messages should be written. Since loggers are hierarchical, the configuration for the root logger serves as a default for all of the loggers below it, although you can override any child (or descendant) logger.


// Type: concept
// ModuleID: default-debezium-logging-configuration
// Title: Default {prodname} logging configuration
[id="understanding-default-logging-configuration"]
== Understanding the default logging configuration

If you are running {prodname} connectors in a Kafka Connect process,
then Kafka Connect uses the Log4j configuration file (for example, `/opt/kafka/config/connect-log4j.properties`) in the Kafka installation.
By default, this file contains the following configuration:

.connect-log4j.properties
[source,properties,options="nowrap"]
----
log4j.rootLogger=INFO, stdout  // <1>

log4j.appender.stdout=org.apache.log4j.ConsoleAppender  // <2>
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout  // <3>
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n  // <4>
...
----
<1> The root logger, which defines the default logger configuration.
By default, loggers include `INFO`, `WARN`, and `ERROR` messages.
These log messages are written to the `stdout` appender.
<2> The `stdout` appender writes log messages to the console (as opposed to a file).
<3> The `stdout` appender uses a pattern matching algorithm to format the log messages.
<4> The pattern for the `stdout` appender (see the https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html[Log4j documentation] for details).

Unless you configure other loggers,
all of the loggers that {prodname} uses inherit the `rootLogger` configuration.


// Type: assembly
// ModuleID: configuring-debezium-logging
// Title: Configuring {prodname} logging
[id="configuring-logging"]
== Configuring logging

By default, {prodname} connectors write all `INFO`, `WARN`, and `ERROR` messages to the console.
You can change the default logging configuration by using one of the following methods:

* xref:changing-logging-level[Setting the logging level by configuring loggers]
* xref:setting-the-logging-level-with-the-kafka-connect-rest-api[Dynamically setting the logging level with the Kafka Connect REST API]
* xref:adding-mapped-diagnostic-contexts[Setting the logging level by adding mapped diagnostic contexts]

[NOTE]
====
There are other methods that you can use to configure {prodname} logging with Log4j.
For more information, search for tutorials about setting up and using appenders to send log messages to specific destinations.
====

// Type: procedure
// ModuleID: changing-the-debezium-logging-level
// Title: Changing the {prodname} logging level by configuring loggers
[id="changing-logging-level"]
=== Setting the logging level by configuring loggers

The default {prodname} logging level provides sufficient information to show whether a connector is healthy or not.
However, if a connector is not healthy, you can change its logging level to troubleshoot the issue.

In general, {prodname} connectors send their log messages to loggers with names that match the fully-qualified name of the Java class that is generating the log message.
{prodname} uses packages to organize code with similar or related functions.
This means that you can control all of the log messages for a specific class or for all of the classes within or under a specific package.

.Procedure

. Open the `log4j.properties` file.

. Configure a logger for the connector.
+
--
This example configures loggers for the MySQL connector and the database history implementation used by the connector,
and sets them to log `DEBUG` level messages:

.log4j.properties
[source,properties,options="nowrap"]
----
...
log4j.logger.io.debezium.connector.mysql=DEBUG, stdout  // <1>
log4j.logger.io.debezium.relational.history=DEBUG, stdout  // <2>

log4j.additivity.io.debezium.connector.mysql=false  // <3>
log4j.additivity.io.debezium.relational.history=false  // <3>
...
----
<1> Configures the logger named `io.debezium.connector.mysql` to send `DEBUG`, `INFO`, `WARN`, and `ERROR` messages to the `stdout` appender.
<2> Configures the logger named `io.debezium.relational.history` to send `DEBUG`, `INFO`, `WARN`, and `ERROR` messages to the `stdout` appender.
<3> Turns off _additivity_,
which results in log messages not being sent to the appenders of parent loggers (this can prevent seeing duplicate log messages when using multiple appenders).
--

. If necessary, change the logging level for a specific subset of the classes within the connector.
+
--
Increasing the logging level for the entire connector increases the log verbosity,
which can make it difficult to understand what is happening.
In these cases,
you can change the logging level just for the subset of classes that are related to the issue that you are troubleshooting.
--

.. Set the connector's logging level to either `DEBUG` or `TRACE`.

.. Review the connector's log messages.
+
--
Find the log messages that are related to the issue that you are troubleshooting.
The end of each log message shows the name of the Java class that produced the message.
--

.. Set the connector's logging level back to `INFO`.

.. Configure a logger for each Java class that you identified.
+
--
For example, consider a scenario in which you are unsure why the MySQL connector is skipping some events when it is processing the binlog.
Rather than turn on `DEBUG` or `TRACE` logging for the entire connector,
you can keep the connector's logging level at `INFO` and then configure `DEBUG` or `TRACE` on just the class that is reading the binlog:

.log4j.properties
[source,properties,options="nowrap"]
----
...
log4j.logger.io.debezium.connector.mysql=INFO, stdout
log4j.logger.io.debezium.connector.mysql.BinlogReader=DEBUG, stdout
log4j.logger.io.debezium.relational.history=INFO, stdout

log4j.additivity.io.debezium.connector.mysql=false
log4j.additivity.io.debezium.relational.history=false
log4j.additivity.io.debezium.connector.mysql.BinlogReader=false
...
----
--

// Type: procedure
// ModuleID: setting-the-debezium-logging-level-with-the-kafka-connect-rest-api
// Title: Dynamically changing the {prodname} logging level with the Kafka Connect API
[id="setting-the-logging-level-with-the-kafka-connect-rest-api"]
=== Dynamically setting the logging level with the Kafka Connect REST API

You can use the Kafka Connect REST API to set logging levels for a connector dynamically at runtime.
Unlike log level changes that you set in `log4j.properties`, changes that you make via the API take effect immediately, and do not require you to restart the worker.

The log level setting that you specify in the API applies only to the worker at the endpoint that receives the request.
The log levels of other workers in the cluster remain unchanged.

The specified level is not persisted after the worker restarts.
To make persistent changes to the logging level, set the log level in `log4j.properties` by xref:changing-logging-level[configuring loggers] or xref:adding-mapped-diagnostic-contexts[adding mapped diagnostic contexts].

.Procedure

* Set the log level by sending a PUT request to the `admin/loggers` endpoint that specifies the following information:
** The package for which you want to change the log level.
** The log level that you want to set.
+
[source,shell,subs="+attributes,+quotes", options="nowrap"]
----
curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/admin/loggers/io.debezium.connector._<connector_package>_ -d '{"level": "_<level>_"}'
----
+
For example, to log debug information for a {prodname} MySQL connector, send the following request to Kafka Connect:
+
[source,shell,options="nowrap"]
----
curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/admin/loggers/io.debezium.connector.mysql -d '{"level": "DEBUG"}'
----

// Type: procedure
// ModuleID: adding-debezium-mapped-diagnostic-contexts
// Title: Changing the {prodname} logging levely by adding mapped diagnostic contexts
[id="adding-mapped-diagnostic-contexts"]
=== Setting the logging level with mapped diagnostic contexts

Most {prodname} connectors (and the Kafka Connect workers) use multiple threads to perform different activities.
This can make it difficult to look at a log file and find only those log messages for a particular logical activity.
To make the log messages easier to find,
{prodname} provides several _mapped diagnostic contexts_ (MDC) that provide additional information for each thread.

{prodname} provides the following MDC properties:

`dbz.connectorType`::
A short alias for the type of connector.
For example, `MySql`, `Mongo`, `Postgres`, and so on.
All threads associated with the same _type_ of connector use the same value,<commit_msg>DBZ-3270 Update replaceable string in command to be more specific<commit_after>// Category: debezium-using
// Type: assembly
// ModuleID: debezium-logging
[id="debezium-logging"]
= {prodname} logging

:linkattrs:
:icons: font
:source-highlighter: highlight.js
:toc:
:toc-placement: macro

toc::[]

{prodname} has extensive logging built into its connectors,
and you can change the logging configuration to control which of these log statements appear in the logs
and where those logs are sent.
{prodname} (as well as Kafka, Kafka Connect, and Zookeeper) use the https://logging.apache.org/log4j/1.2/[Log4j] logging framework for Java.

By default, the connectors produce a fair amount of useful information when they start up,
but then produce very few logs when the connector is keeping up with the source databases.
This is often sufficient when the connector is operating normally,
but may not be enough when the connector is behaving unexpectedly.
In such cases, you can change the logging level so that the connector generates much more verbose log messages describing what the connector is doing and what it is not doing.

// Type: concept
// ModuleID: debezium-logging-concepts
// Title: {prodname} logging concepts
[id="logging-concepts"]
== Logging concepts

Before configuring logging, you should understand what Log4J _loggers_, _log levels_, and _appenders_ are.

[discrete]
=== Loggers

Each log message produced by the application is sent to a specific _logger_
(for example, `io.debezium.connector.mysql`).
Loggers are arranged in hierarchies.
For example, the `io.debezium.connector.mysql` logger is the child of the `io.debezium.connector` logger,
which is the child of the `io.debezium` logger.
At the top of the hierarchy,
the _root logger_ defines the default logger configuration for all of the loggers beneath it.

[discrete]
=== Log levels

Every log message produced by the application also has a specific _log level_:

1. `ERROR` - errors, exceptions, and other significant problems
2. `WARN` - _potential_ problems and issues
3. `INFO` - status and general activity (usually low-volume)
4. `DEBUG` - more detailed activity that would be useful in diagnosing unexpected behavior
5. `TRACE` - very verbose and detailed activity (usually very high-volume)

[discrete]
=== Appenders

An _appender_ is essentially a destination where log messages are written.
Each appender controls the format of its log messages,
giving you even more control over what the log messages look like.

To configure logging, you specify the desired level for each logger and the appender(s) where those log messages should be written. Since loggers are hierarchical, the configuration for the root logger serves as a default for all of the loggers below it, although you can override any child (or descendant) logger.


// Type: concept
// ModuleID: default-debezium-logging-configuration
// Title: Default {prodname} logging configuration
[id="understanding-default-logging-configuration"]
== Understanding the default logging configuration

If you are running {prodname} connectors in a Kafka Connect process,
then Kafka Connect uses the Log4j configuration file (for example, `/opt/kafka/config/connect-log4j.properties`) in the Kafka installation.
By default, this file contains the following configuration:

.connect-log4j.properties
[source,properties,options="nowrap"]
----
log4j.rootLogger=INFO, stdout  // <1>

log4j.appender.stdout=org.apache.log4j.ConsoleAppender  // <2>
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout  // <3>
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n  // <4>
...
----
<1> The root logger, which defines the default logger configuration.
By default, loggers include `INFO`, `WARN`, and `ERROR` messages.
These log messages are written to the `stdout` appender.
<2> The `stdout` appender writes log messages to the console (as opposed to a file).
<3> The `stdout` appender uses a pattern matching algorithm to format the log messages.
<4> The pattern for the `stdout` appender (see the https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html[Log4j documentation] for details).

Unless you configure other loggers,
all of the loggers that {prodname} uses inherit the `rootLogger` configuration.


// Type: assembly
// ModuleID: configuring-debezium-logging
// Title: Configuring {prodname} logging
[id="configuring-logging"]
== Configuring logging

By default, {prodname} connectors write all `INFO`, `WARN`, and `ERROR` messages to the console.
You can change the default logging configuration by using one of the following methods:

* xref:changing-logging-level[Setting the logging level by configuring loggers]
* xref:setting-the-logging-level-with-the-kafka-connect-rest-api[Dynamically setting the logging level with the Kafka Connect REST API]
* xref:adding-mapped-diagnostic-contexts[Setting the logging level by adding mapped diagnostic contexts]

[NOTE]
====
There are other methods that you can use to configure {prodname} logging with Log4j.
For more information, search for tutorials about setting up and using appenders to send log messages to specific destinations.
====

// Type: procedure
// ModuleID: changing-the-debezium-logging-level
// Title: Changing the {prodname} logging level by configuring loggers
[id="changing-logging-level"]
=== Setting the logging level by configuring loggers

The default {prodname} logging level provides sufficient information to show whether a connector is healthy or not.
However, if a connector is not healthy, you can change its logging level to troubleshoot the issue.

In general, {prodname} connectors send their log messages to loggers with names that match the fully-qualified name of the Java class that is generating the log message.
{prodname} uses packages to organize code with similar or related functions.
This means that you can control all of the log messages for a specific class or for all of the classes within or under a specific package.

.Procedure

. Open the `log4j.properties` file.

. Configure a logger for the connector.
+
--
This example configures loggers for the MySQL connector and the database history implementation used by the connector,
and sets them to log `DEBUG` level messages:

.log4j.properties
[source,properties,options="nowrap"]
----
...
log4j.logger.io.debezium.connector.mysql=DEBUG, stdout  // <1>
log4j.logger.io.debezium.relational.history=DEBUG, stdout  // <2>

log4j.additivity.io.debezium.connector.mysql=false  // <3>
log4j.additivity.io.debezium.relational.history=false  // <3>
...
----
<1> Configures the logger named `io.debezium.connector.mysql` to send `DEBUG`, `INFO`, `WARN`, and `ERROR` messages to the `stdout` appender.
<2> Configures the logger named `io.debezium.relational.history` to send `DEBUG`, `INFO`, `WARN`, and `ERROR` messages to the `stdout` appender.
<3> Turns off _additivity_,
which results in log messages not being sent to the appenders of parent loggers (this can prevent seeing duplicate log messages when using multiple appenders).
--

. If necessary, change the logging level for a specific subset of the classes within the connector.
+
--
Increasing the logging level for the entire connector increases the log verbosity,
which can make it difficult to understand what is happening.
In these cases,
you can change the logging level just for the subset of classes that are related to the issue that you are troubleshooting.
--

.. Set the connector's logging level to either `DEBUG` or `TRACE`.

.. Review the connector's log messages.
+
--
Find the log messages that are related to the issue that you are troubleshooting.
The end of each log message shows the name of the Java class that produced the message.
--

.. Set the connector's logging level back to `INFO`.

.. Configure a logger for each Java class that you identified.
+
--
For example, consider a scenario in which you are unsure why the MySQL connector is skipping some events when it is processing the binlog.
Rather than turn on `DEBUG` or `TRACE` logging for the entire connector,
you can keep the connector's logging level at `INFO` and then configure `DEBUG` or `TRACE` on just the class that is reading the binlog:

.log4j.properties
[source,properties,options="nowrap"]
----
...
log4j.logger.io.debezium.connector.mysql=INFO, stdout
log4j.logger.io.debezium.connector.mysql.BinlogReader=DEBUG, stdout
log4j.logger.io.debezium.relational.history=INFO, stdout

log4j.additivity.io.debezium.connector.mysql=false
log4j.additivity.io.debezium.relational.history=false
log4j.additivity.io.debezium.connector.mysql.BinlogReader=false
...
----
--

// Type: procedure
// ModuleID: setting-the-debezium-logging-level-with-the-kafka-connect-rest-api
// Title: Dynamically changing the {prodname} logging level with the Kafka Connect API
[id="setting-the-logging-level-with-the-kafka-connect-rest-api"]
=== Dynamically setting the logging level with the Kafka Connect REST API

You can use the Kafka Connect REST API to set logging levels for a connector dynamically at runtime.
Unlike log level changes that you set in `log4j.properties`, changes that you make via the API take effect immediately, and do not require you to restart the worker.

The log level setting that you specify in the API applies only to the worker at the endpoint that receives the request.
The log levels of other workers in the cluster remain unchanged.

The specified level is not persisted after the worker restarts.
To make persistent changes to the logging level, set the log level in `log4j.properties` by xref:changing-logging-level[configuring loggers] or xref:adding-mapped-diagnostic-contexts[adding mapped diagnostic contexts].

.Procedure

* Set the log level by sending a PUT request to the `admin/loggers` endpoint that specifies the following information:
** The package for which you want to change the log level.
** The log level that you want to set.
+
[source,shell,subs="+attributes,+quotes", options="nowrap"]
----
curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/admin/loggers/io.debezium.connector._<connector_package>_ -d '{"level": "_<log_level>_"}'
----
+
For example, to log debug information for a {prodname} MySQL connector, send the following request to Kafka Connect:
+
[source,shell,options="nowrap"]
----
curl -s -X PUT -H "Content-Type:application/json" http://localhost:8083/admin/loggers/io.debezium.connector.mysql -d '{"level": "DEBUG"}'
----

// Type: procedure
// ModuleID: adding-debezium-mapped-diagnostic-contexts
// Title: Changing the {prodname} logging levely by adding mapped diagnostic contexts
[id="adding-mapped-diagnostic-contexts"]
=== Setting the logging level with mapped diagnostic contexts

Most {prodname} connectors (and the Kafka Connect workers) use multiple threads to perform different activities.
This can make it difficult to look at a log file and find only those log messages for a particular logical activity.
To make the log messages easier to find,
{prodname} provides several _mapped diagnostic contexts_ (MDC) that provide additional information for each thread.

{prodname} provides the following MDC properties:

`dbz.connectorType`::
A short alias for the type of connector.
For example, `MySql`, `Mongo`, `Postgres`, and so on.
All threads associated with the same _type_ of connector use the same value,
{'fraction_non_alphanumeric': 0.083994708994709, 'fraction_numerical': 0.005731922398589065, 'mean_word_length': 4.714537666918619, 'pattern_counts': {'":': 4, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1368300', 'n_tokens_mistral': 6265, 'n_tokens_neox': 6120, 'n_words': 2812}