<commit_before><!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<h3><a id="upgrade" href="#upgrade">1.5 Upgrading From Previous Versions</a></h3>

<h4><a id="upgrade_10" href="#upgrade_10">Upgrading from 0.8.x or 0.9.x to 0.10.0.0</a></h4>
0.10.0.0 has <a href="#upgrade_10_breaking">potential breaking changes</a> (please review before upgrading) and
there may be a <a href="#upgrade_10_performance_impact">performance impact during the upgrade</a>. Because new protocols
are introduced, it is important to upgrade your Kafka clusters before upgrading your clients.
<p/>
<b>Notes to clients with version 0.9.0.0: </b>Due to a bug introduced in 0.9.0.0,
clients that depend on ZooKeeper (old Scala high-level Consumer and MirrorMaker if used with the old consumer) will not
work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded to 0.9.0.1 <b>before</b> brokers are upgraded to
0.10.0.x. This step is not necessary for 0.8.X or 0.9.0.1 clients.

<p><b>For a rolling upgrade:</b></p>

<ol>
    <li> Update server.properties file on all brokers and add the following property: inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2 or 0.9.0.0).
         We recommend that users set log.message.format.version=CURRENT_KAFKA_VERSION as well to avoid a performance regression
         during upgrade. See <a href="#upgrade_10_performance_impact">potential performance impact during upgrade</a> for the details.
    </li>
    <li> Upgrade the brokers. This can be done a broker at a time by simply bringing it down, updating the code, and restarting it. </li>
    <li> Once the entire cluster is upgraded, bump the protocol version by editing inter.broker.protocol.version and setting it to 0.10.0.0. </li>
    <li> Restart the brokers one by one for the new protocol version to take effect. </li>
</ol>

<p><b>Note:</b> If you are willing to accept downtime, you can simply take all the brokers down, update the code and start all of them. They will start with the new protocol by default.

<p><b>Note:</b> Bumping the protocol version and restarting can be done any time after the brokers were upgraded. It does not have to be immediately after.

<h5><a id="upgrade_10_performance_impact" href="#upgrade_10_performance_impact">Potential performance impact during upgrade to 0.10.0.0</a></h5>
<p>
    The message format in 0.10.0 includes a new timestamp field and uses relative offsets for compressed messages.
    The on disk message format can be configured through log.message.format.version in the server.properties file.
    The default on-disk message format is 0.10.0. If a consumer client is on a version before 0.10.0.0, it only understands
    message formats before 0.10.0. In this case, the broker is able to convert messages from the 0.10.0 format to an earlier format
    before sending the response to the consumer on an older version. However, the broker can't use zero-copy transfer in this case.

    To avoid such message conversion before consumers are upgraded to 0.10.0.0, one can set the message format to
    e.g. 0.9.0 when upgrading the broker to 0.10.0.0. This way, the broker can still use zero-copy transfer to send the
    data to the old consumers. Once most consumers are upgraded, one can change the message format to 0.10.0 on the broker.
</p>
<p>
    For clients that are upgraded to 0.10.0.0, there is no performance impact.
</p>
<p>
    <b>Note:</b> By setting the message format version, one certifies that all existing messages are on or below that
    message format version. Otherwise consumers before 0.10.0.0 might break. In particular, after the message format
    is set to 0.10.0, one should not change it back to an earlier format as it may break consumers on versions before 0.10.0.0.
</p>
<p>
    <b>Note:</b> Due to the additional timestamp introduced in each message, producers sending small messages may see a
    message throughput degradation because of the increased overhead. When receiving compressed messages, 0.10.0
    brokers avoid recompressing the messages, which in general reduces the latency and improves the throughput. In
    certain cases, this may reduce the batching size on the producer, which could lead to worse throughput. If this
    happens, users can tune linger.ms and batch.size of the producer for better throughput.
</p>

<h5><a id="upgrade_10_breaking" href="#upgrade_10_breaking">Potential breaking changes in 0.10.0.0</a></h5>
<ul><commit_msg>MINOR: document increased network bandwidth of 0.10 under replication<commit_after><!--
 Licensed to the Apache Software Foundation (ASF) under one or more
 contributor license agreements.  See the NOTICE file distributed with
 this work for additional information regarding copyright ownership.
 The ASF licenses this file to You under the Apache License, Version 2.0
 (the "License"); you may not use this file except in compliance with
 the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,
 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
-->

<h3><a id="upgrade" href="#upgrade">1.5 Upgrading From Previous Versions</a></h3>

<h4><a id="upgrade_10" href="#upgrade_10">Upgrading from 0.8.x or 0.9.x to 0.10.0.0</a></h4>
0.10.0.0 has <a href="#upgrade_10_breaking">potential breaking changes</a> (please review before upgrading) and
there may be a <a href="#upgrade_10_performance_impact">performance impact during the upgrade</a>. Because new protocols
are introduced, it is important to upgrade your Kafka clusters before upgrading your clients.
<p/>
<b>Notes to clients with version 0.9.0.0: </b>Due to a bug introduced in 0.9.0.0,
clients that depend on ZooKeeper (old Scala high-level Consumer and MirrorMaker if used with the old consumer) will not
work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded to 0.9.0.1 <b>before</b> brokers are upgraded to
0.10.0.x. This step is not necessary for 0.8.X or 0.9.0.1 clients.

<p><b>For a rolling upgrade:</b></p>

<ol>
    <li> Update server.properties file on all brokers and add the following property: inter.broker.protocol.version=CURRENT_KAFKA_VERSION (e.g. 0.8.2 or 0.9.0.0).
         We recommend that users set log.message.format.version=CURRENT_KAFKA_VERSION as well to avoid a performance regression
         during upgrade. See <a href="#upgrade_10_performance_impact">potential performance impact during upgrade</a> for the details.
    </li>
    <li> Upgrade the brokers. This can be done a broker at a time by simply bringing it down, updating the code, and restarting it. </li>
    <li> Once the entire cluster is upgraded, bump the protocol version by editing inter.broker.protocol.version and setting it to 0.10.0.0. </li>
    <li> Restart the brokers one by one for the new protocol version to take effect. </li>
</ol>

<p><b>Note:</b> If you are willing to accept downtime, you can simply take all the brokers down, update the code and start all of them. They will start with the new protocol by default.

<p><b>Note:</b> Bumping the protocol version and restarting can be done any time after the brokers were upgraded. It does not have to be immediately after.

<h5><a id="upgrade_10_performance_impact" href="#upgrade_10_performance_impact">Potential performance impact during upgrade to 0.10.0.0</a></h5>
<p>
    The message format in 0.10.0 includes a new timestamp field and uses relative offsets for compressed messages.
    The on disk message format can be configured through log.message.format.version in the server.properties file.
    The default on-disk message format is 0.10.0. If a consumer client is on a version before 0.10.0.0, it only understands
    message formats before 0.10.0. In this case, the broker is able to convert messages from the 0.10.0 format to an earlier format
    before sending the response to the consumer on an older version. However, the broker can't use zero-copy transfer in this case.

    To avoid such message conversion before consumers are upgraded to 0.10.0.0, one can set the message format to
    e.g. 0.9.0 when upgrading the broker to 0.10.0.0. This way, the broker can still use zero-copy transfer to send the
    data to the old consumers. Once most consumers are upgraded, one can change the message format to 0.10.0 on the broker.
</p>
<p>
    For clients that are upgraded to 0.10.0.0, there is no performance impact.
</p>
<p>
    <b>Note:</b> By setting the message format version, one certifies that all existing messages are on or below that
    message format version. Otherwise consumers before 0.10.0.0 might break. In particular, after the message format
    is set to 0.10.0, one should not change it back to an earlier format as it may break consumers on versions before 0.10.0.0.
</p>
<p>
    <b>Note:</b> Due to the additional timestamp introduced in each message, producers sending small messages may see a
    message throughput degradation because of the increased overhead.
    Likewise, replication now transmits an additional 8 bytes per message.
    If you're running close to the network capacity of your cluster, it's possible that you'll overwhelm the network cards
    and see failures and performance issues due to the overload. When receiving compressed messages, 0.10.0
    brokers avoid recompressing the messages, which in general reduces the latency and improves the throughput. In
    certain cases, this may reduce the batching size on the producer, which could lead to worse throughput. If this
    happens, users can tune linger.ms and batch.size of the producer for better throughput.
</p>

<h5><a id="upgrade_10_breaking" href="#upgrade_10_breaking">Potential breaking changes in 0.10.0.0</a></h5>
<ul>
{'fraction_non_alphanumeric': 0.07689425868689707, 'fraction_numerical': 0.03015828416221785, 'mean_word_length': 3.982734484367709, 'pattern_counts': {'":': 0, '<': 125, '<?xml version=': 0, '>': 125, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 34, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7202287', 'n_tokens_mistral': 3141, 'n_tokens_neox': 2867, 'n_words': 1486}