<commit_before>## Overview
Apache Kafka is an open-source message broker project developed by the Apache
Software Foundation written in Scala. The project aims to provide a unified,
high-throughput, low-latency platform for handling real-time data feeds. Learn
more at [kafka.apache.org](http://kafka.apache.org/).


## Usage
Kafka requires the Zookeeper distributed coordination service. Deploy and
relate them as follows:

    juju deploy apache-zookeeper zookeeper
    juju deploy apache-kafka kafka
    juju add-relation kafka zookeeper

Once deployed, we can list the zookeeper servers that our kafka brokers
are connected to. The following will list `<ip>:<port>` information for each
zookeeper unit in the environment (e.g.: `10.0.3.221:2181`).

    juju action do kafka/0 list-zks
    juju action fetch <id>  # <-- id from above command

We can create a Kafka topic with:

    juju action do kafka/0 create-topic topic=<topic_name> \
     partitions=<#> replication=<#>
    juju action fetch <id>  # <-- id from above command

We can list topics with:

    juju action do kafka/0 list-topics
    juju action fetch <id>  # <-- id from above command

We can write to a topic with:

    juju action do kafka/0 write-topic topic=<topic_name> data=<data>
    juju action fetch <id>  # <-- id from above command

We can read from a topic with:

    juju action do kafka/0 read-topic topic=<topic_name> partition=<#>
    juju action fetch <id>  # <-- id from above command

And finally, we can delete a topic with:

    juju action do kafka/0 delete-topic topic=<topic_name>
    juju action fetch <id>  # <-- id from above command

## Deploying in Network-Restricted Environments
This charm can be deployed in environments with limited network access. To
deploy in this environment, you will need a local mirror to serve the packages
and resources required by this charm.

### Mirroring Packages
You can setup a local mirror for apt packages using squid-deb-proxy.
For instructions on configuring juju to use this, see the
[Juju Proxy Documentation](https://juju.ubuntu.com/docs/howto-proxies.html).

### Mirroring Resources<commit_msg>[ktsakalozos, r=kwmonroe] Adding instructions for multi broker deployment<commit_after>## Overview
Apache Kafka is an open-source message broker project developed by the Apache
Software Foundation written in Scala. The project aims to provide a unified,
high-throughput, low-latency platform for handling real-time data feeds. Learn
more at [kafka.apache.org](http://kafka.apache.org/).


## Usage
Kafka requires the Zookeeper distributed coordination service. Deploy and
relate them as follows:

    juju deploy apache-zookeeper zookeeper
    juju deploy apache-kafka kafka
    juju add-relation kafka zookeeper

Once deployed, we can list the zookeeper servers that our kafka brokers
are connected to. The following will list `<ip>:<port>` information for each
zookeeper unit in the environment (e.g.: `10.0.3.221:2181`).

    juju action do kafka/0 list-zks
    juju action fetch <id>  # <-- id from above command

We can create a Kafka topic with:

    juju action do kafka/0 create-topic topic=<topic_name> \
     partitions=<#> replication=<#>
    juju action fetch <id>  # <-- id from above command

We can list topics with:

    juju action do kafka/0 list-topics
    juju action fetch <id>  # <-- id from above command

We can write to a topic with:

    juju action do kafka/0 write-topic topic=<topic_name> data=<data>
    juju action fetch <id>  # <-- id from above command

We can read from a topic with:

    juju action do kafka/0 read-topic topic=<topic_name> partition=<#>
    juju action fetch <id>  # <-- id from above command

And finally, we can delete a topic with:

    juju action do kafka/0 delete-topic topic=<topic_name>
    juju action fetch <id>  # <-- id from above command


### Setting up a multinode cluster

Creating a cluster with many brokers is as easy as adding more units to your kafka service:

    juju add-unit kafka

After that you will be able to create topics with replication up to the number of units.

To verify replication is working you can do the following:

    juju add-unit kafka -n 2
    juju action do kafka/0 create-topic topic=my-replicated-topic \
     partitions=1 replication=2
    juju ssh kafka/0

Query for the description of the just created topic:

    kafka-topics.sh --describe --topic my-replicated-topic --zookeeper <zookeeperip>:2181

You should get a response similar to:

    Topic:my-replicated-topic	PartitionCount:1	ReplicationFactor:2	Configs:
	 Topic: my-replicated-topic	Partition: 0	Leader: 2	Replicas: 2,0	Isr: 2,0


## Deploying in Network-Restricted Environments
This charm can be deployed in environments with limited network access. To
deploy in this environment, you will need a local mirror to serve the packages
and resources required by this charm.

### Mirroring Packages
You can setup a local mirror for apt packages using squid-deb-proxy.
For instructions on configuring juju to use this, see the
[Juju Proxy Documentation](https://juju.ubuntu.com/docs/howto-proxies.html).

### Mirroring Resources
{'fraction_non_alphanumeric': 0.07489760093622001, 'fraction_numerical': 0.00994733762434172, 'mean_word_length': 3.7613741875580318, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 36, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2575970', 'n_tokens_mistral': 1587, 'n_tokens_neox': 1494, 'n_words': 666}