<commit_before>
## Data Partitioning

As you read in the [Sharding in Hazelcast section](#sharding-in-hazelcast), Hazelcast shards are called Partitions. Partitions are memory segments each of which can contain hundreds or thousands of data entries depending on your memory capacity. 

By default, Hazelcast offers 271 partitions. Given a key, we serialize, hash and mode it with the number of partitions to find the partition the key belongs to. The partitions themselves are distributed equally among the members of the cluster. Hazelcast also creates the backups of partitions and distributes them among nodes for redundancy.

Hazelcast distributes data entries into the partitions using a hashing algorithm. The key of data is put through this algorithm and the resulted value is mod by 271. After the mod operation, the value we have addresses the place of the data in a partition. Each node in a cluster will know where the key is. 

Partitions in a 1 node Hazelcast cluster.

![](images/NodePartition.jpg)

Partitions in a 2 node cluster. 

![](images/BackupPartitions.jpg)

The blacks are primary partitions and reds are backups. In the above illustration, the first node has 135 primary partitions (black) and each of these partitions are backed up in the second node (red). At the same time, the first node has the backup partitions of the second node's primary partitions.

As you add more nodes, Hazelcast will move one by one some of the primary and backup partitions to new nodes, making all nodes equal and redundant. Only the minimum amount of partitions will be moved to scale out Hazelcast.

![](images/4NodeCluster.jpg)

<commit_msg>Data Partitioning section is growing more<commit_after>
## Data Partitioning

As you read in the [Sharding in Hazelcast section](#sharding-in-hazelcast), Hazelcast shards are called Partitions. Partitions are memory segments each of which can contain hundreds or thousands of data entries depending on your memory capacity. 

By default, Hazelcast offers 271 partitions. When you start a node, these 271 partitions will be owned by that node. The following is an illustration of the partitions in a 1 node Hazelcast cluster.

![](images/NodePartition.jpg)

When you start a second node, i.e. there will be 2 node Hazelcast cluster, the partitions will be distributed as shown in the following illlustration. 

![](images/BackupPartitions.jpg)

The blacks are primary partitions and reds are replicas (backups). In the above illustration, the first node has 135 primary partitions (black) and each of these partitions are backed up in the second node (red). At the same time, the first node has the replica partitions of the second node's primary partitions.

As you add more nodes, Hazelcast moves one by one some of the primary and replica partitions to the new nodes, making all nodes equal and redundant. Only the minimum amount of partitions will be moved to scale out Hazelcast. The following is an illustration of the partition distributions in a 4 node Hazelcast cluster.

![](images/4NodeCluster.jpg)

As you see, the partitions themselves are distributed equally among the members of the cluster. Hazelcast creates the backups of partitions and distributes them among nodes for redundancy, too.

Hazelcast distributes data entries into the partitions using a hashing algorithm. Given an object key (e.g. for map) or an object name (e.g. for topic or list):

- the key or name is serialized, i.e. converted into byte array,
- this byte array is hashed, and
- the result of the hash is mod by the number of partitions.

The result of this modulo - *MOD(hash result, partition count)* -  gives the partition in which the data will be stored.

Each node in a cluster will know where the key is. 
{'fraction_non_alphanumeric': 0.0423773987206823, 'fraction_numerical': 0.006663113006396588, 'mean_word_length': 4.502932551319648, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7581640', 'n_tokens_mistral': 971, 'n_tokens_neox': 885, 'n_words': 580}