<commit_before># Cell Cycle Specification
A proposal for Elastic Distributed Shared Memory in the Cloud Application

# Needs

## Not functional
* You can choose the programming language
*  You can use support libraries and tools to develop your project 
*  System/service with configurable parameters (no hard-coded!)–  Through a configuration file/service 
*  You must test all the functionalities of your developed system/service and present and discuss the testing results in the project report
*  System/service supports multiple, autonomous entities contending for shared resources
*  System/service supports real-time updates to some form of shared state
*  System/service state should be distributed across multiple client or server nodes
–  The only allowed centralized service can be one that supports users logging on, adding or removing clients or servers, and other housekeeping tasks 
*  System/service scalability and elasticity
*  System/service fault tolerance, in particular system/ service continues operation even if one of the participant nodes crashes (optionally, recovers the state of a crashed node so that it can resume operation)

## Functional
* Realize a distributed shared memory system that supports application scale-up and
scale-down
*  Design a totally elastic solution: add new nodes and remove existing nodes without restarting other nodes in the system
*  Scale-down can be challenging due to need to perform state reintegration
*  Handle node failover
*  Support at least two consistency models
*  Some examples: 
https://github.com/memcached/memcached
https://github.com/hazelcast/hazelcast




# Target
Cells Cycle is a P2P architecture for a distributed shared memory in the cloud, elastic. 
Informations are stored like KeyValue objects, values are raw data.

* Memory - System is able to store KeyValue objects, in this case volatile memory like DRAM
* Distributed - System is composed by different machines where software is executing, all cooperating
* Shared - All machines have their own ram. Memory available to all machines is used to offer memory storage service.
* Elastic - System is able to scale up and scale down, using the numbers of active machines. 

# Overview
Nodes of the system are structured as a circle, a cycle. Every element is linked before and after with another node. Nodes are numbered clockwise through float ids.
Each node splits his ram to manage his data (he’s the Master of this data) and backup data of previous node (he’s the Slave of this data).
So, each node is a Master, while it’s the slave of the previous node. 

Values have an expiration time, after that the value can be deleted from the system (if it wasn’t updated, in that case the expiration date is incremented).

Every node could be an entry point for the memory provider service.

# Name Space of keys
To split name space of the keys, every node chooses a contiguous set of keys. Every node continues the name space set of the previous one.

## Example: 
Name space is an integer between 0 and 127. 
* Node 1 has [0,31] set as Memory Master, and [96, 127] as Slave Memory.
* Node 2 has [32, 63] set as Memory Master, and [0,31] as Slave Memory.
* Node 3 has [64, 95] set as Memory Master, and [32, 63] as Slave Memory.
* Node 4 has [96, 127] set as Memory Master, and [64, 95] as Slave Memory.

Values have an expiration time, after that the value can be deleted from the system (if it wasn’t updated, in that case the expiration date is incremented).

Every node could be an entry point for the memory provider service.
 
# Cycle Operations
## Adding a node (cell duplication)
When a node runs off available ram, he can create a new Slave after him. 
The name of the new node is a float number, between the name of the full node and the next one. When the new node boots up, Master memory of the first node is transferred to new one. 
First node drops the second half of his Master keys, transferring to new one. 
The next node now it’s the slave of the new node, not the first anymore. So he has to drop first half of his Slave keys.
The new node takes Master keys from the first node and splits it in his Master keys (first half) and Slave keys (second half). 
Responsibilities of the full node is now been splitted into two nodes, like a cell division. 

Every node adding operation involves only 3 node: the full node, the next one and the new node. 
The computational cost of the adding operation is the transferring of Master keys of full node (we can accept this cost, we assume that system runs on a high speed intranet).





### Example (node 2 is running off ram, it creates node 3):
id | Master Memory | Slave Memory
-------|----------------|---------------
1 | [0,63] | [64,128]-->[96,127] 
2 | [64,127]-->[64,95] | [0,63]
3 | [96,127] | [64,95]

### Example (node 2 is running off ram again, it creates node 2.5):<commit_msg>added example to readme.md<commit_after># Cell Cycle Specification
A proposal for Elastic Distributed Shared Memory in the Cloud Application

# Needs

## Not functional
* You can choose the programming language
*  You can use support libraries and tools to develop your project 
*  System/service with configurable parameters (no hard-coded!)–  Through a configuration file/service 
*  You must test all the functionalities of your developed system/service and present and discuss the testing results in the project report
*  System/service supports multiple, autonomous entities contending for shared resources
*  System/service supports real-time updates to some form of shared state
*  System/service state should be distributed across multiple client or server nodes
–  The only allowed centralized service can be one that supports users logging on, adding or removing clients or servers, and other housekeeping tasks 
*  System/service scalability and elasticity
*  System/service fault tolerance, in particular system/ service continues operation even if one of the participant nodes crashes (optionally, recovers the state of a crashed node so that it can resume operation)

## Functional
* Realize a distributed shared memory system that supports application scale-up and
scale-down
*  Design a totally elastic solution: add new nodes and remove existing nodes without restarting other nodes in the system
*  Scale-down can be challenging due to need to perform state reintegration
*  Handle node failover
*  Support at least two consistency models
*  Some examples: 
https://github.com/memcached/memcached
https://github.com/hazelcast/hazelcast




# Target
Cells Cycle is a P2P architecture for a distributed shared memory in the cloud, elastic. 
Informations are stored like KeyValue objects, values are raw data.

* Memory - System is able to store KeyValue objects, in this case volatile memory like DRAM
* Distributed - System is composed by different machines where software is executing, all cooperating
* Shared - All machines have their own ram. Memory available to all machines is used to offer memory storage service.
* Elastic - System is able to scale up and scale down, using the numbers of active machines. 

# Overview
Nodes of the system are structured as a circle, a cycle. Every element is linked before and after with another node. Nodes are numbered clockwise through float ids.
Each node splits his ram to manage his data (he’s the Master of this data) and backup data of previous node (he’s the Slave of this data).
So, each node is a Master, while it’s the slave of the previous node. 

Values have an expiration time, after that the value can be deleted from the system (if it wasn’t updated, in that case the expiration date is incremented).

Every node could be an entry point for the memory provider service.

# Name Space of keys
To split name space of the keys, every node chooses a contiguous set of keys. Every node continues the name space set of the previous one.

## Example: 
Name space is an integer between 0 and 127. 
* Node 1 has [0,31] set as Memory Master, and [96, 127] as Slave Memory.
* Node 2 has [32, 63] set as Memory Master, and [0,31] as Slave Memory.
* Node 3 has [64, 95] set as Memory Master, and [32, 63] as Slave Memory.
* Node 4 has [96, 127] set as Memory Master, and [64, 95] as Slave Memory.

Values have an expiration time, after that the value can be deleted from the system (if it wasn’t updated, in that case the expiration date is incremented).

Every node could be an entry point for the memory provider service.
 
![picture alt](https://github.com/AQuadroTeam/CellsCycle/blob/master/cellCycleexample.png)
 
# Cycle Operations
## Adding a node (cell duplication)
When a node runs off available ram, he can create a new Slave after him. 
The name of the new node is a float number, between the name of the full node and the next one. When the new node boots up, Master memory of the first node is transferred to new one. 
First node drops the second half of his Master keys, transferring to new one. 
The next node now it’s the slave of the new node, not the first anymore. So he has to drop first half of his Slave keys.
The new node takes Master keys from the first node and splits it in his Master keys (first half) and Slave keys (second half). 
Responsibilities of the full node is now been splitted into two nodes, like a cell division. 

Every node adding operation involves only 3 node: the full node, the next one and the new node. 
The computational cost of the adding operation is the transferring of Master keys of full node (we can accept this cost, we assume that system runs on a high speed intranet).





### Example (node 2 is running off ram, it creates node 3):
id | Master Memory | Slave Memory
-------|----------------|---------------
1 | [0,63] | [64,128]-->[96,127] 
2 | [64,127]-->[64,95] | [0,63]
3 | [96,127] | [64,95]

### Example (node 2 is running off ram again, it creates node 2.5):
{'fraction_non_alphanumeric': 0.058573165788406975, 'fraction_numerical': 0.017024726388325903, 'mean_word_length': 4.0147357723577235, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3643664', 'n_tokens_mistral': 2630, 'n_tokens_neox': 2413, 'n_words': 1548}