<commit_before>spydht
==========

Forked from [kohlrabi23/spydht](https://github.com/kohlrabi23/spydht)

Python 2 and Python 3 implementation of the Kademlia DHT data store.

Useful for distributing a key-value store in a decentralized manner.

This is a more secure rewrite of the original spydht code (itself forked from [isaaczafuta/pydht](https://github.com/isaaczafuta/pydht).)

###Improvements over the original project include:
* Node IDs switched to SHA256 to avoid collisions with MD5.
* Node IDs dereived from the network addr (host + port) to help prevent Sybil attacks (choosing IDs to cluster around a certain key.)
* Routing tables tightened up significantly - two possible entries per IP and restricted to certain ports. This avoids spam.
* Proof of work required for all store messages. Helps avoid spam.
* Size restrictions on DHT messages (currently 5 KB.)
* Proper signature validations!
* Ping commands implemented to keep routing tables fresh.
* Built in data integrity using SHA256 fingerprinting - no more spoofed responses. (Although this means that key indexing is less convenient and you can't update keys but you can delete old keys if you prove ownership.)
* Optional expiry for messages - helps prevent abuse.

I'm not an expert with DHT security so these are just basic things. Sybil attacks are always going to be a problem even with these precautions.

####xample: a two node DHT

Node 1:
```python
from spydht.spydht import DHT

import nacl.signing
import time
import hashlib

key1 = nacl.signing.SigningKey.generate()
host1, port1 = 'localhost', 3100
dht1 = DHT(host1, port1, key1, wan_ip="127.0.0.1")<commit_msg>Updated read me (again.)<commit_after>spydht
==========

Forked from [kohlrabi23/spydht](https://github.com/kohlrabi23/spydht)

Python 2 and Python 3 implementation of the Kademlia DHT data store.

Useful for distributing a key-value store in a decentralized manner.

This is a more secure rewrite of the original spydht code (itself forked from [isaaczafuta/pydht](https://github.com/isaaczafuta/pydht).)

###Improvements over the original project include:
* Node IDs switched to SHA256 to avoid collisions with MD5.
* Node IDs dereived from the network addr (host + port) to help prevent Sybil attacks (choosing IDs to cluster around a certain key.)
* Routing tables tightened up significantly - two possible entries per IP and restricted to certain ports. This avoids spam.
* Proof of work required for all store messages. Helps avoid spam.
* Size restrictions on DHT messages (currently 5 KB.)
* Proper signature validations!
* Ping commands implemented to keep routing tables fresh.
* Built in data integrity using SHA256 fingerprinting - no more spoofed responses. (Although this means that key indexing is less convenient and you can't update keys but you can delete old keys if you prove ownership.)
* Optional expiry for messages - helps prevent abuse.

I'm not an expert with DHT security so these are just basic things. Sybil attacks are always going to be a problem even with these precautions.

####Example: a two node DHT

Node 1:
```python
from spydht.spydht import DHT

import nacl.signing
import time
import hashlib

key1 = nacl.signing.SigningKey.generate()
host1, port1 = 'localhost', 3100
dht1 = DHT(host1, port1, key1, wan_ip="127.0.0.1")
{'fraction_non_alphanumeric': 0.07108287961282517, 'fraction_numerical': 0.019358741681790685, 'mean_word_length': 4.652991452991453, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1103479', 'n_tokens_mistral': 1001, 'n_tokens_neox': 916, 'n_words': 450}