<commit_before>## Technology Stack

### Main dependencies
 
* Python - Implementation language
* concurrent.futures - Standard package for concurrence in Python
* Django - Used by the API server and the WebUI
* HDF5 - Used for storing and managing data
* numpy and scipy - Fundamental packages for scientific computing with Python

### Optional dependencies

* Rtree - a wrapper of libspatialindex that provides a number of advanced spatial indexing features 
* Celery - Distributed task queue library, using the `iterator_native()`
* RabbitMQ - Message broker for Celery tasks, logging channels, and other signalling

### Binary dependencies

Software  | Version(s)
--------- | ----------
Python | 2.7, 3.5
RabbitMQ | 2.6 to 3.2
libgeos | >= 3.2.2
HDF5 | 1.8


### Python dependencies

See the '[setup.py](../setup.py) file for a complete list of the Python dependencies.<commit_msg>Small update in doc<commit_after>## Technology Stack

### Main dependencies
 
* Python - Implementation language
* concurrent.futures - Standard package for concurrence in Python
* Django - Used by the API server and the WebUI
* HDF5 - Used for storing and managing data
* numpy and scipy - Fundamental packages for scientific computing with Python

### Optional dependencies

* Rtree - a wrapper of libspatialindex that provides a number of advanced spatial indexing features 
* Celery - Distributed task queue library, using the `iterator_native()`
* RabbitMQ - Message broker for Celery tasks, logging channels, and other signalling

### Binary dependencies

Software  | Version(s)
--------- | ----------
Python | 2.7, 3.5
RabbitMQ | 2.6 to 3.2
libgeos | >= 3.2.2
HDF5 | 1.8


### Python dependencies

See [setup.py](../setup.py) for a complete list of the Python dependencies.
{'fraction_non_alphanumeric': 0.10034013605442177, 'fraction_numerical': 0.017006802721088437, 'mean_word_length': 3.7192513368983957, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2838140', 'n_tokens_mistral': 517, 'n_tokens_neox': 462, 'n_words': 210}