<commit_before>Scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data called Neurodata Web Services(formerly called the Open Connectome Project).

[![Neurodata.io](https://img.shields.io/badge/Visit-neurodata.io-ff69b4.svg)](http://neurodata.io/)
[![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.19972.svg)](http://dx.doi.org/10.5281/zenodo.19972)
[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)
[![Docs](https://img.shields.io/badge/Docs-latest-brightgreen.svg)](http://docs.neurodata.io/ndstore/)

Root directory of the cutout and annotation services.
Major directories include:

  * spdb -- [Spatial Database submodule](https://github.com/neurodata/spdb)
  * webservices -- Webservices module
  * ndproj -- Project Module
  * ramon -- RAMON and Annotation metadata module
  * django -- Django module
  * setup -- Setup instructions
  * test -- Test module
  * examples -- How to use the service
  * util -- Useful common files across the modules
  * ingest -- Scripts to insert datasets into the databases
  * admin -- Scripts to manage the databases
  * ndlib -- [Common library and Ctype accelerations submodule](https://github.com/neurodata/ndlib)
  * docs -- Sphinx documentation for the project
  * cython -- Cython acclerations for the service (Deprecated)
  * scripts -- Useful general purpose scripts
  * onetime -- Misc scripts for the service
  * NOTE: git submodule init and git submodule update for all submodules

#### Architecture

![](./docs/images/neurodata_cluster.png)

#### Benchmarks

The benchmarks were performed on AWS EC2 instance type i2.8xlarge with a MySQL backend.

##### Read Throughput

![](./docs/images/neurodata_read_throughput.png| width=250)

##### Write Throughput

![](./docs/images/neurodata_write_throughput.png)<commit_msg>[docs] Reverting back<commit_after>Scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data called Neurodata Web Services(formerly called the Open Connectome Project).

[![Neurodata.io](https://img.shields.io/badge/Visit-neurodata.io-ff69b4.svg)](http://neurodata.io/)
[![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.19972.svg)](http://dx.doi.org/10.5281/zenodo.19972)
[![Hex.pm](https://img.shields.io/hexpm/l/plug.svg)](http://www.apache.org/licenses/LICENSE-2.0.html)
[![Docs](https://img.shields.io/badge/Docs-latest-brightgreen.svg)](http://docs.neurodata.io/ndstore/)

Root directory of the cutout and annotation services.
Major directories include:

  * spdb -- [Spatial Database submodule](https://github.com/neurodata/spdb)
  * webservices -- Webservices module
  * ndproj -- Project Module
  * ramon -- RAMON and Annotation metadata module
  * django -- Django module
  * setup -- Setup instructions
  * test -- Test module
  * examples -- How to use the service
  * util -- Useful common files across the modules
  * ingest -- Scripts to insert datasets into the databases
  * admin -- Scripts to manage the databases
  * ndlib -- [Common library and Ctype accelerations submodule](https://github.com/neurodata/ndlib)
  * docs -- Sphinx documentation for the project
  * cython -- Cython acclerations for the service (Deprecated)
  * scripts -- Useful general purpose scripts
  * onetime -- Misc scripts for the service
  * NOTE: git submodule init and git submodule update for all submodules

#### Architecture

![](./docs/images/neurodata_cluster.png)

#### Benchmarks

The benchmarks were performed on AWS EC2 instance type i2.8xlarge with a MySQL backend.

##### Read Throughput

![](./docs/images/neurodata_read_throughput.png)

##### Write Throughput

![](./docs/images/neurodata_write_throughput.png)
{'fraction_non_alphanumeric': 0.12991178829190056, 'fraction_numerical': 0.016840417000801924, 'mean_word_length': 4.112021857923497, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 12, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5225904', 'n_tokens_mistral': 1231, 'n_tokens_neox': 1118, 'n_words': 312}