<commit_before># OSS: A build-in distributed Object Storage Service for dockyard
![oss-arch](../docs/oss-arch.jpg "Dockyard")

## Architecture of OSS
OSS=Object Storage Service ,which consists of three parts: APIServer , ChunkMaster and Chunkserver

- APIServer is a stateless proxy service, which provides RESTful API to upload/download and manage image files and get Chunkserver info and file id from chunkmaster. APIServer  choose a suitable chunkserver group to storage image according to Chunkserver information independently.

- ChunkMaster  is a  master node designed to maintain chunkserver information and allocate the file id

- Chunkserver is a storage node for performance and space efficiency. It appends single small image file into large files and maintain file index in memory keeping the IO overhead to a minimum. Normally, a Chunkserver group is consist of 3 Chunkservers and each image allocated to a Chunkserver group stored in all the Chunkservers.

besides, OSS needs the collaboration of a metadb, which stored metadata of images in key-value manner.

## Example of configs 

```ini
[oss]
ossmode= allinone
masterhost = 127.0.0.1
masterport = 8099
metahost = 10.229.40.121
metaport = 3306
dbuser = root
dbpasswd = <PASSWORD>
db = speedy1
limitcsnum = 1
connpoolcapacity = 200
servers = 1_127.0.0.1:7657;1_127.0.0.1:7658;1_127.0.0.1:7659
errlogpath = /usr/local/chunkserver/errlog
datapath = /usr/local/chunkserver/data
chunknum = 2
apiport= 80
apihttpsport=443
partsizemb = 4
```<commit_msg>Update README of OSS<commit_after># OSS: A build-in distributed Object Storage Service for dockyard
![oss-arch](../docs/oss-arch.jpg "Dockyard")

## Architecture of OSS
OSS=Object Storage Service ,which consists of three parts: APIServer , ChunkMaster and Chunkserver

- APIServer is a stateless proxy service, which provides RESTful API to upload/download and manage image files and get Chunkserver info and file id from chunkmaster. APIServer  choose a suitable chunkserver group to storage image according to Chunkserver information independently.

- ChunkMaster  is a  master node designed to maintain chunkserver information and allocate the file id

- Chunkserver is a storage node for performance and space efficiency. It appends single small image file into large files and maintain file index in memory keeping the IO overhead to a minimum. Normally, a Chunkserver group is consist of 3 Chunkservers and each image allocated to a Chunkserver group stored in all the Chunkservers.

besides, OSS needs the collaboration of a metadb, which stored metadata of images in key-value manner.

## Example of backend driver configs setting in `conf/runtime.conf`

```ini
[oss]
apiport= 80
apihttpsport=443
partsizemb = 4
```
the config of OSS itself please see `oss/oss.conf`
{'fraction_non_alphanumeric': 0.05738880918220947, 'fraction_numerical': 0.027618364418938307, 'mean_word_length': 4.810416666666667, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 5, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7097716', 'n_tokens_mistral': 826, 'n_tokens_neox': 770, 'n_words': 366}