<commit_before>
hashfs API service
==================

Summary
-------
This is an API service that provides a hash-based REST GET/PUT, providing
content-addressible data storage services.

The intention is to have many people/organizations run a hashfs service,
creating a decentralized storage mesh.

This project aims towards a robust, high performance API service
using django, nginx, uwsgi and 21BC.


Theory of Operation
-------------------
The interface is very REST-ful, and intended to behave similarly
to Amazon S3 storage API, which remembers metadata such as content-type.

API users upload data using PUT.  Data is guaranteed to be stored
_at least_ until its expiration date (T + 24 hours), if not longer.

Data may live longer than the minimum guaranteed lifetime.  Data is
removed only when the system reaches total storage capacity, a new data
item is to be added, and sufficient expired data exists to be freed.
HASHFS_MAX_GB in settings.py controls system storage capacity.

In addition, API users may specify a public key hash (bitcoin address)
for authenticating certain requests such as delete (when implemented).


API Overview
============

* GET /
* GET /hashfs/1/$hash
* PUT /hashfs/1/$hash

$hash is the hex-encoded SHA256 hash of the data.

API Pricing
===========

* GET requests: 1 satoshi/request + 2 satoshis/MB transfer
* PUT requests: 1 satoshi/request

Installation
============

Install packages
----------------

	sudo apt-get install sqlite3 python3-django nginx \
		python3-djangorestframework \
		uwsgi-plugin-python3 uwsgi python3-uwsgidecorators

nginx setup & restart
---------------------

	etc/apiservice.conf to /etc/nginx/sites-enabled

create metadata database for storing file information
-----------------------------------------------------

	./mkdb.sh

run uwsgi
---------

	uwsgi_python3 --ini uwsgi.ini

test your endpoint!
-------------------

stop the server
---------------

	uwsgi --stop /tmp/hashfs-uwsgi.pid

API Reference
=============

GET /
-----
Returns: application/json list describing all services at this endpoint

Example output (excluding comments):

    [
        {<commit_msg>README: update for new django-less setup<commit_after>
hashfs API service
==================

Summary
-------
This is an API service that provides a hash-based REST GET/PUT, providing
content-addressible data storage services.

The intention is to have many people/organizations run a hashfs service,
creating a decentralized storage mesh.

This project aims towards a robust, high performance API service
using django, nginx, uwsgi and 21BC.


Theory of Operation
-------------------
The interface is very REST-ful, and intended to behave similarly
to Amazon S3 storage API, which remembers metadata such as content-type.

API users upload data using PUT.  Data is guaranteed to be stored
_at least_ until its expiration date (T + 24 hours), if not longer.

Data may live longer than the minimum guaranteed lifetime.  Data is
removed only when the system reaches total storage capacity, a new data
item is to be added, and sufficient expired data exists to be freed.
HASHFS_MAX_GB in settings.py controls system storage capacity.

In addition, API users may specify a public key hash (bitcoin address)
for authenticating certain requests such as delete (when implemented).


API Overview
============

* GET /
* GET /hashfs/1/$hash
* PUT /hashfs/1/$hash

$hash is the hex-encoded SHA256 hash of the data.

API Pricing
===========

* GET requests: 1 satoshi/request + 2 satoshis/MB transfer
* PUT requests: 1 satoshi/request

Installation
============

Install packages
----------------

	sudo apt-get install sqlite3

Create metadata database for storing file information
-----------------------------------------------------

	./mkdb.sh

Run the server
--------------

	python3 hashfs-server.py

Now test your endpoint!

API Reference
=============

GET /
-----
Returns: application/json list describing all services at this endpoint

Example output (excluding comments):

    [
        {
{'fraction_non_alphanumeric': 0.14490958632648007, 'fraction_numerical': 0.008422095615556105, 'mean_word_length': 3.005952380952381, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1475425', 'n_tokens_mistral': 1090, 'n_tokens_neox': 1022, 'n_words': 490}