<commit_before>GLUDB
=====

|Travis CI Build State| `Travis CI
Details <https://travis-ci.org/memphis-iis/GLUDB>`__

Generalized Learning Utilities Database Library
-----------------------------------------------

For more GLU, see also
`SuperGLU <https://github.com/GeneralizedLearningUtilities/SuperGLU>`__

GLUDB is meant to provide a fairly simple way to read/write data to some
popular datastores (like DynamoDB and Google Cloud Datastore). Depending
on the data backend, we hope to provide:

-  A simple abstraction layer for annotating classes that should be
   stored in the database
-  Support for versioning by automatically storing change history when
   possible
-  A lightweight permissions module for handling authenticated read and
   write
-  Limited query support
-  Automated, configurable backup scheduling

To claim version 1.0 status, we will need to implement backend adaptors
for: \* DynamoDB \* Google Cloud Datastore \* MongoDB

We are currently planning on supporting both Python 2 (2.7+) and Python
3 (3.4+). *However*, please keep in mind that we will be emphasizing
Python 3 going forward.

Installing
----------

You can install from PyPI using pip:

::

    pip install gludb

You will also need to install any dependencies you need based on the
functionality you want to use:

-  DynamoDB Backend - boto
-  Google Cloud Datastore - googledatastore
-  MongoDB - pymongo
-  Backups - boto

These dependencies are included in setup.py so that you can install them
all at the same time (assuming a fairly recent version of pip). As an
example, you could install gludb and the dependencies needed for
dynamodb and backup support into a virtualenv using Python 3 like this:

::

    user@host:~$ virtualenv -p python3 env
    user@host:~$ . env/bin/activate
    user@host:~$ pip install --upgrade pip wheel
    user@host:~$ pip install gludb[dynamodb,backups]

Testing
-------

Please see tests/README.md for details.

Top-level directory contents
----------------------------

Some newcomers might find the top-level directory for a little
overwhelming. Here's a complete breakdown of what you see here (in
alphabetical order):

-  .gitignore - This lists files that will be automatically ignored by
   git
-  .travis.yml - The configuration file use by Travis CI (our continuous
   integration provider)
-  build - Created when you use ./build.sh (in .gitignore)
-  build.sh - A script to build distributions of the gludb library
   suitable for use by others (and upload to PyPI)
-  dev-requirements.txt - Requirements to be installed into a virtualenv
   used for development on gludb. Mainly used by our testing scripts
-  dev-requirements-27.txt - Development requirements specific to Python
   2.7. This is currently necessary because we can only support Google
   Cloud Datastore on Python 2.7
-  dist - Holds the final output of the ./build.sh command (in
   .gitignore)
-  docs - Our documentation in mkdocs format (which is one of the two
   formats used by ReadTheDocs.org)
-  examples - Simple examples for gludb use
-  gludb - Our actual package info
-  gludb.egg-info - Created as part of a process developing the gludb
   package
-  LICENSE - The license under which the contents of this repository are
   placed
-  mkdocs.yml - The configuration file for the documentation in the docs
   directory
-  README.md - This file
-  setup.cfg - Parameters used by setup.py
-  setup.py - The main setup file for the gludb package - used by
   ./build.sh and other sources for install, remove, etc
-  tests - Location for our unit tests

.. |Travis CI Build State| image:: https://travis-ci.org/memphis-iis/GLUDB.svg?branch=master
<commit_msg>rst check [ci skip]<commit_after>GLUDB
=====

|Travis CI Build State| `Travis CI
Details <https://travis-ci.org/memphis-iis/GLUDB>`__

|Documentation State| See our documentation at
http://gludb.readthedocs.org/

Generalized Learning Utilities Database Library
-----------------------------------------------

For more GLU, see also
`SuperGLU <https://github.com/GeneralizedLearningUtilities/SuperGLU>`__

GLUDB provides a fairly simple way to read/write data to some popular
datastores like Amazon's DynamoDB and Google Cloud Datastore. We
provide:

-  A simple abstraction layer for annotating classes that should be
   stored in the database
-  Support for versioning by automatically storing change history with
   the data
-  Automated "indexing", which includes querying on the value of indexes
-  Automated, configurable backup to Amazon's S3 (and Glacier depending
   on how you configure the S3 buckets)

We currently support Python 2 (2.7 and greater) and 3 (3.4 and greater).
The data stores currently supported are:

-  sqlite
-  DynamoDB
-  Google Cloud Datastore
-  MongoDB

Installing
----------

You can install from PyPI using pip:

::

    pip install gludb

You will also need to install any dependencies you need based on the
functionality you want to use:

-  DynamoDB Backend - boto
-  Google Cloud Datastore - googledatastore
-  MongoDB - pymongo
-  Backups - boto

These dependencies are included in setup.py so that you can install them
all at the same time (assuming a fairly recent version of pip). As an
example, you could install gludb and the dependencies needed for
dynamodb and backup support into a virtualenv using Python 3 like this:

::

    user@host:~$ virtualenv -p python3 env
    user@host:~$ . env/bin/activate
    user@host:~$ pip install --upgrade pip wheel
    user@host:~$ pip install gludb[dynamodb,backups]

.. |Travis CI Build State| image:: https://travis-ci.org/memphis-iis/GLUDB.svg?branch=master
.. |Documentation State| image:: https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat

{'fraction_non_alphanumeric': 0.09027414003841452, 'fraction_numerical': 0.004714510214772132, 'mean_word_length': 3.336109008327025, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4682595', 'n_tokens_mistral': 1675, 'n_tokens_neox': 1617, 'n_words': 722}