<commit_before>How to use it
=============

Installing the Django app
-------------------------

The latest official release of the application can be installed from the Python Package Index using ``pip``.

.. code-block:: bash

    $ pip install django-calaccess-raw-data

The app needs to be added to the ``INSTALLED_APPS`` in your Django project's settings.

.. code-block:: python

    INSTALLED_APPS = (
        # ... other apps up here ...
        'calaccess_raw',
    )

Also in the Django settings, create a database configure Django so it can connect to it. You have a choice of two different databases when using this application, MySQL and PostgreSQL.

Using MySQL
~~~~~~~~~~~

Add a database connection string like this to your ``settings.py``.

.. code-block:: python

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'calaccess',
            'USER': 'username',
            'PASSWORD': 'password',
            'HOST': 'localhost',
            'PORT': '3306',
            # You'll need this to use our data loading tricks
            'OPTIONS': {
                'local_infile': 1,
            }
        }
    }

This will create the MySQL database.

.. code-block:: bash

    $ mysqladmin -h localhost -u root -p create calaccess

Using PostgreSQL
~~~~~~~~~~~~~~~~

Add a database connection string like this to your ``settings.py``.

.. code-block:: python

    DATABASES = {
        'default': {
            'NAME': 'calaccess',
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'USER': 'username',
            'PASSWORD': 'password',
            'HOST': 'localhost',
            'PORT': '5432'
        }
    }

This will create the PostgreSQL database.

.. code-block:: bash

    $ createdb calaccess

Loading the data
----------------

Now you're ready to sync the database tables.

.. code-block:: bash

    $ python manage.py migrate

Once everything is set up, this management command will download the latest bulk data release from the state
and load it in the database. This'll take a while. Go grab some coffee.

.. code-block:: bash

    $ python manage.py downloadcalaccessrawdata<commit_msg>Improved docs for #233 with database wrapper requirements<commit_after>How to use it
=============

Installing the Django app
-------------------------

The latest official release of the application can be installed from the Python Package Index using ``pip``.

.. code-block:: bash

    $ pip install django-calaccess-raw-data

The app needs to be added to the ``INSTALLED_APPS`` in your Django project's settings.

.. code-block:: python

    INSTALLED_APPS = (
        # ... other apps up here ...
        'calaccess_raw',
    )

Also in the Django settings, create a database configure Django so it can connect to it. You have a choice of two different databases when using this application, MySQL and PostgreSQL.

Using MySQL
~~~~~~~~~~~

Add a database connection string like this to your ``settings.py``.

.. code-block:: python

    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'calaccess',
            'USER': 'username',
            'PASSWORD': 'password',
            'HOST': 'localhost',
            'PORT': '3306',
            # You'll need this to use our data loading tricks
            'OPTIONS': {
                'local_infile': 1,
            }
        }
    }

This will create the MySQL database.

.. code-block:: bash

    $ mysqladmin -h localhost -u root -p create calaccess
    
If you don't have it already, you'll need to install a Python library that can access MySQL via Django.

.. code-block:: bash

    $ pip install mysqlclient

Using PostgreSQL
~~~~~~~~~~~~~~~~

Add a database connection string like this to your ``settings.py``.

.. code-block:: python

    DATABASES = {
        'default': {
            'NAME': 'calaccess',
            'ENGINE': 'django.db.backends.postgresql_psycopg2',
            'USER': 'username',
            'PASSWORD': 'password',
            'HOST': 'localhost',
            'PORT': '5432'
        }
    }

This will create the PostgreSQL database.

.. code-block:: bash

    $ createdb calaccess

If you don't have it already, you'll need to install a Python library that can access PostgreSQL via Django.

.. code-block:: bash

    $ pip install psycopg2

Loading the data
----------------

Now you're ready to sync the database tables.

.. code-block:: bash

    $ python manage.py migrate

Once everything is set up, this management command will download the latest bulk data release from the state
and load it in the database. This'll take a while. Go grab some coffee.

.. code-block:: bash

    $ python manage.py downloadcalaccessrawdata
{'fraction_non_alphanumeric': 0.1258902387934646, 'fraction_numerical': 0.005027230833682447, 'mean_word_length': 2.293103448275862, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6301027', 'n_tokens_mistral': 1366, 'n_tokens_neox': 1264, 'n_words': 514}