<commit_before>=====
Shiva
=====

Dinosaurs will slowly die.


License
=======

Please read the LICENSE file distributed with this software.


What is Shiva?
==============

The Mozilla Hacks blog kindly published a nice article that explains the ideas
that inspire this software:
`Shiva: More than a RESTful API to your music collection
<https://hacks.mozilla.org/2013/03/shiva-more-than-a-restful-api-to-your-music-collection/>`_


Prerequisites
=============

You need ``ffmpeg`` installed in your system.

If you want Shiva to automatically fetch artists' images from Last.FM while
indexing you are going to need an API key. You can get one at
http://www.last.fm/api/account/create

This makes the whole indexing slower because issues a request on a per-album
and per-artist basis, but does a lot of work automatically for you.

You will need C headers for libxml. On Debian (and derivatives)::

    sudo apt-get install libxml2-dev libxslt-dev

On Mac OS X with `homebrew <http://mxcl.github.com/homebrew/>`_ you can get the
headers with::

    brew install libxml2 libxslt

By default Shiva uses a SQLite database, to install it in Debian-based systems
run::

    sudo apt-get install sqlite

On Mac OS X it should come pre-installed. If it's not::

    brew install sqlite


Installation
============

* Get the source::

.. code:: sh

      $ git clone https://github.com/tooxie/shiva-server.git
      $ cd shiva-server

* Create and activate your virtalenv (highly recommended)::

.. code:: sh

    $ virtualenv .
    $ source bin/activate

* Install::

.. code:: sh

    $ python setup.py develop

* Rename shiva/config/local.py.example to local.py::

.. code:: sh

      $ cp shiva/config/local.py.example shiva/config/local.py

* Edit it and configure the directories to scan for music.

  + See `Scanning directories`_ for more info.

* Run the indexer::

.. code:: sh

  $ shiva-indexer

* Run the server::

.. code:: sh

  $ shiva-server

* Point your browser to a Resource, like: http://127.0.0.1:9002/artists (See `Resources`_)


-----------------
Indexer arguments
-----------------

The indexer receives the following command line arguments.

* ``--lastfm``
* ``--nometadata``
* ``--reindex``

If you set the ``--lastfm`` flag Shiva will retrieve artist and album images
from Last.FM, but for this to work you need to get an API key (see
`Prerequisites`_) and include it in your ``local.py`` config file.

The ``--nometadata`` option saves dummy tracks with only path information,
ignoring the file's metadata. This means that album and artists will not be
saved.

If both flags are set, ``--nometadata`` will take precedence and ``--lastfm``
will be ignored.

With ``--reindex`` the whole database will be dropped and recreated. Be
careful, all existing information **will be deleted**. If you just want to
update your music collection, run the indexer again **without** the
``--reindex`` option.


--------------------
Scanning directories
--------------------

To tell Shiva which directories to scan for music, you will have to configure
your ``shiva/config/local.py`` file. There you will find a ``MEDIA_DIRS``
option where you need to supply ``MediaDir`` objects.

This object allows for media configuration. By instantiating a ``MediaDir``
class you can tell Shiva where to look for the media files and how to serve
those files. It's possible to configure the system to look for files on a
directory and serve those files through a different server.

.. code:: python

    MediaDir(root='/srv/http', dirs=('/music', '/songs),
             url='http://localhost:8080/')

Given that configuration Shiva will scan the directories ``/srv/http/music``
and ``/srv/http/songs`` for media files, but they will be served through
``http://localhost:8080/music/`` and ``http://localhost:8080/songs/``.

If just a dir is provided Shiva will serve it through the same instance. This
is **NOT** recommended, but is useful for developing.

.. code:: python

    MediaDir('/home/fatmike/music')

For more information, check the source of `shiva/media.py`.

Restricting extensions
----------------------<commit_msg>Doc typo<commit_after>=====
Shiva
=====

Dinosaurs will slowly die.


License
=======

Please read the LICENSE file distributed with this software.


What is Shiva?
==============

The Mozilla Hacks blog kindly published a nice article that explains the ideas
that inspire this software:
`Shiva: More than a RESTful API to your music collection
<https://hacks.mozilla.org/2013/03/shiva-more-than-a-restful-api-to-your-music-collection/>`_


Prerequisites
=============

You need ``ffmpeg`` installed in your system.

If you want Shiva to automatically fetch artists' images from Last.FM while
indexing you are going to need an API key. You can get one at
http://www.last.fm/api/account/create

This makes the whole indexing slower because issues a request on a per-album
and per-artist basis, but does a lot of work automatically for you.

You will need C headers for libxml. On Debian (and derivatives)::

    sudo apt-get install libxml2-dev libxslt-dev

On Mac OS X with `homebrew <http://mxcl.github.com/homebrew/>`_ you can get the
headers with::

    brew install libxml2 libxslt

By default Shiva uses a SQLite database, to install it in Debian-based systems
run::

    sudo apt-get install sqlite

On Mac OS X it should come pre-installed. If it's not::

    brew install sqlite


Installation
============

* Get the source::

.. code:: sh

      $ git clone https://github.com/tooxie/shiva-server.git
      $ cd shiva-server

* Create and activate your virtalenv (highly recommended)::

.. code:: sh

    $ virtualenv .
    $ source bin/activate

* Install::

.. code:: sh

    $ python setup.py develop

* Rename shiva/config/local.py.example to local.py::

.. code:: sh

      $ cp shiva/config/local.py.example shiva/config/local.py

* Edit it and configure the directories to scan for music.

  + See `Scanning directories`_ for more info.

* Run the indexer::

.. code:: sh

  $ shiva-indexer

* Run the server::

.. code:: sh

  $ shiva-server

* Point your browser to a Resource, like: http://127.0.0.1:9002/artists (See `Resources`_)


-----------------
Indexer arguments
-----------------

The indexer receives the following command line arguments.

* ``--lastfm``
* ``--nometadata``
* ``--reindex``

If you set the ``--lastfm`` flag Shiva will retrieve artist and album images
from Last.FM, but for this to work you need to get an API key (see
`Prerequisites`_) and include it in your ``local.py`` config file.

The ``--nometadata`` option saves dummy tracks with only path information,
ignoring the file's metadata. This means that album and artists will not be
saved.

If both flags are set, ``--nometadata`` will take precedence and ``--lastfm``
will be ignored.

With ``--reindex`` the whole database will be dropped and recreated. Be
careful, all existing information **will be deleted**. If you just want to
update your music collection, run the indexer again **without** the
``--reindex`` option.


--------------------
Scanning directories
--------------------

To tell Shiva which directories to scan for music, you will have to configure
your ``shiva/config/local.py`` file. There you will find a ``MEDIA_DIRS``
option where you need to supply ``MediaDir`` objects.

This object allows for media configuration. By instantiating a ``MediaDir``
class you can tell Shiva where to look for the media files and how to serve
those files. It's possible to configure the system to look for files on a
directory and serve those files through a different server.

.. code:: python

    MediaDir(root='/srv/http', dirs=('/music', '/songs'),
             url='http://localhost:8080/')

Given that configuration Shiva will scan the directories ``/srv/http/music``
and ``/srv/http/songs`` for media files, but they will be served through
``http://localhost:8080/music/`` and ``http://localhost:8080/songs/``.

If just a dir is provided Shiva will serve it through the same instance. This
is **NOT** recommended, but is useful for developing.

.. code:: python

    MediaDir('/home/fatmike/music')

For more information, check the source of `shiva/media.py`.

Restricting extensions
----------------------
{'fraction_non_alphanumeric': 0.13039786511402232, 'fraction_numerical': 0.00727802037845706, 'mean_word_length': 3.3394736842105264, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 6}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '281021', 'n_tokens_mistral': 2469, 'n_tokens_neox': 2312, 'n_words': 1037}