<commit_before>kestrelweb
==========

A kestrel web ui.

Uses gunicorn + gevent for front end serving.  Also uses gevent to make async requests to the kestrel servers for stats.  It's still in early development so actual production deployment isn't supported yet.

https://github.com/matterkkila/kestrelweb


Dependencies:

 * gevent
 * gunicorn
 * pykestrel
 * dream - https://github.com/matterkkila/dream

Note: All are available on pypi except for the dream library.


Prerequisites
-------------

 * Python 2.7+
 * virtualenv (not really required, but it's a good idea)
 * libevent 1.4+ (for gevent)


Installing
----------

    git clone https://github.com/matterkkila/kestrelweb.git
    cd kestrelweb
    virtualenv --no-site-packages .
    . bin/activate
    pip install gevent
    pip install gunicorn
    pip install pykestrel
    pip install https://github.com/matterkkila/dream/tarball/master
    python ./setup.py develop


Note: If libevent is installed in a non-standard location (I'm looking at you OS X and macports) you'll need to download the source distribution for gevent and install using something similar to:

    python ./setup.py install --libevent /opt/local


Configuring
-----------

Edit the templates/index.html and change the server input to point to your kestrel servers.

Format: server:port,server:port,...


Running
-------

    gunicorn -k gevent -w 2 kestrelweb.main:App


Filtering
---------

You can filter the queue list on any field/value.  Strings only support the '=' and '!=' comparisons and use the re.match() function.  Numeric types support <, >, <=, >=, !=, = and will do the standard numeric comparison.

Examples:

 * queue=my_queue
 * queue=.*_prod$
 * server=10.0.10
 * total_items>=10
 * mem_items<=1024<commit_msg>Updated readme with better info on dependencies and deployment<commit_after>kestrelweb
==========

A kestrel web ui.

Uses gunicorn + gevent for front end serving.  Also uses gevent to make async requests to the kestrel servers for stats.  It's still in early development so actual production deployment isn't supported yet.

https://github.com/matterkkila/kestrelweb


Dependencies:

 * decoroute - used by dream
 * dream - https://github.com/matterkkila/dream
 * gevent
 * gunicorn
 * pykestrel
 * webob - used by dream

Note: All are available on pypi except for the dream library.


Prerequisites
-------------

 * Python 2.7+
 * virtualenv (not really required, but it's a good idea)
 * libevent 1.4+ (for gevent)


Installing
----------

    git clone https://github.com/matterkkila/kestrelweb.git
    cd kestrelweb
    virtualenv --no-site-packages .
    . bin/activate
    pip install gevent
    pip install gunicorn
    pip install pykestrel
    pip install webob
    pip install decoroute
    pip install https://github.com/matterkkila/dream/tarball/master
    python ./setup.py develop

Note: If libevent is installed in a non-standard location (I'm looking at you macports) you'll need to download the source distribution for gevent and install using something similar to:

    python ./setup.py install --libevent /opt/local


Configuring
-----------

Copy the local_settings.py.example to local_settings.py and edit the server list to reflect your queue servers.

    cp local_settings.py.example local_settings.py


Running
-------

    gunicorn -k gevent -w 2 kestrelweb.main:App


Deploying
---------

It's best to front gunicorn with something like nginx.  It is also best to use nginx to serve the static files used by kestrelweb.  Specifically anything in /static/* should be served by the proxy server.


Filtering
---------

You can filter the queue list on any field/value.  Strings only support the '=' and '!=' comparisons and use the re.match() function.  Numeric types support <, >, <=, >=, !=, = and will do the standard numeric comparison.

Examples:

 * queue=my_queue
 * queue!=.*_prod$
 * server=10.0.10
 * total_items>=10
 * mem_items<=1024


Feedback & Questions
--------------------

Any comments, questions, feel free to send me a message on github at https://github.com/matterkkila

All patches and pull requests are welcome.
{'fraction_non_alphanumeric': 0.10944309927360775, 'fraction_numerical': 0.00774818401937046, 'mean_word_length': 3.114541832669323, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3331565', 'n_tokens_mistral': 1263, 'n_tokens_neox': 1163, 'n_words': 465}