<commit_before>.. highlight:: none

Installation
============

First, get a copy of dryscrape_ using Git:

::

  $ git clone https://github.com/niklasb/dryscrape.git dryscrape
  $ cd dryscrape

To install dryscrape, you first need to install its dependencies. These are:

* lxml_
* webkit-server_

You can use pip_ to do this for you (while still in
the dryscrape directory). The ``#`` indicates that you need root privileges for
this:

::

  # pip -r requirements.txt

If you want, you can of course also install the dependencies manually.

Afterwards, you can use the ``setup.py`` script included to install dryscrape:

::

  # python setup.py install

Note that dryscrape is written for Python 2.7+, so make sure that you use both the
``python`` and ``pip`` commands in the right version.

.. _lxml: http://lxml.de/
.. _webkit-server: https://github.com/niklasb/webkit-server/
.. _pip: http://pypi.python.org/pypi/pip
.. _dryscrape: https://github.com/niklasb/dryscrape/<commit_msg>fix wrong command line in installation instructions<commit_after>.. highlight:: none

Installation
============

First, get a copy of dryscrape_ using Git:

::

  $ git clone https://github.com/niklasb/dryscrape.git dryscrape
  $ cd dryscrape

To install dryscrape, you first need to install its dependencies. These are:

* lxml_
* webkit-server_

You can use pip_ to do this for you (while still in
the dryscrape directory). The ``#`` indicates that you need root privileges for
this:

::

  # pip install -r requirements.txt

If you want, you can of course also install the dependencies manually.

Afterwards, you can use the ``setup.py`` script included to install dryscrape:

::

  # python setup.py install

Note that dryscrape is written for Python 2.7+, so make sure that you use both the
``python`` and ``pip`` commands in the right version.

.. _lxml: http://lxml.de/
.. _webkit-server: https://github.com/niklasb/webkit-server/
.. _pip: http://pypi.python.org/pypi/pip
.. _dryscrape: https://github.com/niklasb/dryscrape/
{'fraction_non_alphanumeric': 0.11607142857142858, 'fraction_numerical': 0.001984126984126984, 'mean_word_length': 3.8485576923076925, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 6}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1640106', 'n_tokens_mistral': 674, 'n_tokens_neox': 621, 'n_words': 240}