<commit_before>==========
HTTPStream
==========

HTTPStream is an HTTP client library for Python that is designed to allow
incremental receipt and handling of web content as opposed to full document
retrieval. Additionally, large JSON documents may be parsed incrementally and
recombined into a series of smaller objects and arrays as required.


Installation
============

HTTPStream is hosted on PyPI and so to install, simply use pip::

    pip install httpstream


No external dependencies are required, the entire package is self-contained.


Basic Usage
===========

HTTPStream is based around {{{Resource}}} objects which are defined by a URI
and may be accessed via methods such as {{{get}}} and {{{post}}}. A
{{{Resource}}} may be declared as follows::

    >>> from httpstream import Resource
    >>> res = Resource("http://example.com/")


Once defined, accessor methods can be used on a resource in much the same way
as the Python built-in method, {{{open}}}. These return file-like
{{{Response}}} objects which may be read in full or iterated through. To
consume the entire response, simply use the {{{read}}} method on the response::

    >>> print res.get().read()
    <!DOCTYPE html><commit_msg>fixed monospace<commit_after>==========
HTTPStream
==========

HTTPStream is an HTTP client library for Python that is designed to allow
incremental receipt and handling of web content as opposed to full document
retrieval. Additionally, large JSON documents may be parsed incrementally and
recombined into a series of smaller objects and arrays as required.


Installation
============

HTTPStream is hosted on PyPI and so to install, simply use pip::

    pip install httpstream


No external dependencies are required, the entire package is self-contained.


Basic Usage
===========

HTTPStream is based around ``Resource`` objects which are defined by a URI
and may be accessed via methods such as ``get`` and ``post``. A ``Resource``
may be declared as follows::

    >>> from httpstream import Resource
    >>> res = Resource("http://example.com/")


Once defined, accessor methods can be used on a resource in much the same way
as the Python built-in method, ``open``. These return file-like ``Response``
objects which may be read in full or iterated through. To consume the entire
response, simply use the ``read`` method on the response::

    >>> print res.get().read()
    <!DOCTYPE html>
{'fraction_non_alphanumeric': 0.1078838174273859, 'fraction_numerical': 0.0, 'mean_word_length': 4.4795454545454545, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7092184', 'n_tokens_mistral': 607, 'n_tokens_neox': 571, 'n_words': 328}