<commit_before>aiodcard
==============

.. image:: https://travis-ci.org/carlcarl/aiodcard.svg?branch=master
    :target: https://travis-ci.org/carlcarl/aiodcard

.. image:: https://coveralls.io/repos/carlcarl/aiodcard/badge.svg?branch=master&service=github :target: https://coveralls.io/github/carlcarl/aiodcard?branch=master

Dcard crawler using asyncio(coroutine)

Feature
-------
| Get article list and content using coroutine


Dependencies
------------
* Python 3.3 and :mod:`asyncio` or Python 3.4+
* aiohttp


Installation
------------
::

	python setup.py install

or 

::

    pip install aiodcard


Example
-------

::

    import asyncio

    import aiohttp
    import aiodcard

    @asyncio.coroutine
    def get_funny_articles():
        session = aiohttp.ClientSession()
        forum_name = 'funny'
        page_index = 1
        result = yield from aiodcard.get_articles_of_page(session, forum_name, page_index)
        print(result)

    def main():
        loop = asyncio.get_event_loop()
        loop.run_until_complete(get_funny_articles())

    if __name__ == '__main__':
        main()


Todo
----
* test all functions
  

Authors and License
-------------------
The ``aiodcard`` package is written by <NAME>. It’s MIT licensed and freely available.

Feel free to improve this package and send a pull request to GitHub.

<commit_msg>Add pypi package version badge<commit_after>aiodcard
==============

.. image:: https://badge.fury.io/py/aiodcard.svg
    :target: http://badge.fury.io/py/aiodcard
.. image:: https://travis-ci.org/carlcarl/aiodcard.svg?branch=master
    :target: https://travis-ci.org/carlcarl/aiodcard
.. image:: https://coveralls.io/repos/carlcarl/aiodcard/badge.svg?branch=master&service=github :target: https://coveralls.io/github/carlcarl/aiodcard?branch=master

Dcard crawler using asyncio(coroutine)

Feature
-------
| Get article list and content using coroutine


Dependencies
------------
* Python 3.3 and :mod:`asyncio` or Python 3.4+
* aiohttp


Installation
------------
::

	python setup.py install

or 

::

    pip install aiodcard


Example
-------

::

    import asyncio

    import aiohttp
    import aiodcard

    @asyncio.coroutine
    def get_funny_articles():
        session = aiohttp.ClientSession()
        forum_name = 'funny'
        page_index = 1
        result = yield from aiodcard.get_articles_of_page(session, forum_name, page_index)
        print(result)

    def main():
        loop = asyncio.get_event_loop()
        loop.run_until_complete(get_funny_articles())

    if __name__ == '__main__':
        main()


Todo
----
* test all functions
  

Authors and License
-------------------
The ``aiodcard`` package is written by <NAME>. It’s MIT licensed and freely available.

Feel free to improve this package and send a pull request to GitHub.


{'fraction_non_alphanumeric': 0.14842366277010272, 'fraction_numerical': 0.003542330853701736, 'mean_word_length': 2.795698924731183, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6660777', 'n_tokens_mistral': 944, 'n_tokens_neox': 904, 'n_words': 214}