<commit_before>rdd.py
======

rdd.py is a Python implementation of the [Readability Shortener API].

rdd.py comes with a Python module that can be imported via `import rdd` and a
simple command-line tool named `rdd` to utilize it.


Installation
------------

rdd.py requires [Requests] -- the excellent HTTP library by <NAME>
(@kennethreitz). [This page][Requests-install] explains how to install it.

rdd.py itself can be installed via `setup.py`:

```sh
$ python setup.py install
```


Client Usage
------------

(See [rdd(1)] manpage for more information.)

Retrieve information about sub-resources:

```sh
$ rdd resources
{
    "resources": {
        "urls/:id": {
            "href": "/api/shortener/v1/urls/:id",
            "description": "The URL endpoint. GET a URL ID to view available metadata of a shortened link."
        },
        "urls": {
            "href": "/api/shortener/v1/urls",
            "description": "The URLs endpoint. POST a URL to add it to the shortener."
        }
    }
}
```

Create a new shortened URL:

```sh
$ rdd shorten http://www.paulgraham.com/gh.html
{
    "meta": {
        "url": "/api/shortener/v1/urls/ga4qf47t",
        "rdd_url": "http://rdd.me/ga4qf47t",
        "id": "ga4qf47t"
    },
    "messages": [
        "URL shortened."
    ],
    "success": true
}
```

Retrieve available metadata of a shortened link:

```sh
$ rdd metadata ga4qf47t
{
    "meta": {
        "article": {
            "url": "http://www.paulgraham.com/gh.html",
            "title": "Great Hackers",
            "excerpt": "Want to start a startup? Get funded by Y Combinator . July 2004 (This essay is derived from a talk at Oscon 2004.) A few months ago I finished a new book , and in reviews I keep noticing words like&hellip;",
            "word_count": 5147,
            "author": null
        },
        "rdd_url": "http://rdd.me/ga4qf47t",
        "id": "ga4qf47t",
        "full_url": "http://readability.com/articles/ga4qf47t"
    },
    "messages": [
        "Article found."
    ],
    "success": true
}
```


API Usage
---------

```python
>>> import rdd
>>> readability = rdd.Readability()
>>> readability.resources()
{u'resources': {u'urls/:id': {u'href': u'/api/shortener/v1/urls/:id', u'description': u'The URL endpoint. GET a URL ID to view available metadata of a shortened link.'}, u'urls': {u'href': u'/api/shortener/v1/urls', u'description': u'The URLs endpoint. POST a URL to add it to the shortener.'}}}
>>> readability.shorten('http://www.paulgraham.com/gh.html')
{'meta': {'url': '/api/shortener/v1/urls/ga4qf47t', 'rdd_url': 'http://rdd.me/ga4qf47t', 'id': 'ga4qf47t'}, 'messages': ['URL shortened.'], 'success': True}
>>> readability.metadata('ga4qf47t')
{'meta': {'article': {'url': 'http://www.paulgraham.com/gh.html', 'title': 'Great Hackers', 'excerpt': 'Want to start a startup? Get funded by Y Combinator . July 2004 (This essay is derived from a talk at Oscon 2004.) A few months ago I finished a new book , and in reviews I keep noticing words like&hellip;', 'word_count': 5147, 'author': None}, 'rdd_url': 'http://rdd.me/ga4qf47t', 'id': 'ga4qf47t', 'full_url': 'http://readability.com/articles/ga4qf47t'}, 'messages': ['Article found.'], 'success': True}
```


Automated Tests
---------------

The `test` folder contains some automated test scripts powered by [Sharness].

You can run the tests this way:

```sh
$ make -C test/
*** t0001-resources.sh ***
ok 1 - Get information about sub-resources
# passed all 1 test(s)
1..1
*** t0002-shorten.sh ***
ok 1 - Shorten URL http://www.paulgraham.com/gh.html (ga4qf47t)
ok 2 - Shorten URL http://the99percent.com/articles/6943/What-Motivates-Us-To-Do-Great-Work (75x8oaqg)
ok 3 - Shorten URL http://www.inc.com/magazine/20100401/driven-to-distraction.html (yibs1cca)
ok 4 - Shorten URL http://www.forbes.com/sites/venkateshrao/2011/12/05/the-rise-of-developeronomics/ (c1xma652)
ok 5 - Shorten URL http://lubutu.com/idea/ivo (vqvvpmhg)
# passed all 5 test(s)
1..5
*** t0003-metadata.sh ***
ok 1 - Get metadata of ga4qf47t (http://www.paulgraham.com/gh.html)
ok 2 - Get metadata of 75x8oaqg (http://the99percent.com/articles/6943/What-Motivates-Us-To-Do-Great-Work)
ok 3 - Get metadata of yibs1cca (http://www.inc.com/magazine/20100401/driven-to-distraction.html)
ok 4 - Get metadata of c1xma652 (http://www.forbes.com/sites/venkateshrao/2011/12/05/the-rise-of-developeronomics/)
ok 5 - Get metadata of vqvvpmhg (http://lubutu.com/idea/ivo)
# passed all 5 test(s)
1..5

fixed   0
success 11
failed  0
broken  0
total   11
```


License
-------

* rdd.py is licensed under the terms of the MIT License. See [LICENSE] file.
* [Sharness] and all tests are licensed under the terms of the GNU General
  Public License version 2 or higher. See file [COPYING] for full license text.


Contact
-------

* Web: <https://github.com/mlafeldt/rdd.py>
* Mail: <<EMAIL>>
* Twitter: [@mlafeldt](https://twitter.com/mlafeldt)<commit_msg>Rework README<commit_after>rdd.py
======

rdd.py is a Python implementation of the [Readability Shortener API].

The project comes with a Python module that can be imported via `import rdd` and
a simple command-line tool named `rdd` to utilize it.


Installation
------------

rdd.py requires [Requests] - the excellent HTTP library by <NAME>.
[This page][Requests-install] explains how to install it.

rdd.py itself can be installed via `setup.py`:

    $ git clone git://github.com/mlafeldt/rdd.py.git
    $ cd rdd.py
    $ python setup.py install


Client Usage
------------

See [rdd(1)] manual page for the ins and outs of the `rrd` tool.


API Usage
---------

```python
>>> import rdd
>>> readability = rdd.Readability()
>>> readability.resources()
{u'resources': {u'urls/:id': {u'href': u'/api/shortener/v1/urls/:id', u'description': u'The URL endpoint. GET a URL ID to view available metadata of a shortened link.'}, u'urls': {u'href': u'/api/shortener/v1/urls', u'description': u'The URLs endpoint. POST a URL to add it to the shortener.'}}}
>>> readability.shorten('http://www.paulgraham.com/gh.html')
{'meta': {'url': '/api/shortener/v1/urls/ga4qf47t', 'rdd_url': 'http://rdd.me/ga4qf47t', 'id': 'ga4qf47t'}, 'messages': ['URL shortened.'], 'success': True}
>>> readability.metadata('ga4qf47t')
{'meta': {'article': {'url': 'http://www.paulgraham.com/gh.html', 'title': 'Great Hackers', 'excerpt': 'Want to start a startup? Get funded by Y Combinator . July 2004 (This essay is derived from a talk at Oscon 2004.) A few months ago I finished a new book , and in reviews I keep noticing words like&hellip;', 'word_count': 5147, 'author': None}, 'rdd_url': 'http://rdd.me/ga4qf47t', 'id': 'ga4qf47t', 'full_url': 'http://readability.com/articles/ga4qf47t'}, 'messages': ['Article found.'], 'success': True}
```


Tests
-----

The `test` folder contains some automated test scripts powered by [Sharness].

You can run the tests this way:

    $ make -C test


License
-------

* rdd.py is licensed under the terms of the MIT License. See [LICENSE] file.
* [Sharness] and all tests are licensed under the terms of the GNU General
  Public License version 2 or higher. See file [COPYING] for full license text.


Contact
-------

* Web: <https://github.com/mlafeldt/rdd.py>
* Mail: <<EMAIL>>
* Twitter: [@mlafeldt](https://twitter.com/mlafeldt)
{'fraction_non_alphanumeric': 0.16824612003845626, 'fraction_numerical': 0.02980359840681225, 'mean_word_length': 3.33969010727056, 'pattern_counts': {'":': 25, '<': 11, '<?xml version=': 0, '>': 41, 'https://': 4, 'lorem ipsum': 0, 'www.': 12, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3121543', 'n_tokens_mistral': 2685, 'n_tokens_neox': 2510, 'n_words': 769}