<commit_before>.. pyAggr3g470r documentation master file, created by
   sphinx-quickstart on Sat Sep 15 08:55:50 2012.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to pyAggr3g470r's documentation!
========================================

.. toctree::
   :maxdepth: 2


Presentation
------------

pyAggr3g470r_ is a news aggregator with a web interface
based on CherryPy_. Articles are stored in a MongoDB_ base.

* `Ohloh page <http://www.ohloh.net/p/pyAggr3g470r>`_ of pyAggr3g470r;
* `Freecode page <http://freecode.com/projects/pyaggr3g470r>`_ of pyAggr3g470r


Features
--------

* articles are stored in a MongoDB_ database (local or remote);
* article searching;
* e-mail notification;
* export articles to HTML, EPUB, PDF or raw text;
* favorite articles;
* sharing with Diaspora, Google Buzz, Pinboard, Identi.ca, Digg, reddit, Scoopeo, Blogmarks and Twitter;
* generation of QR codes with URLs of articles.


Installation
------------
Requierements
~~~~~~~~~~~~~
Software required
`````````````````

* Python_ >= 3.2.3;
* MongoDB_ and PyMongo_ >= 1.9;
* feedparser_ >= 5.1.2 (for **feedgetter.py**, the RSS feed parser);
* CherryPy_ >= 3.2.2 and Mako_ (for **pyAggr3g470r.py**, the Web interface);
* BeautifulSoup_ >= 4.1.3 (automatically find a feed in a HTML page).


Optional module
```````````````

These modules are not required but enables more features:

* lxml and Genshi;
* Python Imaging Library for the generation of QR codes.

Not yet tested with Python 3.2. Anyway, if you want to install these modules:

.. code-block:: bash

    $ sudo aptitude install python3.2-lxml python-genshi


Script of installation
~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

    sudo aptitude install python3-feedparser python3-bs4
    sudo aptitude install python-pymongo python3-mako
    wget http://download.cherrypy.org/cherrypy/3.2.2/CherryPy-3.2.2.tar.gz
    tar -xzvf CherryPy-3.2.2.tar.gz
    rm -f CherryPy-3.2.2.tar.gz
    cd CherryPy-3.2.2/
    sudo python setup.py install
    cd ..
    sudo rm -Rf CherryPy-3.2.2/
    hg clone https://bitbucket.org/cedricbonhomme/pyaggr3g470r
    cd pyaggr3g470r/
    cp cfg/pyAggr3g470r.cfg-sample cfg/pyAggr3g470r.cfg


Setting
~~~~~~~

List of feeds
`````````````

Rename the file **./cfg/pyAggr3g470r.cfg-sample** to **./cfg/pyAggr3g470r.cfg**.
By default you don't have to edit this file (only to configure mail notification).

Then, indicate the feeds to retrieve in the file **./var/feed.lst**. One feed per line. For example :

.. code-block:: cfg

    http://blog.cedricbonhomme.org/feed/
    http://linuxfr.org/backend/news-homepage/rss20.rss
    http://rss.slashdot.org/Slashdot/slashdot
    http://theinvisiblethings.blogspot.com/feeds/posts/default
    http://torvalds-family.blogspot.com/feeds/posts/default
    http://www.python.org/channews.rdf
    http://www.kde.org/dotkdeorg.rdf
    http://feeds.feedburner.com/internetactu/bcmJ
    http://www.april.org/fr/rss.xml
    http://www.framablog.org/index.php/feed/atom
    http://formats-ouverts.org/rss.php
    http://lwn.net/headlines/newrss
    http://kernelnewbies.org/RecentChanges?action=rss_rc&ddiffs=1&unique=1


Create a new user
`````````````````
Add a username and a password in the file **./var/password** separated by a semicolon, one per line.

The default user is *admin* with the password *admin*. The password should be hashed with the SHA1 function.



Launch
------
To launch pyAggr3g470r in a shell:

.. code-block:: bash

    $ cd ~/pyaggr3g470r/source/
    $ pyAggr3g470r start



Backup
------
If you want to backup your database:

.. code-block:: bash

    $ su
    $ /etc/init.d/mongodb stop
    $ cp /var/lib/mongodb/pyaggr3g470r.* ~


Demo
----
* some `screen shots <http://picasaweb.google.com/cedric.photos/PyAggr3g470r>`_ of **pyAggr3g470r**;
* an old `video <http://youtu.be/Eyxpqn9Rpnw>`_ of pyAggr3g470r;
* an example of `HTML auto-generated archive <http://cedricbonhomme.org/news/>`_.



Donnation
---------
If you wish and if you like pyAggr3g470r, you can donate via bitcoin.

My bitcoin address: 1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ

Thank you!



License
-------
pyAggr3g470r_ is under GPLv3_ license.



Contact
-------
`My home page <http://cedricbonhomme.org/>`_.



.. _Python: http://python.org/
.. _pyAggr3g470r: https://bitbucket.org/cedricbonhomme/pyaggr3g470r/
.. _feedparser: http://feedparser.org/
.. _MongoDB: http://www.mongodb.org/
.. _PyMongo: https://github.com/mongodb/mongo-python-driver
.. _CherryPy: http://cherrypy.org/
.. _Mako: http://www.makotemplates.org/
.. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
.. _GPLv3: http://www.gnu.org/licenses/gpl-3.0.txt
<commit_msg>Setting email notifications.<commit_after>.. pyAggr3g470r documentation master file, created by
   sphinx-quickstart on Sat Sep 15 08:55:50 2012.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to pyAggr3g470r's documentation!
========================================

.. toctree::
   :maxdepth: 2


Presentation
------------

pyAggr3g470r_ is a news aggregator with a web interface
based on CherryPy_. Articles are stored in a MongoDB_ base.

* `Ohloh page <http://www.ohloh.net/p/pyAggr3g470r>`_ of pyAggr3g470r;
* `Freecode page <http://freecode.com/projects/pyaggr3g470r>`_ of pyAggr3g470r


Features
--------

* articles are stored in a MongoDB_ database (local or remote);
* article searching;
* e-mail notification;
* export articles to HTML, EPUB, PDF or raw text;
* favorite articles;
* sharing with Diaspora, Google Buzz, Pinboard, Identi.ca, Digg, reddit, Scoopeo, Blogmarks and Twitter;
* generation of QR codes with URLs of articles.


Installation
------------
Requierements
~~~~~~~~~~~~~
Software required
`````````````````

* Python_ >= 3.2.3;
* MongoDB_ and PyMongo_ >= 1.9;
* feedparser_ >= 5.1.2 (for **feedgetter.py**, the RSS feed parser);
* CherryPy_ >= 3.2.2 and Mako_ (for **pyAggr3g470r.py**, the Web interface);
* BeautifulSoup_ >= 4.1.3 (automatically find a feed in a HTML page).


Optional module
```````````````

These modules are not required but enables more features:

* lxml and Genshi;
* Python Imaging Library for the generation of QR codes.

Not yet tested with Python 3.2. Anyway, if you want to install these modules:

.. code-block:: bash

    $ sudo aptitude install python3.2-lxml python-genshi


Script of installation
~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

    sudo aptitude install python3-feedparser python3-bs4
    sudo aptitude install python-pymongo python3-mako
    wget http://download.cherrypy.org/cherrypy/3.2.2/CherryPy-3.2.2.tar.gz
    tar -xzvf CherryPy-3.2.2.tar.gz
    rm -f CherryPy-3.2.2.tar.gz
    cd CherryPy-3.2.2/
    sudo python setup.py install
    cd ..
    sudo rm -Rf CherryPy-3.2.2/
    hg clone https://bitbucket.org/cedricbonhomme/pyaggr3g470r
    cd pyaggr3g470r/
    cp cfg/pyAggr3g470r.cfg-sample cfg/pyAggr3g470r.cfg


Setting
~~~~~~~

List of feeds
`````````````

Rename the file **./cfg/pyAggr3g470r.cfg-sample** to **./cfg/pyAggr3g470r.cfg**.
By default you don't have to edit this file (only to configure mail notification).

Then, indicate the feeds to retrieve in the file **./var/feed.lst**. One feed per line. For example :

.. code-block:: cfg

    http://blog.cedricbonhomme.org/feed/
    http://linuxfr.org/backend/news-homepage/rss20.rss
    http://rss.slashdot.org/Slashdot/slashdot
    http://theinvisiblethings.blogspot.com/feeds/posts/default
    http://torvalds-family.blogspot.com/feeds/posts/default
    http://www.python.org/channews.rdf
    http://www.kde.org/dotkdeorg.rdf
    http://feeds.feedburner.com/internetactu/bcmJ
    http://www.april.org/fr/rss.xml
    http://www.framablog.org/index.php/feed/atom
    http://formats-ouverts.org/rss.php
    http://lwn.net/headlines/newrss
    http://kernelnewbies.org/RecentChanges?action=rss_rc&ddiffs=1&unique=1


Create a new user
`````````````````
Add a username and a password in the file **./var/password** separated by a semicolon, one per line.

The default user is *admin* with the password *<PASSWORD>*. The password should be hashed with the SHA1 function.

Mail notification
`````````````````
If you wish to stay tuned from new articles of a feed by email, you have to edit the configuration file:

* your mail address (address of the recipient of the news);
* a SMTP server;
* the address used by pyAggr3g470r to send the mail, and;
* the corresponding password. 

.. code-block:: cfg

    [MongoDB]
    address = mongodb.alwaysdata.com
    port = 27017
    dbname = bob_pyaggr3g470r
    user = bob
    password = <PASSWORD>
    [mail]
    mail_from = <EMAIL>
    mail_to = <EMAIL>
    smtp = smtp.example.com
    username = <EMAIL>
    password = <PASSWORD>
    [misc]
    diaspora_pod = joindiaspora.com

Then just click on "Stay tuned" for the desired feed(s) at the main page of pyAggr3g470r in your browser.


Launch
------
To launch pyAggr3g470r in a shell:

.. code-block:: bash

    $ cd ~/pyaggr3g470r/source/
    $ pyAggr3g470r start



Backup
------
If you want to backup your database:

.. code-block:: bash

    $ su
    $ /etc/init.d/mongodb stop
    $ cp /var/lib/mongodb/pyaggr3g470r.* ~


Demo
----
* some `screen shots <http://picasaweb.google.com/cedric.photos/PyAggr3g470r>`_ of **pyAggr3g470r**;
* an old `video <http://youtu.be/Eyxpqn9Rpnw>`_ of pyAggr3g470r;
* an example of `HTML auto-generated archive <http://cedricbonhomme.org/news/>`_.



Donnation
---------
If you wish and if you like pyAggr3g470r, you can donate via bitcoin.

My bitcoin address: 1GVmhR9fbBeEh7rP1qNq76jWArDdDQ3otZ

Thank you!



License
-------
pyAggr3g470r_ is under GPLv3_ license.



Contact
-------
`My home page <http://cedricbonhomme.org/>`_.



.. _Python: http://python.org/
.. _pyAggr3g470r: https://bitbucket.org/cedricbonhomme/pyaggr3g470r/
.. _feedparser: http://feedparser.org/
.. _MongoDB: http://www.mongodb.org/
.. _PyMongo: https://github.com/mongodb/mongo-python-driver
.. _CherryPy: http://cherrypy.org/
.. _Mako: http://www.makotemplates.org/
.. _BeautifulSoup: http://www.crummy.com/software/BeautifulSoup/
.. _GPLv3: http://www.gnu.org/licenses/gpl-3.0.txt

{'fraction_non_alphanumeric': 0.15290907318974759, 'fraction_numerical': 0.034791930611051555, 'mean_word_length': 3.3612409689757756, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 31, 'https://': 6, 'lorem ipsum': 0, 'www.': 18, 'xml': 6}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '601368', 'n_tokens_mistral': 3980, 'n_tokens_neox': 3606, 'n_words': 959}