<commit_before>Overview
--------
[FriendFeed](http://friendfeed.com/)'s web server is a relatively simple,
non-blocking web server written in Python. The FriendFeed application is
written using a web framework that looks a bit like
[web.py](http://webpy.org/) or Google's
[webapp](http://code.google.com/appengine/docs/python/tools/webapp/),
but with additional tools and optimizations to take advantage of the
non-blocking web server and tools.

[Tornado](http://github.com/facebook/tornado) is an open source
version of this web server and some of the tools we use most often at
FriendFeed. The framework is distinct from most mainstream web server
frameworks (and certainly most Python frameworks) because it is
non-blocking and reasonably fast. Because it is non-blocking
and uses [epoll](http://www.kernel.org/doc/man-pages/online/pages/man4/epoll.4.html), it can handle 1000s of simultaneous standing connections,
which means the framework is ideal for real-time web services. We built the
web server specifically to handle FriendFeed's real-time features &mdash;
every active user of FriendFeed maintains an open connection to the
FriendFeed servers. (For more information on scaling servers to support
thousands of clients, see
[The C10K problem](http://www.kegel.com/c10k.html).)

Here is the canonical "Hello, world" example app:

    import tornado.httpserver
    import tornado.ioloop
    import tornado.web

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("Hello, world")

    application = tornado.web.Application([
        (r"/", MainHandler),
    ])

    if __name__ == "__main__":
        http_server = tornado.httpserver.HTTPServer(application)
        http_server.listen(8888)
        tornado.ioloop.IOLoop.instance().start()

See [Tornado walkthrough](#tornado-walkthrough) below for a detailed
walkthrough of the `tornado.web` package.

We attempted to clean up the code base to reduce interdependencies between
modules, so you should (theoretically) be able to use any of the modules
independently in your project without using the whole package.


Download
--------
Download the most recent version of Tornado from GitHub:

> [tornado-0.2.tar.gz](/static/tornado-0.2.tar.gz)

You can also [browse the source](http://github.com/facebook/tornado) on GitHub. To install Tornado:

    tar xvzf tornado-0.2.tar.gz
    cd tornado-0.2
    python setup.py build
    sudo python setup.py install

After installation, you should be able to run any of the demos in the `demos`
directory included with the Tornado package.

    ./demos/helloworld/helloworld.py

### Prerequisites

Tornado has been tested on Python 2.5 and 2.6. To use all of the features of Tornado, you need to have [PycURL](http://pycurl.sourceforge.net/) and a JSON library like [simplejson](http://pypi.python.org/pypi/simplejson/) installed. Complete installation instructions for Mac OS X and Ubuntu are included below for convenience.

**Mac OS X 10.5/10.6**

    sudo easy_install setuptools pycurl==7.16.2.1 simplejson

**Ubuntu Linux**

    sudo apt-get install python-dev python-pycurl python-simplejson


Module index
------------
The most important module is [`web`](http://github.com/facebook/tornado/blob/master/tornado/web.py), which is the web framework
that includes most of the meat of the Tornado package. The other modules
are tools that make `web` more useful. See
[Tornado walkthrough](#tornado-walkthrough) below for a detailed
walkthrough of the `web` package.

### Main modules
  * [`web`](http://github.com/facebook/tornado/blob/master/tornado/web.py) - The web framework on which FriendFeed is built. `web` incorporates most of the important features of Tornado
  * [`escape`](http://github.com/facebook/tornado/blob/master/tornado/escape.py) - XHTML, JSON, and URL encoding/decoding methods
  * [`database`](http://github.com/facebook/tornado/blob/master/tornado/database.py) - A simple wrapper around `MySQLdb` to make MySQL easier to use
  * [`template`](http://github.com/facebook/tornado/blob/master/tornado/template.py) - A Python-based web templating language
  * [`httpclient`](http://github.com/facebook/tornado/blob/master/tornado/httpclient.py) - A non-blocking HTTP client designed to work with `web` and `httpserver`
  * [`auth`](http://github.com/facebook/tornado/blob/master/tornado/auth.py) - Implementation of third party authentication and authorization schemes (Google OpenID/OAuth, Facebook Platform, Yahoo BBAuth, FriendFeed OpenID/OAuth, Twitter OAuth)
  * [`locale`](http://github.com/facebook/tornado/blob/master/tornado/locale.py) - Localization/translation support
  * [`options`](http://github.com/facebook/tornado/blob/master/tornado/options.py) - Command line and config file parsing, optimized for server environments

### Low-level modules
  * [`httpserver`](http://github.com/facebook/tornado/blob/master/tornado/httpserver.py) - A very simple HTTP server built on which `web` is built
  * [`iostream`](http://github.com/facebook/tornado/blob/master/tornado/iostream.py) - A simple wrapper around non-blocking sockets to aide common reading and writing patterns
  * [`ioloop`](http://github.com/facebook/tornado/blob/master/tornado/ioloop.py) - Core I/O loop

### Random modules
  * [`s3server`](http://github.com/facebook/tornado/blob/master/tornado/s3server.py) - A web server that implements most of the [Amazon S3](http://aws.amazon.com/s3/) interface, backed by local file storage


Tornado walkthrough
-------------------

### Request handlers and request arguments

A Tornado web application maps URLs or URL patterns to subclasses of
`tornado.web.RequestHandler`. Those classes define `get()` or `post()`
methods to handle HTTP `GET` or `POST` requests to that URL.

This code maps the root URL `/` to `MainHandler` and the URL pattern
`/story/([0-9]+)` to `StoryHandler`. Regular expression groups are passed
as arguments to the `RequestHandler` methods:

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("You requested the main page")

    class StoryHandler(tornado.web.RequestHandler):
        def get(self, story_id):
            self.write("You requested the story " + story_id)

    application = tornado.web.Application([
        (r"/", MainHandler),
        (r"/story/([0-9]+)", StoryHandler),
    ])

You can get query string arguments and parse `POST` bodies with the
`get_argument()` method:

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write('<html><body><form action="/" method="post">'
                       '<input type="text" name="message">'
                       '<input type="submit" value="Submit">'
                       '</form></body></html>')

        def post(self):
            self.set_header("Content-Type", "text/plain")
            self.write("You wrote " + self.get_argument("message"))

If you want to send an error response to the client, e.g., 403 Unauthorized,
you can just raise a `tornado.web.HTTPError` exception:

    if not self.user_is_logged_in():
        raise tornado.web.HTTPError(403)

The request handler can access the object representing the current request
with `self.request`. The `HTTPRequest` object includes a number of useful
attribute, including:

 * `arguments` - all of the `GET` and `POST` arguments
 * `files` - all of the uploaded files (via `multipart/form-data` POST requests)
 * `path` - the request path (everything before the `?`)<commit_msg>Correcting a grammatical error (attribute -> attributes).<commit_after>Overview
--------
[FriendFeed](http://friendfeed.com/)'s web server is a relatively simple,
non-blocking web server written in Python. The FriendFeed application is
written using a web framework that looks a bit like
[web.py](http://webpy.org/) or Google's
[webapp](http://code.google.com/appengine/docs/python/tools/webapp/),
but with additional tools and optimizations to take advantage of the
non-blocking web server and tools.

[Tornado](http://github.com/facebook/tornado) is an open source
version of this web server and some of the tools we use most often at
FriendFeed. The framework is distinct from most mainstream web server
frameworks (and certainly most Python frameworks) because it is
non-blocking and reasonably fast. Because it is non-blocking
and uses [epoll](http://www.kernel.org/doc/man-pages/online/pages/man4/epoll.4.html), it can handle 1000s of simultaneous standing connections,
which means the framework is ideal for real-time web services. We built the
web server specifically to handle FriendFeed's real-time features &mdash;
every active user of FriendFeed maintains an open connection to the
FriendFeed servers. (For more information on scaling servers to support
thousands of clients, see
[The C10K problem](http://www.kegel.com/c10k.html).)

Here is the canonical "Hello, world" example app:

    import tornado.httpserver
    import tornado.ioloop
    import tornado.web

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("Hello, world")

    application = tornado.web.Application([
        (r"/", MainHandler),
    ])

    if __name__ == "__main__":
        http_server = tornado.httpserver.HTTPServer(application)
        http_server.listen(8888)
        tornado.ioloop.IOLoop.instance().start()

See [Tornado walkthrough](#tornado-walkthrough) below for a detailed
walkthrough of the `tornado.web` package.

We attempted to clean up the code base to reduce interdependencies between
modules, so you should (theoretically) be able to use any of the modules
independently in your project without using the whole package.


Download
--------
Download the most recent version of Tornado from GitHub:

> [tornado-0.2.tar.gz](/static/tornado-0.2.tar.gz)

You can also [browse the source](http://github.com/facebook/tornado) on GitHub. To install Tornado:

    tar xvzf tornado-0.2.tar.gz
    cd tornado-0.2
    python setup.py build
    sudo python setup.py install

After installation, you should be able to run any of the demos in the `demos`
directory included with the Tornado package.

    ./demos/helloworld/helloworld.py

### Prerequisites

Tornado has been tested on Python 2.5 and 2.6. To use all of the features of Tornado, you need to have [PycURL](http://pycurl.sourceforge.net/) and a JSON library like [simplejson](http://pypi.python.org/pypi/simplejson/) installed. Complete installation instructions for Mac OS X and Ubuntu are included below for convenience.

**Mac OS X 10.5/10.6**

    sudo easy_install setuptools pycurl==7.16.2.1 simplejson

**Ubuntu Linux**

    sudo apt-get install python-dev python-pycurl python-simplejson


Module index
------------
The most important module is [`web`](http://github.com/facebook/tornado/blob/master/tornado/web.py), which is the web framework
that includes most of the meat of the Tornado package. The other modules
are tools that make `web` more useful. See
[Tornado walkthrough](#tornado-walkthrough) below for a detailed
walkthrough of the `web` package.

### Main modules
  * [`web`](http://github.com/facebook/tornado/blob/master/tornado/web.py) - The web framework on which FriendFeed is built. `web` incorporates most of the important features of Tornado
  * [`escape`](http://github.com/facebook/tornado/blob/master/tornado/escape.py) - XHTML, JSON, and URL encoding/decoding methods
  * [`database`](http://github.com/facebook/tornado/blob/master/tornado/database.py) - A simple wrapper around `MySQLdb` to make MySQL easier to use
  * [`template`](http://github.com/facebook/tornado/blob/master/tornado/template.py) - A Python-based web templating language
  * [`httpclient`](http://github.com/facebook/tornado/blob/master/tornado/httpclient.py) - A non-blocking HTTP client designed to work with `web` and `httpserver`
  * [`auth`](http://github.com/facebook/tornado/blob/master/tornado/auth.py) - Implementation of third party authentication and authorization schemes (Google OpenID/OAuth, Facebook Platform, Yahoo BBAuth, FriendFeed OpenID/OAuth, Twitter OAuth)
  * [`locale`](http://github.com/facebook/tornado/blob/master/tornado/locale.py) - Localization/translation support
  * [`options`](http://github.com/facebook/tornado/blob/master/tornado/options.py) - Command line and config file parsing, optimized for server environments

### Low-level modules
  * [`httpserver`](http://github.com/facebook/tornado/blob/master/tornado/httpserver.py) - A very simple HTTP server built on which `web` is built
  * [`iostream`](http://github.com/facebook/tornado/blob/master/tornado/iostream.py) - A simple wrapper around non-blocking sockets to aide common reading and writing patterns
  * [`ioloop`](http://github.com/facebook/tornado/blob/master/tornado/ioloop.py) - Core I/O loop

### Random modules
  * [`s3server`](http://github.com/facebook/tornado/blob/master/tornado/s3server.py) - A web server that implements most of the [Amazon S3](http://aws.amazon.com/s3/) interface, backed by local file storage


Tornado walkthrough
-------------------

### Request handlers and request arguments

A Tornado web application maps URLs or URL patterns to subclasses of
`tornado.web.RequestHandler`. Those classes define `get()` or `post()`
methods to handle HTTP `GET` or `POST` requests to that URL.

This code maps the root URL `/` to `MainHandler` and the URL pattern
`/story/([0-9]+)` to `StoryHandler`. Regular expression groups are passed
as arguments to the `RequestHandler` methods:

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("You requested the main page")

    class StoryHandler(tornado.web.RequestHandler):
        def get(self, story_id):
            self.write("You requested the story " + story_id)

    application = tornado.web.Application([
        (r"/", MainHandler),
        (r"/story/([0-9]+)", StoryHandler),
    ])

You can get query string arguments and parse `POST` bodies with the
`get_argument()` method:

    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write('<html><body><form action="/" method="post">'
                       '<input type="text" name="message">'
                       '<input type="submit" value="Submit">'
                       '</form></body></html>')

        def post(self):
            self.set_header("Content-Type", "text/plain")
            self.write("You wrote " + self.get_argument("message"))

If you want to send an error response to the client, e.g., 403 Unauthorized,
you can just raise a `tornado.web.HTTPError` exception:

    if not self.user_is_logged_in():
        raise tornado.web.HTTPError(403)

The request handler can access the object representing the current request
with `self.request`. The `HTTPRequest` object includes a number of useful
attributes, including:

 * `arguments` - all of the `GET` and `POST` arguments
 * `files` - all of the uploaded files (via `multipart/form-data` POST requests)
 * `path` - the request path (everything before the `?`)
{'fraction_non_alphanumeric': 0.11515715329922253, 'fraction_numerical': 0.006777858993953087, 'mean_word_length': 3.816, 'pattern_counts': {'":': 2, '<': 19, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '653050', 'n_tokens_mistral': 4497, 'n_tokens_neox': 4298, 'n_words': 1471}