<commit_before>===================================
Self-Documentation for Pyramid Apps
===================================

.. warning::

  2013/09/13: though functional, this package is pretty new... Come
  back in a couple of weeks if you don't like living on the
  beta-edge!

A pyramid plugin that describes a pyramid application URL hierarchy,
either by responding to an HTTP request or on the command line, via
application inspection and reflection. It has built-in support for
plain-text hierachies, reStructuredText, HTML, PDF, JSON, YAML, WADL,
and XML, however other custom formats can be added easily.

Exposing an application's structure via HTTP is useful to dynamically
generate an API description (via WADL, JSON, or YAML) or to create
documentation directly from source code.

On the command-line it is useful to get visibility into an
application's URL structure and hierarchy so that it can be understood
and maintained.

.. note::

  Although pyramid-describe is intended to be able to describe any
  kind of pyramid application, currently it only supports
  pyramid-controllers_ based dispatch.

.. note::

  Currently, pyramid-describe can only inspect the first Controller
  it finds -- this will eventually be fixed to correctly implement
  the `inspect` option.


Project Info
============

* Project Page: https://github.com/cadithealth/pyramid_describe
* Bug Tracking: https://github.com/cadithealth/pyramid_describe/issues


TL;DR
=====

Install:

.. code-block:: bash

  $ pip install pyramid-describe

Command-line example:

.. code-block:: bash

  $ pdescribe example.ini --format txt
  /                       # The application root.
  ├── contact/            # Contact manager.
  │   ├── <POST>          # Creates a new 'contact' object.
  │   └── {CONTACTID}     # RESTful access to a specific contact.
  │       ├── <DELETE>    # Delete this contact.
  │       ├── <GET>       # Get this contact's details.
  │       └── <PUT>       # Update this contact's details.
  ├── login               # Authenticate against the server.
  └── logout              # Remove authentication tokens.

.. TODO - figure out how to serve these assets with the correct Content-Type...

Examples of the above application in all other formats with built-in
support are available at:
`text (pure-ASCII) <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.txt.asc>`_,
`reStructuredText <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.rst>`_,
`HTML <http://htmlpreview.github.io/?https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.html>`_,
`PDF <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.pdf>`_,
`JSON <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.json>`_,
`YAML <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.yaml>`_,
`WADL <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.wadl>`_,
and `XML <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.xml>`_.


Installation
============

Install with the usual python mechanism, e.g. here with ``pip``:

.. code-block:: bash

  $ pip install pyramid-describe


Usage
=====

There are three mechanisms to use pyramid-describe: via standard
pyramid inclusion which will add routes to the current application, by
explicitly embedding a ``pyramid_describe.DescribeController``
instance, or by directly calling the ``pyramid_describe.Describer``
object methods.


Pyramid Inclusion
=================

Pyramid-describe can be added via standard pyramid inclusion, either
in the INI file or directly in your `main` function. For example:

.. code-block:: python

  def main(global_config, **settings):
    # ...
    config.include('pyramid_describe')

When using pyramid inclusion, pyramid-describe expects to find
configuration options in the application settings. See the `Options`_
section for a listed of all supported options, with a short example
here:

.. code-block:: ini

  [app:main]

  describe.attach                        = /doc
  describe.formats                       = html json pdf
  describe.format.default.title          = My Application
  describe.format.html.default.cssPath   = myapp:static/doc.css
  describe.entries.filters               = myapp.describe.entry_docify

Note that multiple describers, each with different configurations, can
be added via pyramid inclusion by using the `describe.prefixes`
option.


DescribeController
==================

Pyramid-describe can also be added to your application by embedding a
DescribeController object. The DescribeController constructor takes
the following parameters:

`view`:

  An instance of ``pyramid.interfaces.IView``, which is the view that
  should be inspected and reflected.

`root`:

  The root path to the specified URL, so that host-relative URLs can
  be generated to the views found.

`settings`:

  A dictionary of all the options to apply to this describer. Note that
  in this case, the options should not have any prefix.

Example:

.. code-block:: python

  from pyramid_describe import DescribeController

  def main(global_config, **settings):
    # ...
    config.include('pyramid_controllers')

    settings = {
        'formats'                       : ['html', 'json', 'pdf'],
        'format.default.title'          : 'My Application',
        'format.html.default.cssPath'   : 'myapp:static/doc.css',
        'entries.filters'               : 'myapp.describe.entry_docify',
      }

    config.add_controller('MyAppDescriber', '/doc', DescribeController(settings))


Describer
=========

Pyramid-describe can also be added to your application by directly
calling the Describer's functionality. This is an even lower-level
approach than, but still quite similar to, embedding the
`DescribeController`_; the constructor takes the same `settings`
parameter as the DescribeController, and then a call to the `describe`
method actually generates the output. The `describe` method takes as
parameters a `context` and a `format`, and returns a dictionary with
the following attributes:

.. TODO - document `context` and `format`...

`content_type`:

  The MIME content-type associated with the rendered output.

`charset`:

  The character set that the output is encoded in.

`content`:

  The actual rendering output.

Example:

.. code-block:: python

  from pyramid_describe import Describer

  def my_describer(request):

    settings = {
        'formats'                       : ['html', 'json', 'pdf'],
        'format.default.title'          : 'My Application',
        'format.html.default.cssPath'   : 'myapp:static/doc.css',
        'entries.filters'               : 'myapp.describe.entry_docify',
      }

    describer = Describer(settings=settings)
    context   = dict(request=request)
    result    = describer.describe(context=context, format='pdf')

    request.response.content_type = result['content_type']
    request.response.charset      = result['charset']
    request.response.body         = result['content']

    return request.response


Options
=======

The configuration of pyramid-describe is done by setting any of the
following options. Note that if specified in the application settings
(i.e. the INI file), then they must be prefixed with (by default)
``describe.``. Otherwise, when passing a dictionary of settings to the
constructors, the prefix is left off. The following options exist:

* ``describe.prefixes`` : list(str), default: 'describe'

  Defines the prefix or the list of prefixes that pyramid-describe
  settings will be searched for in the configuration. For each prefix,
  a separate DescribeController will be created and attached to the
  application router. The following example attaches two controllers
  at ``/desc-one`` and ``/desc-two``:

  .. code-block:: ini

    [app:main]
    describe.prefixes = describe-one describe-two
    describe-one.attach  = /desc-one
    # other `describe-one` options...
    describe-two.attach  = /desc-two
    # other `describe-two` options...

* ``describe.class`` : resolve-spec, default: pyramid_describe.DescribeController

  Sets the global default Controller class that will be instantiated
  for each of the stanzas defined in `describe.prefixes`. Note that
  this option can be overriden on a per-stanza basis.

* ``{PREFIX}.class`` : resolve-spec, default: `describe.class`

  Sets the Controller class that will be instantiated for this PREFIX
  stanza, overriding `describe.class`.

* ``{PREFIX}.attach`` : str, default: /describe

  Specifies the path to attach the controller to the current
  application's router. Note that this uses the `add_controller`
  directive, and ensures that pyramid-controllers has already been
  added via an explicit call to ``config.include()``. This path will
  serve the default format: to request alternate formats, use
  "PATH/FILENAME.EXT" (where FILENAME is controlled by the
  ``{PREFIX}.filename`` configuration and EXT specifies the format)
  or use the "format=EXT" query-string. Examples using the default
  settings:

  .. code-block:: text

    http://localhost:8080/describe/application.txt
    http://localhost:8080/describe/application.json
    http://localhost:8080/describe?format=json

* ``{PREFIX}.fullname`` : str, default: 'application'

  Sets the filename (excluding the extension) that the output will be
  served at using the DescribeController. The extension provided by
  the request will determine which format to serve, and must be listed
  in the `formats` option. If the format is not listed, a 404 is
  returned. Typically, this is set to the application's name and
  might also include the application version.

* ``{PREFIX}.basename`` : str, default: null

  Similar to the `fullname` option, this option sets a filename base
  component that will either redirect to the current `fullname` or
  actually serve the content based on the `base-redirect` option. This
  allows there to be a persistent known location that can be used if
  the `filename` option is dynamic or changes with revisions.

* ``{PREFIX}.index-redirect`` : { bool, int, str }, default: true

  Controls what happens when a request comes to the index location
  of the DescribeController, i.e. the value of the `attach` option.
  The following values are accepted:

  falsy

    Responds with the actual content using the default format.

  truthy

    Redirects with a 302 to the `basename` if set, otherwise to
    the `fullname`, using the default format's extension.

  int

    Same as if truthy, but uses the specified response code (e.g.
    301 instead of 302).

  str

    Responds with a redirect using the specified string as the
    ``Location`` header. By default, issues a 302 unless the string is
    prefixed with the code and a space, e.g. ``301
    /path/to/filename``. If the location is not absolute, it will be
    evaluated relative to the current URL.

* ``{PREFIX}.base-redirect`` : { bool, int, str }, default: true

  If `basename` is set, then this controls how the response is handled
  -- see the `index-redirect` option for accepted values, with the
  adjustment that the default redirect location is the `fullname`.

* ``{PREFIX}.inspect`` : str, default: /

  Specifies the top-level URL to start the application inspection at.

  TODO: this does not work.

  WARNING: this does not work.

  SERIOUSLY: this does not work, it only adds the specified path as a
  URL prefix... doh!

* ``{PREFIX}.include`` : list(regex-spec), default: null

  The `include` option lists encapsulated regular expressions that an
  endpoint must match at least one of in order to be included in the
  output. This option can be used with the `exclude` option, in which
  case endpoints are first matched for inclusion, then matched for
  exclusion (i.e. the order is "allow,deny" in apache terminology).

  Encapsulated regular expressions are expressed in the syntax
  "/EXPR/FLAGS", where the "/" can be replaced by any character
  otherwise not found in the rest of the expression. The flags can
  be any combination of the following characters:

  * ``i``: Case-insensitive matching.
  * ``l``: Use locale-dependent processing (for \w, \W, etc.).<commit_msg>updated docs for setting `describe.inspect`<commit_after>===================================
Self-Documentation for Pyramid Apps
===================================

A pyramid plugin that describes a pyramid application URL hierarchy,
either by responding to an HTTP request or on the command line, via
application inspection and reflection. It has built-in support for
plain-text hierachies, reStructuredText, HTML, PDF, JSON, YAML, WADL,
and XML, however other custom formats can be added easily.

Exposing an application's structure via HTTP is useful to dynamically
generate an API description (via WADL, JSON, or YAML) or to create
documentation directly from source code.

On the command-line it is useful to get visibility into an
application's URL structure and hierarchy so that it can be understood
and maintained.

.. note::

  Although pyramid-describe is intended to be able to describe any
  kind of pyramid application, currently it only supports
  pyramid-controllers_ based dispatch.

.. note::

  Currently, pyramid-describe can only inspect the first Controller
  it finds -- this will eventually be fixed to correctly implement
  the `inspect` option.


Project Info
============

* Project Page: https://github.com/cadithealth/pyramid_describe
* Bug Tracking: https://github.com/cadithealth/pyramid_describe/issues


TL;DR
=====

Install:

.. code-block:: bash

  $ pip install pyramid-describe

Command-line example:

.. code-block:: bash

  $ pdescribe example.ini --format txt
  /                       # The application root.
  ├── contact/            # Contact manager.
  │   ├── <POST>          # Creates a new 'contact' object.
  │   └── {CONTACTID}     # RESTful access to a specific contact.
  │       ├── <DELETE>    # Delete this contact.
  │       ├── <GET>       # Get this contact's details.
  │       └── <PUT>       # Update this contact's details.
  ├── login               # Authenticate against the server.
  └── logout              # Remove authentication tokens.

.. TODO - figure out how to serve these assets with the correct Content-Type...

Examples of the above application in all other formats with built-in
support are available at:
`text (pure-ASCII) <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.txt.asc>`_,
`reStructuredText <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.rst>`_,
`HTML <http://htmlpreview.github.io/?https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.html>`_,
`PDF <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.pdf>`_,
`JSON <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.json>`_,
`YAML <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.yaml>`_,
`WADL <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.wadl>`_,
and `XML <https://raw.github.com/cadithealth/pyramid_describe/master/doc/example.xml>`_.


Installation
============

Install with the usual python mechanism, e.g. here with ``pip``:

.. code-block:: bash

  $ pip install pyramid-describe


Usage
=====

There are three mechanisms to use pyramid-describe: via standard
pyramid inclusion which will add routes to the current application, by
explicitly embedding a ``pyramid_describe.DescribeController``
instance, or by directly calling the ``pyramid_describe.Describer``
object methods.


Pyramid Inclusion
=================

Pyramid-describe can be added via standard pyramid inclusion, either
in the INI file or directly in your `main` function. For example:

.. code-block:: python

  def main(global_config, **settings):
    # ...
    config.include('pyramid_describe')

When using pyramid inclusion, pyramid-describe expects to find
configuration options in the application settings. See the `Options`_
section for a list of all supported options, with a short example
here:

.. code-block:: ini

  [app:main]

  describe.attach                        = /doc
  describe.formats                       = html json pdf
  describe.format.default.title          = My Application
  describe.format.html.default.cssPath   = myapp:static/doc.css
  describe.entries.filters               = myapp.describe.entry_docify

Note that multiple describers, each with different configurations, can
be added via pyramid inclusion by using the `describe.prefixes`
option.


DescribeController
==================

Pyramid-describe can also be added to your application by embedding a
DescribeController object. The DescribeController constructor takes
the following parameters:

`view`:

  An instance of ``pyramid.interfaces.IView``, which is the view that
  should be inspected and reflected.

`root`:

  The root path to the specified URL, so that host-relative URLs can
  be generated to the views found.

`settings`:

  A dictionary of all the options to apply to this describer. Note that
  in this case, the options should not have any prefix.

Example:

.. code-block:: python

  from pyramid_describe import DescribeController

  def main(global_config, **settings):
    # ...
    config.include('pyramid_controllers')

    settings = {
        'formats'                       : ['html', 'json', 'pdf'],
        'format.default.title'          : 'My Application',
        'format.html.default.cssPath'   : 'myapp:static/doc.css',
        'entries.filters'               : 'myapp.describe.entry_docify',
      }

    config.add_controller('MyAppDescriber', '/doc', DescribeController(settings))


Describer
=========

Pyramid-describe can also be added to your application by directly
calling the Describer's functionality. This is an even lower-level
approach than, but still quite similar to, embedding the
`DescribeController`_; the constructor takes the same `settings`
parameter as the DescribeController, and then a call to the `describe`
method actually generates the output. The `describe` method takes as
parameters a `context` and a `format`, and returns a dictionary with
the following attributes:

.. TODO - document `context` and `format`...

`content_type`:

  The MIME content-type associated with the rendered output.

`charset`:

  The character set that the output is encoded in.

`content`:

  The actual rendering output.

Example:

.. code-block:: python

  from pyramid_describe import Describer

  def my_describer(request):

    settings = {
        'formats'                       : ['html', 'json', 'pdf'],
        'format.default.title'          : 'My Application',
        'format.html.default.cssPath'   : 'myapp:static/doc.css',
        'entries.filters'               : 'myapp.describe.entry_docify',
      }

    describer = Describer(settings=settings)
    context   = dict(request=request)
    result    = describer.describe(context=context, format='pdf')

    request.response.content_type = result['content_type']
    request.response.charset      = result['charset']
    request.response.body         = result['content']

    return request.response


Options
=======

The configuration of pyramid-describe is done by setting any of the
following options. Note that if specified in the application settings
(i.e. the INI file), then they must be prefixed with (by default)
``describe.``. Otherwise, when passing a dictionary of settings to the
constructors, the prefix is left off. The following options exist:

* ``describe.prefixes`` : list(str), default: 'describe'

  Defines the prefix or the list of prefixes that pyramid-describe
  settings will be searched for in the configuration. For each prefix,
  a separate DescribeController will be created and attached to the
  application router. The following example attaches two controllers
  at ``/desc-one`` and ``/desc-two``:

  .. code-block:: ini

    [app:main]
    describe.prefixes = describe-one describe-two
    describe-one.attach  = /desc-one
    # other `describe-one` options...
    describe-two.attach  = /desc-two
    # other `describe-two` options...

* ``describe.class`` : resolve-spec, default: pyramid_describe.DescribeController

  Sets the global default Controller class that will be instantiated
  for each of the stanzas defined in `describe.prefixes`. Note that
  this option can be overriden on a per-stanza basis.

* ``{PREFIX}.class`` : resolve-spec, default: `describe.class`

  Sets the Controller class that will be instantiated for this PREFIX
  stanza, overriding `describe.class`.

* ``{PREFIX}.attach`` : str, default: /describe

  Specifies the path to attach the controller to the current
  application's router. Note that this uses the `add_controller`
  directive, and ensures that pyramid-controllers has already been
  added via an explicit call to ``config.include()``. This path will
  serve the default format: to request alternate formats, use
  "PATH/FILENAME.EXT" (where FILENAME is controlled by the
  ``{PREFIX}.filename`` configuration and EXT specifies the format)
  or use the "format=EXT" query-string. Examples using the default
  settings:

  .. code-block:: text

    http://localhost:8080/describe/application.txt
    http://localhost:8080/describe/application.json
    http://localhost:8080/describe?format=json

* ``{PREFIX}.fullname`` : str, default: 'application'

  Sets the filename (excluding the extension) that the output will be
  served at using the DescribeController. The extension provided by
  the request will determine which format to serve, and must be listed
  in the `formats` option. If the format is not listed, a 404 is
  returned. Typically, this is set to the application's name and
  might also include the application version.

* ``{PREFIX}.basename`` : str, default: null

  Similar to the `fullname` option, this option sets a filename base
  component that will either redirect to the current `fullname` or
  actually serve the content based on the `base-redirect` option. This
  allows there to be a persistent known location that can be used if
  the `filename` option is dynamic or changes with revisions.

* ``{PREFIX}.index-redirect`` : { bool, int, str }, default: true

  Controls what happens when a request comes to the index location
  of the DescribeController, i.e. the value of the `attach` option.
  The following values are accepted:

  falsy

    Responds with the actual content using the default format.

  truthy

    Redirects with a 302 to the `basename` if set, otherwise to
    the `fullname`, using the default format's extension.

  int

    Same as if truthy, but uses the specified response code (e.g.
    301 instead of 302).

  str

    Responds with a redirect using the specified string as the
    ``Location`` header. By default, issues a 302 unless the string is
    prefixed with the code and a space, e.g. ``301
    /path/to/filename``. If the location is not absolute, it will be
    evaluated relative to the current URL.

* ``{PREFIX}.base-redirect`` : { bool, int, str }, default: true

  If `basename` is set, then this controls how the response is handled
  -- see the `index-redirect` option for accepted values, with the
  adjustment that the default redirect location is the `fullname`.

* ``{PREFIX}.inspect`` : str, default: /

  Specifies the top-level URL to start the application inspection at.

  IMPORTANT: this is not currently implemented the way that it should
  be... the current workaround simply adds the specified path (and its
  descendants) to the `include` list.

* ``{PREFIX}.include`` : list(regex-spec), default: null

  The `include` option lists encapsulated regular expressions that an
  endpoint must match at least one of in order to be included in the
  output. This option can be used with the `exclude` option, in which
  case endpoints are first matched for inclusion, then matched for
  exclusion (i.e. the order is "allow,deny" in apache terminology).

  Encapsulated regular expressions are expressed in the syntax
  "/EXPR/FLAGS", where the "/" can be replaced by any character
  otherwise not found in the rest of the expression. The flags can
  be any combination of the following characters:

  * ``i``: Case-insensitive matching.
  * ``l``: Use locale-dependent processing (for \w, \W, etc.).
{'fraction_non_alphanumeric': 0.09965142505638712, 'fraction_numerical': 0.0027885995489030143, 'mean_word_length': 3.607217079161156, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 20, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3156301', 'n_tokens_mistral': 6953, 'n_tokens_neox': 6472, 'n_words': 2688}