<commit_before>Documenting Chapel
==================

The Chapel standard libraries are documenting using ``chpldoc``, which
generates HTML-based documentation for Chapel code from comments embedded
within the code.

How it works
------------

The Chapel standard library documentation is generated by running chpldoc over
all Chapel source files in ``$CHPL_HOME/modules/standard/``.

The markup used in the comments is reSTructuredText_. reStructuredText is
developed by the `docutils`_ projects and is amended by custom directives to
support documenting Chapel code. Sphinx_ is used by ``chpldoc`` to render
reStructuredText as HTML.

.. _reStructuredText: http://docutils.sf.net/rst.html
.. _docutils: http://docutils.sourceforge.net/
.. _Sphinx: http://sphinx-doc.org/

reStructured Text Primer
------------------------

This section is a brief introduction to reStructuredText (aka reST). It is
intended to provide Chapel developers with enough information to write
documentation in comments in the source code.

The Python and Sphinx projects have thorough primers of reST. Please see those
for more detail.

* `Sphinx reST Primer`_
* `Python reST Primer`_

The authoritive `reStructuredText User Guide`_ is also helpful.

.. _Sphinx reST Primer: http://sphinx-doc.org/rest.html
.. _Python reST Primer: https://docs.python.org/devguide/documenting.html#restructuredtext-primer
.. _reStructuredText User Guide: http://docutils.sourceforge.net/rst.html

Paragraphs
~~~~~~~~~~

Simple chunks of text. Paragraphs can have line breaks to improve source
readability. Separate paragraphs with an empty line. Indentation is significant
in reST. All lines of paragraph must be left-aligned.

Inline markup
~~~~~~~~~~~~~

* one asterisk: ``*text*`` for emphasis (italics),
* two asterisks: ``**text**`` for strong emphasis (boldface), and
* backquotes: ````text```` for code samples.

If asterisks or backquotes appear in running text and could be confused with
inline markup delimiters, they have to be escaped with a backslash.

Lists and Quotes
~~~~~~~~~~~~~~~~

To create a list, put an asterisk at the start of a paragraph and indent
accordingly. Create numbered lists by using the literal numbers, e.g. ``1.``,
``2.``. Automatically numbered lists begin with ``#.``::

   * This is a bulleted list.
   * It has two items, the second
     item uses two lines.

   1. This is a numbered list.
   2. It has two items too.

   #. This is a numbered list.
   #. It has two items too.

Nested lists are possible, but be aware that they must be separated from the
parent list items by blank lines::

   * a
   * b

     * nested
     * blah

   * my final item, in the parent list

Source Code
~~~~~~~~~~~

Literal code blocks are introduced by ending a paragraph with the special marker
``::``.  The literal block must be indented::

   This is a normal text paragraph. The next paragraph is a code sample::

      It is not processed in any way, except
      that the indentation is removed.

      It can span multiple lines.

   This is a normal text paragraph again.

The handling of the ``::`` marker is smart:

* If it occurs as a paragraph of its own, that paragraph is completely left
  out of the document.
* If it is preceded by whitespace, the marker is removed.
* If it is preceded by non-whitespace, the marker is replaced by a single
  colon.

That way, the second sentence in the above example's first paragraph would be
rendered as "The next paragraph is a code sample:".

Hyperlinks
~~~~~~~~~~

External links
++++++++++++++

Use ```Link text <http://target>`_`` for inline web links.  If the link text
should be the web address, you don't need special markup at all, the parser
finds links and mail addresses in ordinary text.

Internal links
++++++++++++++

Internal linking is done via a special reST role, see the section on specific
markup `cross-references`_.

Sections
~~~~~~~~

Section headers are created by underlining (and optionally overlining) the
section title with a punctuation character, at least as long as the text::

   This is a heading
   =================

There are no heading levels assigned to certain characters. The structure is
determined from the succession of headings.

Comments
~~~~~~~~

Every explicit markup block, which is not a valid markup construct is regarded
as a comment. For example::

   This is a normal paragraph.
   It is interesting.

   .. TODO: Make it more interesting.

   Another paragraph goes here.

   .. add another paragraph below

You can indent text after a comment start to form multiline comments::

   ..
      This whole indented block
      is a comment.

      Still in the comment.

Showing code examples
~~~~~~~~~~~~~~~~~~~~~

The ``code-block`` directive can be used to specify the highlight language of a
single code block. For example::

   .. code-block:: chapel

      use Foo;

      proc bar() {
        writeln("Fooy!");
      }

If highlighting with the specified language fails, e.g. if the syntax is not
parsable, the block is not highlighted in anyway.

.. _cross-references:

Inline markup
~~~~~~~~~~~~~

As said before, Sphinx uses interpreted text roles to insert semantic markup in
documents.

Names of builtins, like `true`, `false`, types like `int(64)`, and local
variables, such as function/method arguments, are an exception, they should be
marked simply with ```myVar```.

For all other roles, you have to write ``:rolename:`content```.

There are some additional facilities that make cross-referencing roles more
versatile:

* You may supply an explicit title and reference target, like in reST direct
  hyperlinks: ``:role:`title <target>``` will refer to *target*, but the link
  text will be *title*.

* If you prefix the content with ``!``, no reference/hyperlink will be created.

* For the Chapel roles, if you prefix the content with ``~``, the link text
  will only be the last component of the target.  For example,
  ``:meth:`~Random.RandomStream.fillRandom``` will refer to
  ``Random.RandomStream.fillRandom`` but only display ``fillRandom`` as the
  link text.

  In HTML output, the link's ``title`` attribute (that is e.g. shown as a
  tool-tip on mouse-hover) will always be the full target name.

The following roles refer to objects in modules and are possibly hyperlinked if
a matching identifier is found:

``:chpl:mod:``

    Reference a module; a dotted name may be used. See `cross-references`_ for
    details on dotted and non-dotted names.

``:chpl:proc:``
``:chpl:iter:``

    Reference a Chapel function or iterator. The role text needs not include
    trailing parentheses to enhance readability.

    These can also be used to reference a method or iterator on an object
    (class or record instance). The role text can include the type name and the
    method, in those cases. If it occurs within the description of a type, the
    type name can be omitted.

    Dotted names may be used for any form.

``:chpl:data:``
``:chpl:const:``
``:chpl:var:``
``:chpl:param:``
``:chpl:type:``

    Reference a module-level variable, constant, compiler param, or type.

``:chpl:class:``
``:chpl:record:``

    Reference a class or record; a dotted name may be used.

``:chpl:attr:``

    Reference a data attribute (const, var, param, generic type) of an object.

For example::

    Uses :chpl:proc:`Random.RandomStream.fillRandom` and real->int casts to
    generate a vector of random integers. See :chpl:attr:`RandomStream.seed`
    and description of :chpl:mod:`Random` for details on PRNG.

    Relies on :chpl:iter:`MyModule.Set.these` to iterate over all values in the
    given :chpl:record:`MyModule.Set`.

Documenting argument, return, and yield values and types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Inside Chapel description directives, reST field lists with these fields are
recognized and formatted nicely:

* ``arg``, ``argument``: Description of a parameter.
* ``type``: Type of a parameter. Creates a link if possible.
* ``returns``, ``return``: Description of the return value.
* ``rtype``: Return type. Creates a link if possible.
* ``yields``, ``yield``: Description of the yield value, often used for
  iterators.
* ``ytype``: Yield type. Creates a link if possible.

For example, when documenting a chapel method::

   /*
    * Calculates number of pipes and returns fooy.
    * 
    * :arg bars: number of bars
    * :type bars: int
    * 
    * :arg hours: hours available
    * :type hours: real
    * 
    * :returns: amount of fooy available
    * :rtype: Foo
    */
   proc foo(x, y): Foo {}

.. note:: These fields must be left-aligned with the outer most paragraphs.

Paragraph level markup
~~~~~~~~~~~~~~~~~~~~~~

These directives create short paragraphs and can be used inside information
units as well as normal text:<commit_msg>Add more info about fields.<commit_after>Documenting Chapel
==================

The Chapel standard libraries are documenting using ``chpldoc``, which
generates HTML-based documentation for Chapel code from comments embedded
within the code.

How it works
------------

The Chapel standard library documentation is generated by running chpldoc over
all Chapel source files in ``$CHPL_HOME/modules/standard/``.

The markup used in the comments is reSTructuredText_. reStructuredText is
developed by the `docutils`_ projects and is amended by custom directives to
support documenting Chapel code. Sphinx_ is used by ``chpldoc`` to render
reStructuredText as HTML.

.. _reStructuredText: http://docutils.sf.net/rst.html
.. _docutils: http://docutils.sourceforge.net/
.. _Sphinx: http://sphinx-doc.org/

reStructured Text Primer
------------------------

This section is a brief introduction to reStructuredText (aka reST). It is
intended to provide Chapel developers with enough information to write
documentation in comments in the source code.

The Python and Sphinx projects have thorough primers of reST. Please see those
for more detail.

* `Sphinx reST Primer`_
* `Python reST Primer`_

The authoritive `reStructuredText User Guide`_ is also helpful.

.. _Sphinx reST Primer: http://sphinx-doc.org/rest.html
.. _Python reST Primer: https://docs.python.org/devguide/documenting.html#restructuredtext-primer
.. _reStructuredText User Guide: http://docutils.sourceforge.net/rst.html

Paragraphs
~~~~~~~~~~

Simple chunks of text. Paragraphs can have line breaks to improve source
readability. Separate paragraphs with an empty line. Indentation is significant
in reST. All lines of paragraph must be left-aligned.

Inline markup
~~~~~~~~~~~~~

* one asterisk: ``*text*`` for emphasis (italics),
* two asterisks: ``**text**`` for strong emphasis (boldface), and
* backquotes: ````text```` for code samples.

If asterisks or backquotes appear in running text and could be confused with
inline markup delimiters, they have to be escaped with a backslash.

Lists and Quotes
~~~~~~~~~~~~~~~~

To create a list, put an asterisk at the start of a paragraph and indent
accordingly. Create numbered lists by using the literal numbers, e.g. ``1.``,
``2.``. Automatically numbered lists begin with ``#.``::

   * This is a bulleted list.
   * It has two items, the second
     item uses two lines.

   1. This is a numbered list.
   2. It has two items too.

   #. This is a numbered list.
   #. It has two items too.

Nested lists are possible, but be aware that they must be separated from the
parent list items by blank lines::

   * a
   * b

     * nested
     * blah

   * my final item, in the parent list

Source Code
~~~~~~~~~~~

Literal code blocks are introduced by ending a paragraph with the special marker
``::``.  The literal block must be indented::

   This is a normal text paragraph. The next paragraph is a code sample::

      It is not processed in any way, except
      that the indentation is removed.

      It can span multiple lines.

   This is a normal text paragraph again.

The handling of the ``::`` marker is smart:

* If it occurs as a paragraph of its own, that paragraph is completely left
  out of the document.
* If it is preceded by whitespace, the marker is removed.
* If it is preceded by non-whitespace, the marker is replaced by a single
  colon.

That way, the second sentence in the above example's first paragraph would be
rendered as "The next paragraph is a code sample:".

Hyperlinks
~~~~~~~~~~

External links
++++++++++++++

Use ```Link text <http://target>`_`` for inline web links.  If the link text
should be the web address, you don't need special markup at all, the parser
finds links and mail addresses in ordinary text.

Internal links
++++++++++++++

Internal linking is done via a special reST role, see the section on specific
markup `cross-references`_.

Sections
~~~~~~~~

Section headers are created by underlining (and optionally overlining) the
section title with a punctuation character, at least as long as the text::

   This is a heading
   =================

There are no heading levels assigned to certain characters. The structure is
determined from the succession of headings.

Comments
~~~~~~~~

Every explicit markup block, which is not a valid markup construct is regarded
as a comment. For example::

   This is a normal paragraph.
   It is interesting.

   .. TODO: Make it more interesting.

   Another paragraph goes here.

   .. add another paragraph below

You can indent text after a comment start to form multiline comments::

   ..
      This whole indented block
      is a comment.

      Still in the comment.

Showing code examples
~~~~~~~~~~~~~~~~~~~~~

The ``code-block`` directive can be used to specify the highlight language of a
single code block. For example::

   .. code-block:: chapel

      use Foo;

      proc bar() {
        writeln("Fooy!");
      }

If highlighting with the specified language fails, e.g. if the syntax is not
parsable, the block is not highlighted in anyway.

.. _cross-references:

Inline markup
~~~~~~~~~~~~~

As said before, Sphinx uses interpreted text roles to insert semantic markup in
documents.

Names of builtins, like `true`, `false`, types like `int(64)`, and local
variables, such as function/method arguments, are an exception, they should be
marked simply with ```myVar```.

For all other roles, you have to write ``:rolename:`content```.

There are some additional facilities that make cross-referencing roles more
versatile:

* You may supply an explicit title and reference target, like in reST direct
  hyperlinks: ``:role:`title <target>``` will refer to *target*, but the link
  text will be *title*.

* If you prefix the content with ``!``, no reference/hyperlink will be created.

* For the Chapel roles, if you prefix the content with ``~``, the link text
  will only be the last component of the target.  For example,
  ``:meth:`~Random.RandomStream.fillRandom``` will refer to
  ``Random.RandomStream.fillRandom`` but only display ``fillRandom`` as the
  link text.

  In HTML output, the link's ``title`` attribute (that is e.g. shown as a
  tool-tip on mouse-hover) will always be the full target name.

The following roles refer to objects in modules and are possibly hyperlinked if
a matching identifier is found:

``:chpl:mod:``

    Reference a module; a dotted name may be used. See `cross-references`_ for
    details on dotted and non-dotted names.

``:chpl:proc:``
``:chpl:iter:``

    Reference a Chapel function or iterator. The role text needs not include
    trailing parentheses to enhance readability.

    These can also be used to reference a method or iterator on an object
    (class or record instance). The role text can include the type name and the
    method, in those cases. If it occurs within the description of a type, the
    type name can be omitted.

    Dotted names may be used for any form.

``:chpl:data:``
``:chpl:const:``
``:chpl:var:``
``:chpl:param:``
``:chpl:type:``

    Reference a module-level variable, constant, compiler param, or type.

``:chpl:class:``
``:chpl:record:``

    Reference a class or record; a dotted name may be used.

``:chpl:attr:``

    Reference a data attribute (const, var, param, generic type) of an object.

For example::

    Uses :chpl:proc:`Random.RandomStream.fillRandom` and real->int casts to
    generate a vector of random integers. See :chpl:attr:`RandomStream.seed`
    and description of :chpl:mod:`Random` for details on PRNG.

    Relies on :chpl:iter:`MyModule.Set.these` to iterate over all values in the
    given :chpl:record:`MyModule.Set`.

Documenting argument, return, and yield values and types
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Inside Chapel description directives, reST field lists with these fields are
recognized and formatted nicely:

* ``arg``, ``argument``: Description of a parameter.
* ``type``: Type of a parameter. Creates a link if possible.
* ``returns``, ``return``: Description of the return value.
* ``rtype``: Return type. Creates a link if possible.
* ``yields``, ``yield``: Description of the yield value, often used for
  iterators.
* ``ytype``: Yield type. Creates a link if possible.

``type``, ``rtype``, and ``ytype`` should be concise and literal type
definitions, like ``int``, ``int(64)``, ``bool``, ``[] int``, ``RandomStream``,
etc. More verbose descriptions, qualifications, and limitations of those types
should go in the corresponding ``arg``, ``returns``, or ``yields`` field.

For example, when documenting a chapel method::

   /*
    * Calculates number of pipes and returns fooy.
    * 
    * :arg bars: Number of bars. Must be more than 1 and less than 1000.
    * :type bars: int
    * 
    * :arg hours: Hours available. Default is 1.0.
    * :type hours: real
    * 
    * :returns: Amount of fooy available.
    * :rtype: Foo
    */
   proc foo(x, y=1.0): Foo
   {
     ...
   }

.. note:: These fields must be left-aligned with the outer most paragraphs.

Paragraph level markup
~~~~~~~~~~~~~~~~~~~~~~

These directives create short paragraphs and can be used inside information
units as well as normal text:
{'fraction_non_alphanumeric': 0.11136401487814357, 'fraction_numerical': 0.0012768556042857936, 'mean_word_length': 3.770656779661017, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5990401', 'n_tokens_mistral': 5136, 'n_tokens_neox': 4827, 'n_words': 2400}