<commit_before>gollum -- A wiki built on top of Git
====================================

## DESCRIPTION

Gollum is a simple wiki built on top of Git that powers GitHub Wikis. Gollum
is designed so that you can create a compliant Git repository (known as a
Gollum repository) either by hand, by calling API methods, or via the web
interface.

We are making the source available so that you may test your wikis locally if
you choose to edit them by hand.


## REPO STRUCTURE

A Gollum repository's contents are designed to be human editable. Page content
is written in `page files` and may be organized into directories any way you
choose. Special footers can be created in `footer files`. Other content
(images, PDFs, etc) may also be present and organized in the same way.


## PAGE FILES

Page files may be written in any format supported by
[GitHub-Markup](http://github.com/defunkt/github-markup). The current list of
formats and allowed extensions is:

  * Markdown: .markdown, .mdown, .mkdn, .mkd, .md
  * Textile: .textile
  * RDoc: .rdoc
  * Org Mode: .org
  * ReStructured Text: .rest.txt, .rst.txt, .rest, .rst
  * ASCIIDoc: .asciidoc
  * POD: .pod
  * Roff: .1, .2, .3, ...

Gollum detects the page file format via the extension, so files must have one
of the supported extensions in order to be converted.

Page file names may contain any printable UTF-8 character except space
(U+0020) and forward slash (U+002F). If you commit a page file with any of
these characters in the name it will not be accessible via the web interface.

Even though page files may be placed in any directory, there is still only a
single namespace for page names, so all page files should have globally unique
names regardless of where they are located in the repository.

The special page file `Home.ext` (where the extension is one of the supported
formats) will be used as the entrance page to your wiki. If it is missing, an
automatically generated table of contents will be shown instead.


## FOOTER FILES

Footer files allow you to add a simple footer to your wiki. Footer files must
be named `_Footer.ext` where the extension is one of the supported formats.
Footers affect all pages in their directory and any subdirectories that do not
have a footer file of their own.


## HTML SANITIZATION

For security and compatibility reasons Gollum wikis may not contain custom CSS
or JavaScript. These tags will be stripped from the converted HTML.


## PAGE LINKS

To link to another Gollum wiki page, use the Gollum Page Link Tag.

    [[Frodo]]

The above tag will create a link to the corresponding page file named
`Frodo.ext` where `ext` may be any of the allowed extension types. The
conversion is as follows:

  1. Strip any non-printables ()
  2. Replace any spaces () with dashes
  3. Replace any slashes () with dashes

If you'd like the link text to be something that doesn't map directly to the
page name, you can specify the actual page name after a pipe:

    [[Frodo|Frodo Baggins]]

The above tag will link to `Frodo-Baggins.ext` using "Frodo" as the link text.

The page file may exist anywhere in the directory structure of the repository.
Gollum does a breadth first search and uses the first match that it finds.

Here are a few more examples:

    [[<NAME>]]    -> J.-R.-R.-Tolkien.ext
    [[Movies / The Hobbit]] -> Movies---The-Hobbit.ext
    [[モルドール]]               -> モルドール.ext


## ABSOLUTE VS. RELATIVE PATHS

For Gollum tags that operate on static files (images, PDFs, etc), the paths
may be referenced as either relative or absolute. Relative paths point to a
static file relative to the page file within the directory structure of the
Gollum repo (even though after conversion, all page files appear to be top
level). These paths are NOT prefixed with a slash. For example:

    gollum.pdf
    docs/diagram.png

Absolute paths point to a static file relative to the Gollum repo's
root, regardless of where the page file is stored within the directory
structure. These paths ARE prefixed with a slash. For example:

    /pdfs/gollum.pdf
    /docs/diagram.png

All of the examples in this README use relative paths, but you may use
whatever works best in your situation.


## FILE LINKS

To link to static files that are contained in the Gollum repository you should
use the Gollum File Link Tag.

    [[Gollum|gollum.pdf]]
<commit_msg>lining up utf8 is hard<commit_after>gollum -- A wiki built on top of Git
====================================

## DESCRIPTION

Gollum is a simple wiki built on top of Git that powers GitHub Wikis. Gollum
is designed so that you can create a compliant Git repository (known as a
Gollum repository) either by hand, by calling API methods, or via the web
interface.

We are making the source available so that you may test your wikis locally if
you choose to edit them by hand.


## REPO STRUCTURE

A Gollum repository's contents are designed to be human editable. Page content
is written in `page files` and may be organized into directories any way you
choose. Special footers can be created in `footer files`. Other content
(images, PDFs, etc) may also be present and organized in the same way.


## PAGE FILES

Page files may be written in any format supported by
[GitHub-Markup](http://github.com/defunkt/github-markup). The current list of
formats and allowed extensions is:

  * Markdown: .markdown, .mdown, .mkdn, .mkd, .md
  * Textile: .textile
  * RDoc: .rdoc
  * Org Mode: .org
  * ReStructured Text: .rest.txt, .rst.txt, .rest, .rst
  * ASCIIDoc: .asciidoc
  * POD: .pod
  * Roff: .1, .2, .3, ...

Gollum detects the page file format via the extension, so files must have one
of the supported extensions in order to be converted.

Page file names may contain any printable UTF-8 character except space
(U+0020) and forward slash (U+002F). If you commit a page file with any of
these characters in the name it will not be accessible via the web interface.

Even though page files may be placed in any directory, there is still only a
single namespace for page names, so all page files should have globally unique
names regardless of where they are located in the repository.

The special page file `Home.ext` (where the extension is one of the supported
formats) will be used as the entrance page to your wiki. If it is missing, an
automatically generated table of contents will be shown instead.


## FOOTER FILES

Footer files allow you to add a simple footer to your wiki. Footer files must
be named `_Footer.ext` where the extension is one of the supported formats.
Footers affect all pages in their directory and any subdirectories that do not
have a footer file of their own.


## HTML SANITIZATION

For security and compatibility reasons Gollum wikis may not contain custom CSS
or JavaScript. These tags will be stripped from the converted HTML.


## PAGE LINKS

To link to another Gollum wiki page, use the Gollum Page Link Tag.

    [[Frodo]]

The above tag will create a link to the corresponding page file named
`Frodo.ext` where `ext` may be any of the allowed extension types. The
conversion is as follows:

  1. Strip any non-printables ()
  2. Replace any spaces () with dashes
  3. Replace any slashes () with dashes

If you'd like the link text to be something that doesn't map directly to the
page name, you can specify the actual page name after a pipe:

    [[Frodo|Frodo Baggins]]

The above tag will link to `Frodo-Baggins.ext` using "Frodo" as the link text.

The page file may exist anywhere in the directory structure of the repository.
Gollum does a breadth first search and uses the first match that it finds.

Here are a few more examples:

    [[<NAME>]] -> J.-R.-R.-Tolkien.ext
    [[Movies / The Hobbit]] -> Movies---The-Hobbit.ext
    [[モルドール]] -> モルドール.ext


## ABSOLUTE VS. RELATIVE PATHS

For Gollum tags that operate on static files (images, PDFs, etc), the paths
may be referenced as either relative or absolute. Relative paths point to a
static file relative to the page file within the directory structure of the
Gollum repo (even though after conversion, all page files appear to be top
level). These paths are NOT prefixed with a slash. For example:

    gollum.pdf
    docs/diagram.png

Absolute paths point to a static file relative to the Gollum repo's
root, regardless of where the page file is stored within the directory
structure. These paths ARE prefixed with a slash. For example:

    /pdfs/gollum.pdf
    /docs/diagram.png

All of the examples in this README use relative paths, but you may use
whatever works best in your situation.


## FILE LINKS

To link to static files that are contained in the Gollum repository you should
use the Gollum File Link Tag.

    [[Gollum|gollum.pdf]]

{'fraction_non_alphanumeric': 0.06458285451620317, 'fraction_numerical': 0.003332567225925075, 'mean_word_length': 3.724755700325733, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '154300', 'n_tokens_mistral': 2541, 'n_tokens_neox': 2405, 'n_words': 1362}