<commit_before>"""
Dataset for images and related functionality.

"""
__authors__ = "<NAME>"
__copyright__ = "Copyright 2015, <NAME>"
__credits__ = ["<NAME>"]
__license__ = "3-clause BSD"<commit_msg>Some more documentation for Images<commit_after>"""
Dataset for images and related functionality.

This module does not have dependencies inside pyl2extra package, so you
can just copy-paste it inside your source tree.

To use this dataset prepare a .csv file with targets (integers or real numbers)
on first column and file paths on the second column:

..code:

    0,file1.png
    1,file2.png

Image file paths are relative to current directory (``os.getcwd()``). The
images need not be square and can be in any format recognized by the
``Image`` module. Internally, the images are converted to RGB and are made
square for you.

Use it in a .yaml file like so:

..code:

    dataset: &trndataset !obj:pyl2extra.datasets.images.Images {
        source: 'train.csv',
        image_size: 128
    }

The ``image_size`` can be skipped, in which case the size of the images is
derived from first image that is provided.

By default the class assumes a classification problem (targets are integers).
If you need to uset it in a regression problem create it like so:

..code:

    dataset: &trndataset !obj:pyl2extra.datasets.images.Images {
        source: 'train.csv',
        image_size: 128,
        regression: True
    }

As the dataset simply wraps the ``DenseDesignMatrix``, parameters like
``rng`` (random number generator), ``preprocessor`` and ``fit_preprocessor``
can be used and will be passed to ``DenseDesignMatrix`` superclass.
"""
__authors__ = "<NAME>"
__copyright__ = "Copyright 2015, <NAME>"
__credits__ = ["<NAME>"]
__license__ = "3-clause BSD"
{'fraction_non_alphanumeric': 0.09670079635949944, 'fraction_numerical': 0.013083048919226393, 'mean_word_length': 4.098550724637681, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5497855', 'n_tokens_mistral': 531, 'n_tokens_neox': 504, 'n_words': 222}