<commit_before>pylearn2 tutorial example: "deep_trainer" by <NAME>

This directory contains a simple example to give you a feeling of how you can
use a few kinds of unsupervised learning to do layerwise pretraining of a
deep model formed by stacking shallow models.

This is not the best-supported tutorial in pylearn2, in part because most of
the active pylearn2 developers are no longer researching stacked layerwise
pretraining. However, if you're interested in doing layerwise pretraining,
this is the best pylearn2 tutorial available.

The recommended usage of pylearn2 is to write YAML configuration files
describing experiments and pass them to the train.py script. This method is
described in most of the other tutorials, such as grbm_smd and the .ipynb
tutorials. This tutorial teaches an alternate method, which is just building
the experiment in your own .py file and using exclusively python interfaces
everywhere. This method is not quite as well supported, and this tutorial
shows you how to make sure the proper steps are taken to make sure datasets
get their correct YAML description when using this method.

usage:

make sure pylearn2 can be found in your PYTHONPATH

to train models on toy dataset: python run_deep_trainer.py -d toy

to train models on cifar10 dataset: python run_deep_trainer.py -d cifar10

to train models on mnist dataset: python run_deep_trainer.py -d mnist

To visualize the first-layer weights trained on, for example, mnist:

show_weights.py grbm.pkl

(Visualization of deeper layer weights is not conceptually straightforward,<commit_msg>added ref to Mehdi's tutorial<commit_after>pylearn2 tutorial example: "deep_trainer" by <NAME>

This directory contains a simple example to give you a feeling of how you can
use a few kinds of unsupervised learning to do layerwise pretraining of a
deep model formed by stacking shallow models.

This is not the best-supported tutorial in pylearn2, in part because most of
the active pylearn2 developers are no longer researching stacked layerwise
pretraining.

The recommended usage of pylearn2 is to write YAML configuration files
describing experiments and pass them to the train.py script. This method is
described in most of the other tutorials, such as grbm_smd and the .ipynb
tutorials. This tutorial teaches an alternate method, which is just building
the experiment in your own .py file and using exclusively python interfaces
everywhere. This method is not quite as well supported, and this tutorial
shows you how to make sure the proper steps are taken to make sure datasets
get their correct YAML description when using this method.

You can also read stacked_autoencoders.ipynb, a tutorial on how to do
layerwise pretraining using YAML files. That tutorial only covers autoencoder
pretraining but demonstrates how to use the recommended YAML interface.

usage:

make sure pylearn2 can be found in your PYTHONPATH

to train models on toy dataset: python run_deep_trainer.py -d toy

to train models on cifar10 dataset: python run_deep_trainer.py -d cifar10

to train models on mnist dataset: python run_deep_trainer.py -d mnist

To visualize the first-layer weights trained on, for example, mnist:

show_weights.py grbm.pkl

(Visualization of deeper layer weights is not conceptually straightforward,
{'fraction_non_alphanumeric': 0.027043451838347006, 'fraction_numerical': 0.005773321178972957, 'mean_word_length': 4.920863309352518, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3865096', 'n_tokens_mistral': 880, 'n_tokens_neox': 844, 'n_words': 491}