<commit_before>
== Introduction ==

This package contains all the source code to reproduce the nummerical
experiments described in [1]. It contains a parallelized implementation
of the MCA generative model training algorithm.


If you have problems running the code, please contact
 <NAME> <<EMAIL>>

== Overview ==

pulp/       - Python library/framework for MPI parallelized 
               EM-based algorithms. The MCA implementation
               is in pulp/em/camodels/mca_et.py

examples/   - Small example programs for the ulp library

gf-learning - Actual main program that performs EM based learing using 
               pulp.em
               
preproc/    - Scripts used to extract patches from the Van-Hateren
               database and to preprocess them usnig pseudo-whitening.

data/       - Traing data. This package does not include the training data
               but contains a script that can be used to doenload the datasets.

== Data Sets ==

The script "data/download.sh" will download 3 datasets:<commit_msg>Fix README typo and rename README<commit_after>
== Introduction ==

This package contains all the source code to reproduce the numerical
experiments described in [1]. It contains a parallelized implementation
of the MCA generative model training algorithm.


If you have problems running the code, please contact
 <NAME> <<EMAIL>>

== Overview ==

pulp/       - Python library/framework for MPI parallelized 
               EM-based algorithms. The MCA implementation
               is in pulp/em/camodels/mca_et.py

examples/   - Small example programs for the ulp library

gf-learning - Actual main program that performs EM based learing using 
               pulp.em
               
preproc/    - Scripts used to extract patches from the Van-Hateren
               database and to preprocess them usnig pseudo-whitening.

data/       - Traing data. This package does not include the training data
               but contains a script that can be used to doenload the datasets.

== Data Sets ==

The script "data/download.sh" will download 3 datasets:
{'fraction_non_alphanumeric': 0.05174892189746047, 'fraction_numerical': 0.0019166267369429804, 'mean_word_length': 2.6890459363957597, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7500514', 'n_tokens_mistral': 534, 'n_tokens_neox': 508, 'n_words': 248}