<commit_before>This README describes how to run the DumpMultiAlign pipeline 

The dumps are grouped by the toplevel regions of the reference species.
Alignments not containing the reference species are written to files called
*.other_*. Each file contains split_size alignments (default 200).
Alignments containing duplications in the reference species are dumped once
per duplicated segment.

All the scripts are located relative to $ENSEMBL_CVS_ROOT_DIR (location of the GIT checkout)

1) Edit $ENSEMBL_CVS_ROOT_DIR/ensembl-compara/modules/Bio/EnsEMBL/Compara/PipeConfig/DumpMultiAlign_conf.pm

The location of the compara and core databases containing the alignments
can be defined using the parameter 'reg_conf' and 'compara_db'. Defaults
have been set in the DumpMultiAlign_conf.pm. Most options likely to change
are done on the command line.

Options most likely to need chaging are:
compara_db  => Where the alignments are. Either a URL or a name found in the Registry
reg_conf    => Registry configuration file
format      => Many formats are accepted. "emf" (default), "maf", and "emf2maf" are the most common
export_dir  => Where to dump files. The pipeline will create 1 directory per alignment there

Other options are:
split_size  => Maximum number of blocks per file
masked_seq  => As in DumpMultiAlign.pl (0 for unmasked, 1 for soft-masked, 2 for hard-masked)
mode        => Either 'dir' (directory of compressed files -this is the default-),
               'tar' (compressed tar archive of a directory of uncompressed files),
               or 'file' (single compressed file)
method_link_types => In case you only want to dump a particular kind of alignments


2) Run init_pipeline.pl

  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db mysql://ensro@comparaX/msa_db_to_dump --export_dir where/the/dumps/will/be/
  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db mysql://ensro@ens-staging1/ensembl_compara_80 --reg_conf path/to/production_reg_conf.pl
  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db compara_prev --reg_conf path/to/production_reg_conf.pl --format maf --method_link_types EPO

<commit_msg>Added a comment about running the pipeline for an extra alignment<commit_after>This README describes how to run the DumpMultiAlign pipeline 

The dumps are grouped by the toplevel regions of the reference species.
Alignments not containing the reference species are written to files called
*.other_*. Each file contains split_size alignments (default 200).
Alignments containing duplications in the reference species are dumped once
per duplicated segment.

All the scripts are located relative to $ENSEMBL_CVS_ROOT_DIR (location of the GIT checkout)

1) Edit $ENSEMBL_CVS_ROOT_DIR/ensembl-compara/modules/Bio/EnsEMBL/Compara/PipeConfig/DumpMultiAlign_conf.pm

The location of the compara and core databases containing the alignments
can be defined using the parameter 'reg_conf' and 'compara_db'. Defaults
have been set in the DumpMultiAlign_conf.pm. Most options likely to change
are done on the command line.

Options most likely to need chaging are:
compara_db  => Where the alignments are. Either a URL or a name found in the Registry
reg_conf    => Registry configuration file
format      => Many formats are accepted. "emf" (default), "maf", and "emf2maf" are the most common
export_dir  => Where to dump files. The pipeline will create 1 directory per alignment there

Other options are:
split_size  => Maximum number of blocks per file
masked_seq  => As in DumpMultiAlign.pl (0 for unmasked, 1 for soft-masked, 2 for hard-masked)
mode        => Either 'dir' (directory of compressed files -this is the default-),
               'tar' (compressed tar archive of a directory of uncompressed files),
               or 'file' (single compressed file)
method_link_types => In case you only want to dump a particular kind of alignments


2) Run init_pipeline.pl

  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db mysql://ensro@comparaX/msa_db_to_dump --export_dir where/the/dumps/will/be/
  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db mysql://ensro@ens-staging1/ensembl_compara_80 --reg_conf path/to/production_reg_conf.pl
  init_pipeline.pl Bio::EnsEMBL::Compara::PipeConfig::DumpMultiAlign_conf --host comparaY --compara_db compara_prev --reg_conf path/to/production_reg_conf.pl --format maf --method_link_types EPO


3) Advanced usage

To add an extra job to dump the alignment with mlss_id XXX, using "favourite_species" to group the files
  seed_pipeline.pl -url mysql://hive/database -logic_name initJobs -input_id '{"mlss_id" => XXX, "species" => "favourite_species"}'


{'fraction_non_alphanumeric': 0.07166324435318275, 'fraction_numerical': 0.005544147843942505, 'mean_word_length': 4.717136150234742, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4322677', 'n_tokens_mistral': 1484, 'n_tokens_neox': 1424, 'n_words': 549}