<commit_before>cgpipe - HPC/HTC analysis pipelines 
-----------------------------------
Usage: cgpipe {opts} {args} {target1 target2...}

Options:
    -f filename   Pipeline script filename (local or remote, required)
    -h            Show help for pipeline
    -l logfile    Log debug messages here
    -o outfile    Write all printed lines to this file
    -j joblog     Output a summary of submitted jobs 
                  (and their outputs, jobids, etc)

    -dr           Dry run (Don't actually build targets)
    
    -v            Verbose logging
    -s            Silence writing to console
    -nolog        Suppress logging to stderr
    
    -license      Show the license information
    

    Additional pipeline-specific arguments can be set by using the format:
        --foo bar --arg one --arg two
    
    This will set the variable "foo" to be "bar", and "arg" to be a list:
    ["one", "two"]. These variables can then be used by the pipeline script.

    The pipeline script can either be a local file or a remote pipeline. By
    default the compgen.io pipeline repository is available at:
    compgen_io:path/to/script
    
    For more details about remote repositories, check the README.md file.
    

Note: If no target is specified, the first target in the file will be used
<commit_msg>Updated default help text<commit_after>cgpipe - HPC/HTC analysis pipelines 
-----------------------------------
Usage: cgpipe {opts} {args} {target1 target2...}

Options:
    -f filename   Pipeline script filename (local or remote, required)
    -h            Show help for pipeline
    -l logfile    Log debug messages here
    -o outfile    Write all printed lines to this file
    -j joblog     Output a summary of submitted jobs 
                  (and their outputs, jobids, etc)

    -dr           Dry run (Don't actually build targets)
    
    -v            Verbose logging
    -s            Silence writing to console
    -nolog        Suppress logging to stderr
    
    -license      Show the license information
    

    Additional pipeline-specific arguments can be set by using the format:
        --foo bar --arg one --arg two
    
    This will set the variable "foo" to be "bar", and "arg" to be a list:
    ["one", "two"]. These variables can then be used by the pipeline script.

    The pipeline script can either be a local file or a remote pipeline. By
    default the compgen.io pipeline repository is available at:
    compgen_io:path/to/script
    
    For more details about remote repositories, check the README.md file.
    

Note: If no target is specified, the first target in the file will be used


Additional commands:
	show-pending       Show pending/running jobs from a job-log
	cancel-pending     Cancel pending jobs from a job-log
	vaccuum            Clean up a job-log

{'fraction_non_alphanumeric': 0.07820240622788394, 'fraction_numerical': 0.0014154281670205238, 'mean_word_length': 2.056216216216216, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4107127', 'n_tokens_mistral': 742, 'n_tokens_neox': 719, 'n_words': 362}