<commit_before>draft-transforms
================

tools for modifying drafts of Connexions content in a batch manner, from the commandline<commit_msg>additional doco<commit_after>draft-transforms
================

tools for modifying drafts of Connexions content in a batch manner, from the
commandline.

Needs an installable version of requests-toolbelt, which seems to mean 
version > 0.1.2

This repo installs a single master command `draft-transform`, which performs
operations on all the modules in a particular legacy Connexions workgroup.
This command will do a checkout on any modules in the 'published' state, as well.

Common options to all commands are:
  -h help
  -H --host hostname requires https access for non-localhost
        defaults to qa.cnx.org
  -a --auth username:password (required)
  -w --workgroup ID  Workgroup to operate against
        defaults to private workspace
  -u --upload Send transformed content back to workgroup
  -p --publish message Publish the modules after upload
  -P publish_only message Publish all modules in workgroup, no download or upload
  -s --save-dir directory save intermediate results to this directory, named <id>.xml
        will create the directort if necessary
  

Available subcommands are:
    download [directory]
        download the module body, save it in directly (same as -s above, but no
        transform)
    xslt file.xsl apply xslt from file.xsl to each module contents.

Examples:

draft-transform -H qa.cnx.org -a 'me:mypass' download my_modules
    dowloads and saves all modules from private workspace to the directly my_modules

draft-transform -H cnx.org -a 'me:mypass' -w wg123 -s fixed_files xslt myfix.xsl
    downloads, transforms and saves transformed content

draft-transform -H cnx.org -a 'me:mypass' -w wg123 -s fixed_files -u xslt myfix.xsl
    downloads, transforms, saves, and uploads transformed content

draft-transform -H cnx.org -a 'me:mypass' -w wg123 -P 'Fixes that markup bug'
    Publishes all the modules in the workgroup

draft-transform -H cnx.org -a 'me:mypass' -w wg123 -s fixed_files -u -p 'Fixed that markup bug' xslt myfix.xsl
    downloads, transforms, saves, and uploads, and publishes transformed content

{'fraction_non_alphanumeric': 0.07397383852052324, 'fraction_numerical': 0.006765899864682003, 'mean_word_length': 3.759656652360515, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1104144', 'n_tokens_mistral': 647, 'n_tokens_neox': 615, 'n_words': 288}