<commit_before># c4-pkg scripts

Common scripts to ease working on C4.

## Setup

The `setup.sh` script will modify a users `.bash_profile` and `.bashrc` 
so that `.bashrc` is always sourced and so that `.bashrc` sources
the `vars.sh` script to update the module path and install a default
set of modules.  There is also a `vars.csh` script for csh afficionados;
but since that is not the default shell, I assume these people know enough
about what they are doing to update their own startup files.

## IPython profile setup

The `setup_ipython_htc.sh` script sets up an IPython profile (`htc`)
for running `ipcluster` jobs on C4.

TODO: I should also write a `setup_ipython_mpi.sh` script to run
jobs that call `mpi4py`.

## MPI and UPC submissions

The `mpisub` and `upcsub` wrappers are installed atop the `openmpi-1.6.5`
and `upc` local subtrees.  These wrappers generate and submit an HTCondor
submission script.  Both assume the user is going to run out of a shared
directory (whatever directory the script was called form) rather than
the scratch directory that HTCondor dumps them to.
<commit_msg>Updated script/README.md to refer to IPython MPI setup.<commit_after># c4-pkg scripts

Common scripts to ease working on C4.

## Setup

The `setup.sh` script will modify a users `.bash_profile` and `.bashrc` 
so that `.bashrc` is always sourced and so that `.bashrc` sources
the `vars.sh` script to update the module path and install a default
set of modules.  There is also a `vars.csh` script for csh afficionados;
but since that is not the default shell, I assume these people know enough
about what they are doing to update their own startup files.

## IPython profile setup

The `setup_ipython_htc.sh` script sets up an IPython profile (`htc`)
for running `ipcluster` jobs on C4.  The `setup_ipython_mpi.sh` script
sets up a similar IPython profile (`mpi`) for running `ipcluster` jobs
on C4 with `mpi4py` enabled.

## MPI and UPC submissions

The `mpisub` and `upcsub` wrappers are installed atop the `openmpi-1.6.5`
and `upc` local subtrees.  These wrappers generate and submit an HTCondor
submission script.  Both assume the user is going to run out of a shared
directory (whatever directory the script was called form) rather than
the scratch directory that HTCondor dumps them to.

{'fraction_non_alphanumeric': 0.06445993031358885, 'fraction_numerical': 0.006533101045296167, 'mean_word_length': 4.366822429906542, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3551330', 'n_tokens_mistral': 717, 'n_tokens_neox': 686, 'n_words': 352}